CI is failing (timing out) on Julia nightly about tullio.jl HOT 4 CLOSED

I like this suggestion. I saw something similar for a package tested on a server using GitLab runners, great that it seems relatively easy to setup now with GitHub CI. I think it'd be a good idea for some of my own packages as well.

But I think the 1 hour -> 6 hour regression sounds like something to investigate.

I see it prints some times:

 Test Summary:          | Pass  Broken  Total
parsing all the things |  244       4    248
Test Summary:        | Pass  Broken  Total
tests from Einsum.jl |  684       5    689
[ Info: Basic tests took 106.6 seconds
Test Summary:   | Pass  Total
internal pieces |  408    408
Test Summary:         | Pass  Total
matrix multiplication |   33     33
Test Summary:                  | Pass  Broken  Total
gradients: Tracker + DiffRules |   62       1     63
Test Summary:                    | Pass  Broken  Total
gradients: Tracker + ForwardDiff |   37       1     38
[ Info: Tracker tests took 197.9 seconds
Error: The operation was canceled.

But these printed times don't come close to adding up to the 6 hours it took before getting cancelled.

from tullio.jl.

I can't believe I have a package whose tests take hours, how did this happen?

But hopefully not really 6 hours. The next set of tests after Tracker is for KernelAbstractions, so perhaps this timeout is the same as JuliaGPU/KernelAbstractions.jl#155?

Even without timeouts, though, this split is an interesting idea. It would give you quicker feedback on which things are broken. And the implementation actually looks very simple.

from tullio.jl.

Oh, this might be the same as JuliaGPU/KernelAbstractions.jl#155!

Maybe this is the easiest first step: we create two test groups: group 1 is everything except the KernelAbstractions tests, and group 2 is only the KernelAbstractions tests.

from tullio.jl.

I think this can be closed, thanks again for #62.

from tullio.jl.