CP-2135 Coverage: fail (but still complete) task if one or more test suites fail about dart_dev HOT 6 CLOSED

Any updates on this? Being able to run just coverage instead of tests+coverage would allow us to shave 9 minutes off our CI build. Which would be really nice 😃 .

Could we do something like grep the logs for that error message and set the exit code accordingly? A little hacky, but we should be fine as long as we have solid test coverage.

from dart_dev.

@maxwellpeterson-wf We are running each of our 5 test entry points separately, since combining them results in test loading timeouts (see dart-lang/test#390).

Running in the background is a good idea; we'll give that a whirl, thanks! Hopefully that'll give us a decent speedup.

from dart_dev.

@greglittlefield-wf are you using a test entry points (unit_tests.dart, integration_tests.dart) or is each file being loaded into coverage separately? That's a big area for improvement if not.

Besides that, you could look into running coverage in the background: https://github.com/Workiva/tasker-ui/blob/master/smithy.yaml#L17

from dart_dev.

@greglittlefield-wf even if this issue was fixed, I still wouldn't recommend relying on the coverage run alone. Historically it's been too brittle and I'm not as confident in it as I am the test package.

from dart_dev.

from dart_dev.

As an update, trying to run coverage/tests at the same time or separately during other parts of the build result in slowdowns that lead to that same timeout issue :(.

from dart_dev.