Number of tests is misreported on codebetter CI about fakeiteasy HOT 8 CLOSED

It seems to be something to do with TeamCity's auto detection of test execution. The build logs indicate that all 928 unit tests are being run plus 52 integration tests, as well as the specs.

from fakeiteasy.

Renamed from 'Not all tests are running on codebetter CI' to 'Number of tests is misreported on codebetter CI'

from fakeiteasy.

@philippdolder, @adamralph
check it out

from fakeiteasy.

woot! 👏 what did you do?

from fakeiteasy.

Added a build feature - XML reports. See on
http://teamcity.codebetter.com/admin/editBuildRunners.html?id=buildType:bt1255

toward the bottom, there.

I guess it picks up the Spec tests from the output, but can't for NUnit. If we'd used the TeamCity NUnit runner, it would've Just Worked(TM.

from fakeiteasy.

Great! Let's get it into the official build config. I don't want to use the TeamCity NUnit runner since then the CI server build diverges from a local rake.

Interestingly, with xunit.net tests, this isn't required. It Just Works. I think the xunit.net runner has some special magic inside it to detect that it's running in the context of TeamCity.

from fakeiteasy.

XUnit may. I suppose an alternative could be to hook up a custom listener to NUnit that would emit the correct... I forget what they're called "notes to TeamCity", but only when running under TeamCity.

I wonder if we'd used the /labels flag for NUnit if the test would've been detected.

I can experiment with either of those approaches if you like. But I think the thing I did today is pretty good. If you're okay with it, I'll move it to the official configuration (or anyone can) later today.

from fakeiteasy.

Done!

I think this fix is absolutely fine. No point spending more time on it.

At some point though, personally, I'd like to switch to xunit.net 😉.

from fakeiteasy.