Disable comparisons to earlier test runs about publish-unit-test-result-action HOT 4 CLOSED

Hi, thanks for using this action!

I can see that this is a valid situation where you don't want to compare to earlier commits as numbers become meaningless. That should be easy to add.

However, especially in your situation of partial testing, identifying tests that are accidentally removed or skipped becomes crucial. I like the idea of some kind of baseline. What I could think of is to fetch the test result files from the target branch (the commit we compare PRs against) and then have the unit tests overwrite them. This way, the action could compare results as is. Except for where suites are being removed, it should be able to identify all changes.

from publish-unit-test-result-action.

I can see that this is a valid situation where you don't want to compare to earlier commits as numbers become meaningless. That should be easy to add.

That would be great. I have a simple implementation for just the check bit in my fork at diem@3261e64 -- I haven't thought too much about a good UI but wanted to share this.

However, especially in your situation of partial testing, identifying tests that are accidentally removed or skipped becomes crucial. I like the idea of some kind of baseline. What I could think of is to fetch the test result files from the target branch (the commit we compare PRs against) and then have the unit tests overwrite them.

Oh that sounds like an interesting plan: kind of maintaining the baseline state and overwriting it incrementally as necessary. I'm not sure if we can put that specific plan into practice though: if we rely on artifacts, this would add a serial execution dependency between runs across commits that I'd rather avoid for now.

Regardless, I'm coming to realize that we'll need to make some fairly extensive customizations to the action to meet our needs, so I may end up building this out in a way that works for us. I think the way we'll probably do it is publish a baseline artifact with a list of tests in the same JUnit format, then use that as the input to perform the diff against. Happy to talk about upstreaming that implementation once it's done, or just maintaining a patchset of our changes ourselves.

from publish-unit-test-result-action.

I have merged the configuration bit to master, please give it a try:

uses: EnricoMi/publish-unit-test-result-action/composite@master
with:
  compare_to_earlier_commit: false

I'd be happy to see how you have solved the baseline bit. Just open a new issue and we can talk about your use case and experience there. Thanks for raising this.

from publish-unit-test-result-action.

This has been released and is now available via @v1.

from publish-unit-test-result-action.