Automate posting Host OS build test results about tests HOT 9 CLOSED

@rquander @rmatinata-ibm @olavphibm @maurorodrigues @bssrikanth I am not sure, how much relevance weekly build results to community, we can discuss in detail internally, results for every release say 1.0, 1.5 make sense?

from tests.

I agree that this requires additional discussion and will likely also require more code to implement properly. I am shifting this feature to the December 31st delivery milestone.

from tests.

After further discussion, we believe full detailed test results for each build will not be used by most and is likely wasted effort.

Instead, a more concise, brief test results overview should be posted for each updated build that includes info such as:

1. Which specific test suites were run (host_sanity, guest_sanity, etc.).
2. How long the entire test run took to complete.
3. The resulting percent pass/fail.

This info will likely be included in the Weekly Build Issues of the Host OS builds repository.

After implementing the above strategy, new Issues will be opened to track additional enhancements to this feature (more detailed failure info if failures occur, etc.).

from tests.

I am currently working on automating the announcement of release notes and creating pages for them in https://github.com/open-power-host-os/open-power-host-os.github.io (see open-power-host-os/builds#43). Maybe that'll be a good place to publish the test results.

from tests.

@rquander @sathnaga
We now have automated release notes creation in https://open-power-host-os.github.io/
If you decide to post the test results there too, we can work together on the format to display the release notes together with tests.

from tests.

The method to automate posting test results is clear, but we are still investigating which specific conditions must be met before automatic test results are posted. Since internal testing is performed on builds that may not be released to GitHub, we must avoid posting test results for a build that does not exist from a GitHub user's standpoint. Internal Continuous Integration jobs will need adjustment and testing before automatic test results posting is enabled.

from tests.

@mopsfelder @olavphibm @sathnaga
We need to discuss this item further in the new year to determine the best way to combine the continuous integration test results with the build release notes. I believe we can integrate building and testing into a more tightly integrated job and create a single PR for release notes when necessary.

from tests.

I think integrating test results into the release notes would be ideal. Let us wait for the new year to discuss this then.

from tests.

------- Comment From [email protected] 2018-08-23 15:07:49 EDT-------
Assigning to Murilo.

from tests.