Comments (7)
I can do that. I considered that to begin with, but decided not to check, with the rationale that a missing jacoco.csv may be due to a typo in a workflow file, and that letting the workflow fail would draw attention to such a bug, while simply skipping it would report a coverage percentage that may be inaccurate.
I guess I can skip missing reports, but log them.
from jacoco-badge-generator.
I found a way around it, uploading and downloading artifacts.
It makes the workflow a bit complex and I'm afraid it will raise another problem.
It also requires to call merge
action from maven-jacoco-plugin
in coverage
profile of pom.xml
.
Here is a bunch of code (sorry, it is a bit long):
name: Java CI with Maven
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
test-suite:
runs-on: ubuntu-latest
# Tests matrix
strategy:
#fail-fast: false
matrix:
group: [ "gr1" , "gr2", "gr3" ]
# The different steps
steps:
- uses: actions/checkout@v2
- name: Set up JDK 11
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
# Regression tests
- name: Test ${{ matrix.group }}
run: mvn -B --file pom.xml package -Pcoverage -DtestFailureIgnore=true -Dgroups=${{ matrix.group }}
# upload jacoco-csv files
- name: Prepare upload
run: mkdir -p jacoco/
- run: cp modul1/target/site/jacoco/jacoco.csv jacoco/mod1.csv
- run: cp module2/target/site/jacoco/jacoco.csv jacoco/mod2.csv
if: ${{ always() }}
- name: Upload jacoco-csv files
if: ${{ always() }}
uses: actions/upload-artifact@master
with:
name: ${{ matrix.group }}
path: |
jacoco/mod1.csv
jacoco/mod2.csv
if-no-files-found: ignore
coverage:
needs: [ test-suite ]
runs-on: ubuntu-latest
steps:
# Download jacoco-csv files
- uses: actions/download-artifact@v2
name: Download jacoco-csv files
with:
path: jacoco/
- name: Generate JaCoCo Badge
id: jacoco
uses: cicirello/[email protected]
with:
generate-branches-badge: true
jacoco-csv-file: >
jacoco/gr1/mod1.csv
jacoco/gr1/mod2.csv
jacoco/gr2/mod1.csv
# jacoco/gr2/mod2.csv does not exist
# jacoco/gr3/mod1.csv does not exist
jacoco/gr3/mod2.csv
- name: Log coverage percentage
run: |
echo "coverage = ${{ steps.jacoco.outputs.coverage }}"
echo "branch coverage = ${{ steps.jacoco.outputs.branches }}"
That worked but problem is that I'm sure additions JacocoBadgeGenerator.py#L94 reflect the union expected.
from jacoco-badge-generator.
I'm going to implement a fix either way.
But what causes a jacoco report to generate sometimes but not other times? The only reason I can think of is if one or more tests fail. But wouldn't test case failures cause the build itself to fail? And if not, and if the missing reports are for the modules with test case failures, then computing coverage skipping missing reports may give you inaccurate results.
Here is what I'm thinking of doing:
- If a report is missing, log a warning to standard out, so you can see which were missing in the workflow output.
- Compute coverage from whatever reports are there.
- Add an input to give the user of the action control over whether a badge is generated if one or more expected reports are missing.
- Also give control to user of action whether or not to fail the workflow if expected report is missing. Perhaps a boolean input ´fail-on-warning´.
Numbers 3 and 4 above would essentially enable user to decide how to handle missing reports: fail workflow, or don't fail workflow but also don't generate badges, or generate badges anyway. Either way the names of the missing reports would be found in the workflow output.
Perhaps one input might cover both. Maybe something like ´on-warning´ with possible values of ´fail´ to fail the run, ´quiet´ to not fail the run but also not generate badges, and ´badges´ to generate badges despite missing reports.
from jacoco-badge-generator.
Sorry I wasn't clear at all.
Short version
I doubt that the simple sum of the indicators read from the csv files represents the true code coverage in my case.
Long version
As you can see, I have a multi-module maven project and my configuration is based on a matrix.
Each element of the matrix represents a group of tests (via TestNG). The execution of a group covers part of the code between the 2 modules and produces (at most) 2 jacoco-csv
file, one for each module. Then all files are collected and passed to the action which sums percentages.
In my case, it happens that two test suites cover the same class from a module. But since the rows covered are not specified in each csv file, the sum of the indicators does not represent the union of coverage. Indeed, it can be equal to 150%.
Or there is something I missed.
For instance, here are the indicators for the class PropMinBC
from 2 files, each relating to a test suite.
// First test suite
choco-solver,org.chocosolver.solver.constraints.ternary,PropMinBC,31,295,7,21,5,27,8,11,1,4
// second test suite
choco-solver,org.chocosolver.solver.constraints.ternary,PropMinBC,115,211,17,11,12,20,11,8,2,3
The first one covers 295/326 instructions, the second 211/326 instructions.
Sum the values is not correct (506 >> 326), neither is averaging (253) nor take the maximum (295).
The appropriate approach would be to merge result (based on the binaries exec
I suppose).
A workaround would be to avoid using the matrix strategy, but tests require too much time (> 1h).
from jacoco-badge-generator.
Oh I see now. Computing coverage by combining the data in the 2 csv files definitely won't be correct in your case due to the overlap. I believe it would give you the equivalent of an average or at least it would if both are based on the exact same lines of code, since it assumes the reports are independent. The 326 lines of code in each is just assumed different, interpreting it as 652 lines of code. But an average doesn't tell you anything particularly meaningful.
There is also not enough info in either the csv or the xml reports to properly merge the results. Probably the best approach is to use jacoco:merge and let jacoco take care of merging into one report.
from jacoco-badge-generator.
I created a GitHub action jacoco-merge which should be combined with a JaCoCo report.
I let you fix/close this issue but I don't think you should make too much of it either
from jacoco-badge-generator.
@cprudhom You might also take a look at jacoco:report-aggregate. I'm not sure if it applies to your case or not.
from jacoco-badge-generator.
Related Issues (20)
- Add major release tag as recommended in actions docs
- Ability to customize the coverage intervals for the colors
- Customize set of colors
- Add example with all possible inputs to documentation
- Option to generate Shields.io JSON endpoints instead of SVG badges
- Feature Req: add exact coverage value, with virtually infinite precision into some hidden field of JSON and SVG for future comparisons HOT 10
- Feature Req: Add documentation to execute without github actions or docker HOT 2
- Feature Req: Change coverage badge text HOT 2
- Feature Req: How to use directly without github action? HOT 2
- BUG: An extra __pycache__ file with root permission will break the workflow HOT 3
- Feature request: absolute path for jacoco-csv-file HOT 5
- Sample badges not showing in GitHub Marketplace listing
- Refactor main block to extract a main function
- Add alternate entry point for running as command-line utility outside of GitHub Actions
- Workflow to publish package to PyPi during release for CLI usage
- Question: Badges was not generated HOT 3
- Feature Req: Generate badges from JaCoCo reports in xml format HOT 7
- Replace the use of GitHub Action's deprecated set-output HOT 7
- Output a GitHub Actions workflow job summary
- Replacement for set-output may break some self-hosted cases
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jacoco-badge-generator.