fabriciorby / maven-surefire-junit5-tree-reporter Goto Github PK
View Code? Open in Web Editor NEWTree view console report for Maven Surefire JUnit5 plugin. :^)
License: Apache License 2.0
Tree view console report for Maven Surefire JUnit5 plugin. :^)
License: Apache License 2.0
What different report formats exist and what does it influence?
Just had a look in my Surefire reports output directory and spotted the following:
target/surefire-reports/2022-12-30T14-01-27_616-jvmRun1.dumpstream
# Created at 2022-12-30T14:01:38.848
ForkStarter IOException: Index: 1, Size: 1.
org.apache.maven.plugin.surefire.booterclient.output.MultipleFailureException: Index: 1, Size: 1
at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.<init>(ThreadedStreamConsumer.java:64)
at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer.<init>(ThreadedStreamConsumer.java:122)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:600)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:311)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:268)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1334)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1167)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:931)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:370)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:210)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:195)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1589)
Suppressed: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.base/java.util.Collections$SingletonList.get(Collections.java:4959)
at org.apache.maven.plugin.surefire.report.TreePrinter$TestPrinter.printClass(TreePrinter.java:162)
at org.apache.maven.plugin.surefire.report.TreePrinter$TestPrinter.printTest(TreePrinter.java:96)
at org.apache.maven.plugin.surefire.report.TreePrinter$TestPrinter.access$500(TreePrinter.java:85)
at org.apache.maven.plugin.surefire.report.TreePrinter.lambda$printTests$1(TreePrinter.java:82)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
at org.apache.maven.plugin.surefire.report.TreePrinter.printTests(TreePrinter.java:82)
at org.apache.maven.plugin.surefire.report.TestReportHandler.printTests(TestReportHandler.java:111)
at org.apache.maven.plugin.surefire.report.TestReportHandler.print(TestReportHandler.java:44)
at org.apache.maven.plugin.surefire.report.ConsoleTreeReporterBase.testSetCompleted(ConsoleTreeReporterBase.java:26)
at org.apache.maven.plugin.surefire.report.ConsoleTreeReporterAscii.testSetCompleted(ConsoleTreeReporterAscii.java:32)
at org.apache.maven.plugin.surefire.report.ConsoleTreeReporterBase.testSetCompleted(ConsoleTreeReporterBase.java:8)
at org.apache.maven.plugin.surefire.report.TestSetRunListener.testSetCompleted(TestSetRunListener.java:213)
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$TestSetCompletedListener.handle(ForkClient.java:147)
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$TestSetCompletedListener.handle(ForkClient.java:135)
at org.apache.maven.plugin.surefire.booterclient.output.ForkedProcessEventNotifier.notifyEvent(ForkedProcessEventNotifier.java:233)
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.handleEvent(ForkClient.java:353)
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.handleEvent(ForkClient.java:59)
at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:96)
... 1 more
Tests appear to still be running, though. I don't know how serious this is or whether it is anything to worry about or not.
Other info:
Invocation: $ ./mvnw clean verify -T8
$ uname -a
Linux z170x-fedora 6.0.12-300.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Dec 8 16:58:47 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
$ java -version
java -version
openjdk version "19.0.1" 2022-10-18
OpenJDK Runtime Environment Corretto-19.0.1.10.1 (build 19.0.1+10-FR)
OpenJDK 64-Bit Server VM Corretto-19.0.1.10.1 (build 19.0.1+10-FR, mixed mode, sharing)
$ ./mvnw --version
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /home/ashley/.m2/wrapper/dists/apache-maven-3.8.6-bin/1ks0nkde5v1pk9vtc31i9d0lcd/apache-maven-3.8.6
Java version: 19.0.1, vendor: Amazon.com Inc., runtime: /home/ashley/.sdkman/candidates/java/19-amzn
Default locale: en_GB, platform encoding: UTF-8
OS name: "linux", version: "6.0.12-300.fc37.x86_64", arch: "amd64", family: "unix"
<properties>
<argLine>
-Dorg.slf4j.simpleLogger.log=INFO
-Xshare:off
</argLine>
<maven-surefire-plugin.version>3.0.0-M7</maven-surefire-plugin.version>
<maven-surefire-junit5-tree-reporter.version>1.1.0</maven-surefire-junit5-tree-reporter.version>
</properties>
<plugin>
<!-- Unit testing config -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<excludes>
<exclude>*IntegrationTest</exclude>
</excludes>
<failIfNoTests>true</failIfNoTests>
<runOrder>random</runOrder>
<tempDir>./tmp/unit</tempDir>
<useModulePath>true</useModulePath>
<!--
This block is needed to show @DisplayName and @ParameterizedTest
in reports with the provided names.
-->
<statelessTestsetReporter
implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
<disable>false</disable>
<version>3.0</version>
<usePhrasedFileName>false</usePhrasedFileName>
<usePhrasedTestSuiteClassName>true</usePhrasedTestSuiteClassName>
<usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
<usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
</statelessTestsetReporter>
<consoleOutputReporter>
<disable>${hide-test-logs-in-console}</disable>
</consoleOutputReporter>
<statelessTestsetInfoReporter
implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter">
<disable>false</disable>
<usePhrasedFileName>false</usePhrasedFileName>
<usePhrasedClassNameInRunning>true</usePhrasedClassNameInRunning>
<usePhrasedClassNameInTestCaseSummary>true</usePhrasedClassNameInTestCaseSummary>
</statelessTestsetInfoReporter>
</configuration>
<dependencies>
<dependency>
<groupId>me.fabriciorby</groupId>
<artifactId>maven-surefire-junit5-tree-reporter</artifactId>
<version>${maven-surefire-junit5-tree-reporter.version}</version>
</dependency>
</dependencies>
</plugin>
src/test/resources/junit-platform.properties
junit.jupiter.execution.parallel.enabled=true
I am working with the release 1.2.1
.
It works fine for the @DisplayName
outputs, but not for a TestAbortedException
message, it when an assumption fails. I am not sure if is a missing or incomplete configuration, for more details see:
I read the maven-surefire-junit5-tree-reporter page, but it seems is not possible reach this goal because this support does not exist yet
Thank You
When building multithreaded the logs come out piecemeal and it doesn't read easily. Is it possible to buffer the report and send it all in one go?
Hey, great project!
I have a parameterized test as follows:
@RunWith(JUnitParamsRunner.class)
public class SomeTest {
@Test
@Parameters(method = "values")
public void shouldBeOk(List<Integer> leaderboards, Integer count) {
assertThat(leaderboards.size(), is(count));
}
private Object[] values() {
return new Object[]{
new Object[]{List.of(), 0},
new Object[]{List.of(1), 1},
};
}
}
which looks like:
[INFO] ββ shouldBeOk([], 0) [0] - 0.001s
[INFO] β ββ β null - 0s
[INFO] β ββ β null - 0.001s
Since Surefire 3.1.1 the formatting of the elapsed time comes already with the s
appended.
However, the TreePrinter appends another s
, leading to output like this:
β¦
[INFO] β ββ β validDataUrlIsParsed(String)[1] image/png - 0.006 ss
[INFO] β ββ β validDataUrlIsParsed(String)[2] image/jpeg - 0.001 ss
[INFO] β ββ β validDataUrlIsParsed(String)[3] image/jpg - 0.001 ss
β¦
All @before an @after annotations are not displayed as part of the report
in the original report you can see the time elapsed for the entire class, but in the tree report you just see the elapsed time per method.
e,g:
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 s - in org.example.MyTest
if for example, you have a @before method in your class which takes few seconds, you will not be able to capture that
Reproducer: https://github.com/rkraneis/tree-reporter
Context: https://github.com/rkraneis/tree-reporter/blob/2a656ebb9e4c5c39371b5fd4e849c326641437fb/README.md
Actual:
ββ io.github.rkraneis.treereporter.OuterTest - 0.036s
β ββ β test - 0.011s
ββ ββ InnerSub2 - 0.001s
β ββ β testFromBase - 0s
β ββ β testInnerSub2 - 0.001s
Expected:
ββ io.github.rkraneis.treereporter.OuterTest - 0.036s
β ββ β test - 0.011s
ββ ββ Inner1 - 0.001s
β ββ β testInner1 - 0.001s
ββ ββ Inner2 - 0.001s
β ββ β testInner2 - 0.001s
ββ ββ InnerSub1 - 0.001s
β ββ β testFromBase - 0s
β ββ β testInnerSub1 - 0.001s
ββ ββ InnerSub2 - 0.001s
β ββ β testFromBase - 0s
β ββ β testInnerSub2 - 0.001s
The tests themselves are executed (Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
).
N.B.: I started my own tree view printer a while ago (basically just as a copy of what the junit jupiter console launcher does), but didn't like the messy setup (test execution listener). Your POM based version would be much better, setup-wise.
Running in parallel appears to trigger an IllegalStateException when cloning the reporter object.
<runOrder>random</runOrder>
<reportFormat>plain</reportFormat>
<consoleOutputReporter>
<disable>true</disable>
</consoleOutputReporter>
<statelessTestsetInfoReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode">
</statelessTestsetInfoReporter>
./mvnw --debug -T4C clean package -DforkCount=4 -Dparallel=all 2>&1 | tee output.log
Debug output seems to indicate the following is the root of the error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test (default-test) on project my-project: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test failed: org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test (default-test) on project my-project: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test failed: org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:375)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
at java.lang.Thread.run (Thread.java:829)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test failed: org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
at java.lang.Thread.run (Thread.java:829)
Caused by: java.lang.IllegalStateException: org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode
at org.apache.maven.plugin.surefire.extensions.SurefireStatelessTestsetInfoReporter.clone (SurefireStatelessTestsetInfoReporter.java:78)
at org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoReporter.clone (JUnit5StatelessTestsetInfoReporter.java:120)
at org.apache.maven.plugin.surefire.CommonReflector.createStartupReportConfiguration (CommonReflector.java:96)
at org.apache.maven.plugin.surefire.CommonReflector.createReportingReporterFactory (CommonReflector.java:73)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.getSuitesIterator (ForkStarter.java:746)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkOnceMultiple (ForkStarter.java:351)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run (ForkStarter.java:326)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run (ForkStarter.java:269)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider (AbstractSurefireMojo.java:1334)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked (AbstractSurefireMojo.java:1167)
at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute (AbstractSurefireMojo.java:931)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
at java.lang.Thread.run (Thread.java:829)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :my-project
Is there a workaround for this? If any other info is needed, please let me know! :-)
Reproducer: https://github.com/tmarback/tree-reporter-issue
When using nested tests, the "bars" that draw the test tree are incorrect, with the horizontal lines for the test classes originating a layer above where they should:
(note how for the 1st-level nested classes the horizontal line crosses 2 different vertical lines, while on the 2nd-level nested classes the horizonal line goes past the vertical line)
PS <snip>\tree-reporter> mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------< test:tree-reporter >-------------------------
[INFO] Building tree-reporter 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
<snip>
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] +--Outer - 0.056s
[INFO] | +-- [OK] testOuter1 - 0.016s
[INFO] | +-- [OK] testOuter2 - 0s
[INFO] | '-- [OK] testOuter3 - 0s
[INFO] +--.--Outer Inner 2 - 0.005s
[INFO] | | +-- [OK] testInner1 - 0.001s
[INFO] | | +-- [OK] testInner2 - 0.001s
[INFO] | | '-- [OK] testInner3 - 0.001s
[INFO] +--.--Outer Inner 3 - 0.005s
[INFO] | | +-- [OK] testInner1 - 0s
[INFO] | | +-- [OK] testInner2 - 0.001s
[INFO] | | '-- [OK] testInner3 - 0s
[INFO] | '--.--Outer Inner 3 Inner-er 2 - 0.013s
[INFO] | | +-- [OK] testInnerer1 - 0.001s
[INFO] | | +-- [OK] testInnerer2 - 0s
[INFO] | | '-- [OK] testInnerer3 - 0s
[INFO] | '--.--Outer Inner 3 Inner-er 3 - 0.013s
[INFO] | | +-- [OK] testInnerer1 - 0s
[INFO] | | +-- [OK] testInnerer2 - 0s
[INFO] | | '-- [OK] testInnerer3 - 0.001s
[INFO] | '--.--Outer Inner 3 Inner-er 1 - 0.013s
[INFO] | | +-- [OK] testInnerer1 - 0s
[INFO] | | +-- [OK] testInnerer2 - 0.001s
[INFO] | | '-- [OK] testInnerer3 - 0s
[INFO] +-----Outer Inner 1 - 0.005s
[INFO] | +-- [OK] testInner1 - 0s
[INFO] | +-- [OK] testInner2 - 0.001s
[INFO] | '-- [OK] testInner3 - 0s
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.787 s
[INFO] Finished at: 2023-05-18T03:39:04-07:00
[INFO] ------------------------------------------------------------------------
Also the @DisplayName
names seem to be getting appended at each level, which from the examples doesn't seem to be the expected.
We have a project with 700+ tests and on our CI we noticed multiple times OOME errors (sadly we weren't able to get a dump) when our tests are failing.
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "surefire-forkedjvm-command-thread"
failsafe / surefire are configured with
<!-- Custom reports -->
<reportFormat>plain</reportFormat>
<consoleOutputReporter>
<disable>true</disable>
</consoleOutputReporter>
<statelessTestsetInfoReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter"/>
and JUnit with
junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.mode.default=same_thread
junit.jupiter.execution.parallel.mode.classes.default=concurrent
To display the tests results as tree I suppose the reporting is keeping in memory many informations due to the execution of tests in parallel (I see several statics fileds in https://github.com/fabriciorby/maven-surefire-junit5-tree-reporter/blob/master/src/main/java/org/apache/maven/plugin/surefire/report/TestReportHandler.java#L12-L15)
If there are several failures and the memory is limited like in a CI system a surefire JVM can die with an OOME because of the TestReportHandler size. We could probably increase the tests JVMs size but it's a bit annoying to do it just for the tests reporter.
Thanks for sharing.
It's awesome!
I've been looking for such a solution for a long time.
Although the tree reporter indicates which tests are skipped by colour and the text "is @disabled" the log line is still at INFO level.
This is fine though, as the surefire summary line, e.g.
[WARNING] Tests run: 22, Failures: 0, Errors: 0, Skipped: 4
is enough to indicate a problem (where log output is set to filter out INFO logs)
Ticket made as a reference for future.
BTW, the time reported is still not correct because of a reverse order of events the surefire plugin is sending to the ReportConsole
here's an example:
public class NestedWaitingTest {
@Test
public void outside() throws InterruptedException {
Thread.sleep(1000);
}
@Nested
class Inside1Test {
@Test
public void inside1() throws InterruptedException {
System.out.println("Waiting for 3 seconds");
Thread.sleep(3000);
}
@Test
public void inside1fast() {
}
@Nested
class Inside2Test {
@Test
public void inside2fast() {
}
}
}
}
you get this report (without plugin)
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 s - in NestedWaitingTest$Inside1Test$Inside2Test
[INFO] NestedWaitingTest.outside Time elapsed: 1.019 s
[INFO] NestedWaitingTest$Inside1Test.inside1fast Time elapsed: 0.001 s
[INFO] NestedWaitingTest$Inside1Test.inside1 Time elapsed: 3.007 s
[INFO] NestedWaitingTest$Inside1Test$Inside2Test.inside2fast Time elapsed: 0.001 s
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.021 s - in NestedWaitingTest$Inside1Test
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.053 s - in NestedWaitingTest
this translate into 3 events coming into ConsoleReporter
1st one you use today to print the entire data (includes 4 entires in the TreeSetStats) - but uses the elapsed time of the inner*2 test class
2nd and 3rd event are coming without any entries in the TreeSetStats but with correct elapsed time information
so if you want to be consistent with the elapsed time, you need to implement a different approach
Originally posted by @avishayh in #21 (comment)
In pom.xml we see references to Java 8: https://github.com/fabriciorby/maven-surefire-junit5-tree-reporter/blob/master/pom.xml#L18
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
But in github workflow, we see a reference to Java 11: https://github.com/fabriciorby/maven-surefire-junit5-tree-reporter/blob/master/.github/workflows/maven-publish.yml#L16
- name: Set up JDK 11
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '11'
I suggest aligning these versions.
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] |
[INFO] +-- Nested Sample
[INFO] |
[INFO] +-- Inner Test
[INFO] |
[INFO] +-- Inner Inner Test
[INFO] |
[INFO] +-- Inner Inner Inner Test
[INFO] | +-- [OK] Should pass - 0.036s
[INFO] | +-- [OK] Should pass2 - 0.002s
[INFO] | +-- [OK] Inner test should pass - 0.002s
[INFO] | +-- [OK] Inner test should pass2 - 0.001s
[INFO] | +-- [OK] Inner Inner Test should pass - 0.002s
[INFO] | +-- [OK] Inner Inner Inner Test should pass - 0.002s
I can confirm the plugin is now (v1.0.0) buffering the output by class.
Would it be possible to buffer the output by plugin execution?
Or is this something Maven should be handling?
For example, currently a log might have a line
[INFO] --- maven-jar-plugin:3.2.2:jar (default-jar) @ wwgcommon ---
followed by
[INFO] | +-- [OK] testUnpaidImport - 0.179s
[INFO] | '-- [OK] testReplyImport - 0.002s
I have Test with 2 @nested class in same level.
After run test only one nested test class logged in result.
Hey,
in case of following error:
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.004 s <<< FAILURE! - in com.faceit.hubs.model.entities.competition.SomeTest
[ERROR] initializationError Time elapsed: 0.004 s <<< ERROR!
org.junit.runners.model.InvalidTestClassError:
Invalid test class 'com.faceit.hubs.model.entities.competition.SomeTest':
1. Method shouldBeOk should have no parameters
the output looks as follows:
[INFO] ββ initializationError - 0.003s
[INFO] β ββ β null - 0.001s
Shows error name instead of test name.
Just checked this thing out and love the output, but when I use a parallel test setup I get a warning.
Steps to reproduce:
Configure your maven-surefire-plugin with
<properties>
<configurationParameters>
# PARALELIZATION FOR SUREFIRE ONLY, NOT FOR FAILSAFE
junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.config.strategy=dynamic
junit.jupiter.execution.parallel.config.dynamic.factor=1
junit.jupiter.execution.parallel.mode.default = same_thread
junit.jupiter.execution.parallel.mode.classes.default = concurrent
</configurationParameters>
</properties>
Run your tests
Expected: Just the nice output
Result: The nice output and
WARNING] ForkStarter IOException: Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 2, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 3, Size: 1. See the dump file ***/target/surefire-reports/2022-10-24T22-19-57_613-jvmRun1.dumpstream
Dump file attached.
2022-10-24T22-16-54_224-jvmRun1.dumpstream.zip
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.