Info about skipping of tests aren't writing due to retryAnalyzer at test-methods about agent-java-testng HOT 12 CLOSED

from agent-java-testng.

@DzmitryHumianiuk @avarabyeu @pbortnik Is there any progress on this issue? It is affecting our operations to the extent that there is a dilemma: either retries or ReportPortal. We can't use both retries and Report Portal during test execution at the same time. Please help!

from agent-java-testng.

@DzmitryHumianiuk @avarabyeu @pbortnik I also wait for any updates of this issue

from agent-java-testng.

@Kanaduchi Unfortunately, we cannot reproduce it.
I've pushed your example at our example repository, could you please check it there? The only difference is the latest version of integration. Please, let us know whether it works for you

from agent-java-testng.

@Brankursine i see you closed item.
and deleted example.

it's not actual anymore?

from agent-java-testng.

Sorry for the delay in my response.

Just created simple project (IntelliJ IDEA: pom + suite + test) which reproduce the problem.
ReproduceReTryFail.zip

command line:
test -Dsurefire.suiteXmlFiles=suites/ReproduceItemId.xml -Drp.enable=true -Drp.endpoint=https://******.com -Drp.uuid=******* -Drp.project=platform -Drp.launch=Debug -Drp.tags=debug -Drp.mode=debug

logs:

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building ReproduceReTry 2018.3.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ pj-reproduce ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\Pj__IJ\ReproduceReTryFail\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ pj-reproduce ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 2 source files to C:\Pj__IJ\ReproduceReTryFail\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ pj-reproduce ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 1 resource
[INFO] Copying 1 resource to suites
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ pj-reproduce ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file to C:\Pj__IJ\ReproduceReTryFail\target\test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ pj-reproduce ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2018-08-14 17:29:31,270 [INFO ][TestNG-tests-1 ][ ReproduceItemIdTest] - bcReproduceItemIdTest
java.lang.IllegalArgumentException: ItemID should not be null
at rp.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
at com.epam.reportportal.service.LaunchImpl.finishTestItem(LaunchImpl.java:230)
at com.epam.reportportal.testng.TestNGService.finishTestMethod(TestNGService.java:157)
at com.epam.reportportal.testng.BaseTestNGListener.onTestSkipped(BaseTestNGListener.java:103)
at org.testng.internal.Invoker.runTestListeners(Invoker.java:1723)
at org.testng.internal.Invoker.runTestListeners(Invoker.java:1714)
at org.testng.internal.Invoker.registerSkippedTestResult(Invoker.java:1272)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:605)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:869)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1193)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:126)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.privateRun(TestRunner.java:744)
at org.testng.TestRunner.run(TestRunner.java:602)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:380)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:39)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:414)
at org.testng.internal.thread.ThreadUtil$1.call(ThreadUtil.java:52)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 4.377 s <<< FAILURE! - in TestSuite
[ERROR] bcReproduceItemIdTest(com.mysite.ReproduceItemIdTest) Time elapsed: 1.8 s <<< FAILURE!
java.lang.AssertionError: expected [true] but found [false]
at org.testng.Assert.fail(Assert.java:93)
at org.testng.Assert.failNotEquals(Assert.java:512)
at org.testng.Assert.assertTrue(Assert.java:41)
at org.testng.Assert.assertTrue(Assert.java:51)
at com.mysite.ReproduceItemIdTest.bcReproduceItemIdTest(ReproduceItemIdTest.java:15)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:523)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:224)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:146)
at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:166)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:105)
at org.testng.TestRunner.privateRun(TestRunner.java:744)
at org.testng.TestRunner.run(TestRunner.java:602)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:380)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:39)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:414)
at org.testng.internal.thread.ThreadUtil$1.call(ThreadUtil.java:52)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ReproduceItemIdTest.bcReproduceItemIdTest:15 expected [true] but found [false]
[INFO]
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 1
[INFO]
[ERROR] There are test failures.

Please refer to C:\Pj__IJ\ReproduceReTryFail\target\surefire-reports for the individual test results.
Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.624 s
[INFO] Finished at: 2018-08-14T17:29:34+05:00
[INFO] Final Memory: 18M/259M
[INFO] ------------------------------------------------------------------------

Process finished with exit code 0

from agent-java-testng.

@DzmitryHumianiuk close it by mistake (
Issue is still actual.
Just did some little notes above about reproduce.

Consumer is:

Current version: Analysis Service: 4.2.0; API Service: 4.2.1; Index Service: 4.2.0; Jira Service: 4.2.0; Authorization Service: 4.2.0; Service UI: 4.2.2;

from agent-java-testng.

I can easily reproduce the problem with example provided by @Brankursine

from agent-java-testng.

@avarabyeu and @pbortnik will take a look during this week

from agent-java-testng.

will take a look during this week
@DzmitryHumianiuk Any updates on this?

from agent-java-testng.

@Brankursine @Kanaduchi @ogonekorama
Guys, seems like i fixed the problem.
Could you generate a jar file from master branch and try it with your project ?

To use generated jar file I had to add additionalClasspathElements with the path to the .jar file to maven surefire plugin configs (do not forget to remove maven dependency for the testng agent in your pom.xml):

      <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.21.0</version>
            <configuration>
                <additionalClasspathElements>
                    D:\rp_sandbox\agent-java-testng-4.2.2-SNAPSHOT_3_4_2019.jar
                </additionalClasspathElements>
                <testFailureIgnore>true</testFailureIgnore>
                <trimStackTrace>false</trimStackTrace>
                <useFile>false</useFile>
            </configuration>
        </plugin>

from agent-java-testng.

@Brankursine plz check

from agent-java-testng.