Eclipse Plugin: 6.3.2.201112031323

I've set default JVM parameters including Xmx, Xms, XX:MaxPermSize under "Preferences > TestNG > Run/Debug"

Then, under my run configurations (Run > Run Configurations) I've added different JVM parameters. These JVM parameters get added to the test execution and the original ones are left out. If I remove these local JVM parameters, then the global JVM parameters get included.

I would expect the JVM parameters to get concatenated and even cause the local ones to overwrite the global ones if both locations specifiy the same option.

I've tried to be descriptive, but let me know if you need further information.

When using org.hamcrest.MatcherAssert.assertThat with hamcrest matchers in tests, the expected/actual dialogue box displays "N/A" in both sides when a test fails. Here is an example failed test trace:

Expected: is "blah blah blah"
     but: was "yada yada yada"
    at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
    at com.mundge.mundge.SomeTests.testThis(SomeTests.java:31)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:74)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:673)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:846)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1170)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
    at org.testng.TestRunner.runWorkers(TestRunner.java:1147)
    at org.testng.TestRunner.privateRun(TestRunner.java:749)
    at org.testng.TestRunner.run(TestRunner.java:600)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:317)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:312)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:274)
    at org.testng.SuiteRunner.run(SuiteRunner.java:223)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1039)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:964)
    at org.testng.TestNG.run(TestNG.java:900)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:110)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:205)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:174)

Would you consider supporting this formatting?

I have two integration tests within a single module. Now i just select a single Check1IT class which contains dependsOnGroups via project explorer in Eclipse ( TestNG 6.3.1.20111126_1116 org.testng.eclipse.feature.group Cedric Beust).
So the problem which occures is that the plugin decides to run the integration tests of the second class before the selected class which contains the dependant group but which is NOT selected during "Run As Test NG Test"...

I have checked that with Maven 3.0.3 (failsafe-plugin 2.9) and TestNG on command as well and this will fail with a:

I use to run the Maven command line a thing like this:

  mvn ... -Dit.test=Check1IT clean verify

The following will be printed out

INFO: Executing: [56a9034f-bb9d-441e-96ab-709d879835b3, implicitlyWait {"ms":100
00}]
org.apache.maven.surefire.util.SurefireReflectionException: java.lang.reflect.In
vocationTargetException; nested exception is java.lang.reflect.InvocationTargetE
xception: null
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(
ReflectionUtils.java:164)
        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke
(ProviderFactory.java:110)
        at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(Suref
ireStarter.java:172)
        at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWh
enForked(SurefireStarter.java:104)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:
70)
Caused by: org.testng.TestNGException:
Method "PruefenDerFilterFuerZentralbearbeiterFinanzamtIT.ueberpruefeInhaltFallli
ste()[pri:0, instance:com.rfz.de.test2.zentralbearbeiterfinanzamt.PruefenDerFilt
erFuerZentralbearbeiterFinanzamtIT@5e3974]" depends on nonexistent group "modifi
ziere.daten"
        at org.testng.DependencyMap.getMethodsThatBelongTo(DependencyMap.java:29
)
        at org.testng.TestRunner.createDynamicGraph(TestRunner.java:1071)
        at org.testng.TestRunner.privateRun(TestRunner.java:724)
        at org.testng.TestRunner.run(TestRunner.java:613)
        at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
        at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
        at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
        at org.testng.SuiteRunner.run(SuiteRunner.java:240)
        at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
        at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
        at org.testng.TestNG.runSuitesSequentially(TestNG.java:1137)
        at org.testng.TestNG.runSuitesLocally(TestNG.java:1062)
        at org.testng.TestNG.run(TestNG.java:974)
        at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.ja
va:70)
        at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(Tes
tNGDirectoryTestSuite.java:108)
        at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider
.java:111)
        ... 9 more

I understand the error message...but the question which results from that:

Can i run a Test which contains a dependsOnGroups separately alone?

I'm not sure if this belongs to TestNG or to the Eclipse Plugin or both?

The dialog seems to try to be "intelligent" by refusing any method I type in the dialog. The method doesn't exist before runtime.

Also, it seems the dialog should be able to take more than one method selection.

Given the java class:

public class TestThatTakesParameters {
    @Test
    @Parameters({"parameter_one"})
    public void testMethod(@Optional("parameter_one_default") String parameter) {
        System.out.println(parameter);
    }
}

in a project, if there are no test suite XML files the test runs correctly and the parameter is defaulted.

If you add parameter_file_one.xml to the project:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="ParameterFileTwo" verbose="1" >
  <test name="All Tests">
    <parameter name="parameter_one"  value="xml_file_one_value"/>
  </test>
</suite>

The the parameters from that file will automatically be included, which is not expected because the parameter is defaulted.

If a second file parameter_file_two.xml is added:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="ParameterFileTwo" verbose="1" >
  <test name="All Tests">
    <parameter name="parameter_one"  value="xml_file_two_value"/>
  </test>
</suite>

you will not be able to run the test without selecting a parameter file, which is not expected because the parameter on the method is defaulted.

For the scenario where all the parameters are defaulted, I would expect that selecting a parameter file be optional instead of required.

I use a Factory to generate tests dynamically. The TestNG Eclipse plug-ins could keep in memory the instance the factory is returning to be run, then rerun them when I ask in the Eclipse View plug-ins. This solution should not be hard to implement.

If this cannot be done, then the factory should at least receives enough information from the plug-in to filter out when the user ask the TestNG plug-in to rerun some of the tests.

Hi,

we're using the plugin in version 6.1.1.20110728_1012. We're using snakeyaml 1.7 in our app. When running tests in eclipse (loading some test data via snakeyaml), the snakeyaml classes are not loaded from our classpath, but from the testng jar. Due to differing APIs, the test execution fails with a NoSuchMethodError.

Workaround: include the snakeyaml jar in the test classpath under bootstrap entries.

If I tests under a folder, which contains many classes, that will trigger execution of lot of tests, many for each class.

I don't see progression of execution in Eclipse, like what is the current test running, what are the test that has already run successfully or that have already failed.

We are used to have that with JUnit.

First off, I have Eclipse Helios Service Release 2 and version 6.5.2.20120616_1545 of the TestNG-Eclipse Plugin (which is the latest version according to my Eclipse) installed.
When I run a few Test-classes in Eclipse Helios and try to stop the current run via the stop-button, nothing happens, the run finishes regularly. I switched output to verbose-mode to check and no stop-signal whatsoever was being issued, it looks like the button has no function at the moment.

Hi,
I've got a message during a start of my integration test with TestNG:

"... defines group dependencies that will be ignored. To reliably test methods with group dependencies use a suite definition.
Reason:...., uses group dependencies which due to a plugin limitation will be ignored"

Is that solved in newer versions of the plugin ?

My current installation: " TestNG 6.2.0.20110826_0903 org.testng.eclipse.feature.group Cedric Beust"

Or is there a possibility to define the order of test executions for example via order in the testng.xml file ?

In JUnit there's support for Quick Fixes for missing imports. I'd love to have that for TestNG.

https://picasaweb.google.com/lh/photo/RrWuMKe3yk5C3svT1epUE_X5hsnPQvrKT7ulAA5qFiA?feat=directlink

When I run a test class with "Run As TestNG Test" it clears up the console but the tests won't start.

If I run a Suite with "Run As TestNG Suite" the same thing happens.

Ever since I upgraded the testng-eclipse plugin, I hang at 57% when debugging a test (any test) in Eclipse. The last entry in the error log appears to be:

`
eclipse.buildId=M20120208-0800
java.version=1.7.0_02
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Framework arguments: -product org.eclipse.epp.package.java.product
Command-line arguments: -os win32 -ws win32 -arch x86 -product org.eclipse.epp.package.java.product

Error
Mon Mar 12 17:44:07 EDT 2012
Error

java.net.SocketTimeoutException: Accept timed out
at java.net.TwoStacksPlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:183)
at java.net.ServerSocket.implAccept(ServerSocket.java:522)
at java.net.ServerSocket.accept(ServerSocket.java:490)
at org.testng.remote.strprotocol.BaseMessageSender.initReceiver(BaseMessageSender.java:128)
at org.testng.eclipse.ui.TestRunnerViewPart.startTestRunListening(TestRunnerViewPart.java:402)
at org.testng.eclipse.TestNGPlugin.connectTestRunner(TestNGPlugin.java:215)
at org.testng.eclipse.TestNGPlugin$2.run(TestNGPlugin.java:203)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:135)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4140)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3757)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2701)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2665)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2499)
at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:679)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:668)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:123)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:344)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:622)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:577)
at org.eclipse.equinox.launcher.Main.run(Main.java:1410)
`

I am running:

Eclipse Indigo Service Release 2 (build 20120216-1857)
testng-eclipse plugin 6.4.0.20120308_1617

I do have TestNG 6.4 declared as a Maven dependency (test scope) within my project. I read online that there can be a conflict between the project and plugin's versions of TestNG. However, both are at 6.4 (verified this in both Maven and Eclipse) for this project.

Any thoughts?

Thanks so much!

• Saish

We're using TestNG Eclipse Support 6.8.6.20130517 on Eclipse Platform 3.7.2.v201202080800.

We have several different test groups defined, and have created several run configurations for those test groups. If I bring up the Run Configurations user interface, and switch between the several different run configurations, it seems to get confused about which run configuration I'm looking at, and possibly even combines them all.

Unfortunately it's very hard to reproduce, so I can't give you exact steps yet, but if I can I will let you know. Basically, it just seems to not have a good handle on which run configuration is supposed to be doing what. I can't really explain it. Hopefully this rings a bell or something though for you!

Again, I'll provide more details if I end up being able to come up with some reproduction steps or anything else.

I would suggest to change the default configuration for the test-output directory of the testng plugin in eclipse to target/test-output in case you are in a maven project in relationship with m2eclipise. I don't know if this is possible...

We're using TestNG Eclipse Support 6.8.6.20130517 on Eclipse Platform 3.7.2.v201202080800.

We have several tests that are defined by

@Test(groups = "runtime-tests")

at the class level (not the method level), and we have run a run configuration for this group.

If I add a new class and add this annotation to the top of it, then run my configuration, this new test class does not get run.

A workaround we found in order to get the plugin to run it: I have to open the run configuration, click Browse next to the groups, click OK, then Run. It seems to cause it to refresh whatever it knows about the tests and thus include this test class.

Let me know if you need any additional details.

reposting from testng-team/testng#132

Use case:
Suppose You have the following sources in an eclipse projects:

 package packageA;
 public class ClassOne {
     @Test(dependsOnMethods = {"packageB.ClassTwo.methodB"})
     public void methodA()
    {
    }
 }

 package packageB;
 public class ClassTwo {
     @Test()
     public void methodB()
    {
    }
 }

Now, from the Eclipse project Explorer (or package explorer, etc....), suppose you select "methodA" in ClassOne, right rick and select menu item "Run As/TestNG Test". You'll get the following error:

org.testng.TestNGException: 
packageA.ClassOne.methodA() is depending on method public void packageB.ClassTwo.methodB(), which is not annotated with @Test
    at org.testng.internal.MethodHelper.findDependedUponMethods(MethodHelper.java:95)
    at org.testng.internal.MethodHelper.topologicalSort(MethodHelper.java:245)
    at org.testng.internal.MethodHelper.sortMethods(MethodHelper.java:316)
    at org.testng.internal.MethodHelper.collectAndOrderMethods(MethodHelper.java:51)
    at org.testng.TestRunner.initMethods(TestRunner.java:477)
    at org.testng.TestRunner.init(TestRunner.java:231)
    at org.testng.TestRunner.init(TestRunner.java:201)
    at org.testng.TestRunner.(TestRunner.java:157)
    at org.testng.remote.RemoteTestNG$1.newTestRunner(RemoteTestNG.java:139)
    at org.testng.remote.RemoteTestNG$DelegatingTestRunnerFactory.newTestRunner(RemoteTestNG.java:269)
    at org.testng.SuiteRunner$ProxyTestRunnerFactory.newTestRunner(SuiteRunner.java:562)
    at org.testng.SuiteRunner.init(SuiteRunner.java:157)
    at org.testng.SuiteRunner.(SuiteRunner.java:111)
    at org.testng.TestNG.createSuiteRunner(TestNG.java:1212)
    at org.testng.TestNG.createSuiteRunners(TestNG.java:1199)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1053)
    at org.testng.TestNG.run(TestNG.java:974)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:109)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:202)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:173)

You would expect instead that the plugin would resolve the dependency automatically and run before running the target method.

Note: If you set the dependency on a method that belongs to the same class (or a method defined in its super classes) then the dependency is correctly resolved (and more complex dependency chain seem to be resolve just fine).

The problem only occurs when the dependency points to a different class.

I debugged the issue to the implementation of:

org.testng.eclipse.util.JDTUtil.solveDependencies(MethodDefinition methodDef, Map parsedMethods)

I have a fix that I tested. I'll see if I can attach a patch file to this ticket.

Once my git/github environment is setup I'll propose a fix.

The TestRunner does not remember the view orientation after an Eclipse restart (although I could live with the orientation set to "automatic" as default)

I don't use the test-output directory at all, but it gets in the way of SVN and our workspace directory backup solution. So I would like to disable it completely.

Looks like it's possible to save some space in TestNG panel to leave more space for code and everything else. Take a look at this picture http://linkme.ufanet.ru/images/15d798cd469aa1cc3921b16c672d9b2a.png

The version of TestNG that I use apparently formats Assert failure exceptions like " expected [X] but found [Y]". When double clicking the result to compare, the dialog shows "N/A" "N/A". I'm not sure if this is common for anyone else, but if it is, it would be nice to see in the next update.

I added the following to CompareResultDialog.java, which should allow it to parse exceptions formatted as above:

--- CompareResultDialog.java    2012-08-01 16:02:36.000000000 -0700
+++ CompareResultDialog-new.java    2012-08-02 16:29:35.370372424 -0700
@@ -127,12 +127,19 @@ public class CompareResultDialog extends
       idxStart= trace.indexOf(firstToken);
       nextTokenString= "> but was:<";
     }
+    if(idxStart == -1) {
+      firstToken= "expected [";
+      idxStart= trace.indexOf(firstToken);
+      nextTokenString= "] but found [";
+    }

     if(idxStart != -1) {
       int idxEnd= trace.indexOf(nextTokenString); //trace.indexOf('>', idxStart);
       fExpected= trace.substring(idxStart + firstToken.length(), idxEnd);
       idxStart= idxEnd + nextTokenString.length();
-      fActual= trace.substring(idxStart, trace.lastIndexOf('>'));
+      int iFActual = trace.lastIndexOf('>');
+      if(iFActual == -1) iFActual = trace.lastIndexOf(']');
+      fActual= trace.substring(idxStart, iFActual);
     }
     else {
       fActual= "N/A";

I created a Run Configuration for a TestNG class to run just a single method from that class. This configuration will save and then runs correctly - however, when I view it again, the name of the class gets a comma appended to the end, and the method I had chosen disappears. I can't save or reselect that mehod until I manually delete that comma.

Please add "keep testng running after a test run when debugging" checkbox in the debug configuration testng view.
When it's checked and you run test suite in debug mode, the JVM running the test should wait for new commands instead of termination.
Then, in the test results view the user should be able to re-run single tests (methods) or test groups (classes, suites) without restarting the test JVM.

Rationale:

Especially in case of itnegration tests, test environment startup can take very long. In effect, edit-compile-test cycle takes very long. In many cases we could leverage JVM's code hot swap feature to quickly load corrected test/application code and then re-run the test without a need for restarting whole test environment.

In Eclipse there exists a possibility to execute "Rerun JUnit Test" and to associate it a keyboard shortcut. Would be very convenient if that was possible for TestNG, too.

Note: "Rerun TestNG Test" should rerun the last selected TestNG launch configuration and not create another one, which "Run as TestNG" from the editor does.

I've been using testNG for a couple years and enjoy its flexibility. The only gripe I have is run speed compared to the junit plugin. I recently worked on a project which was running junit, and for a simple test class of a dozen or so tests, run speed was about 1/10th what it is in testng. I assume this is b/c each testng run launches a java process; according to the stats in the console after a run, the internal execution is pretty fast (~200ms) compared to the run-time in the plugin panel (~1000ms). This may not feel like much, but when you are executing tests several times a minute while working on bug fixes and the like, it starts to feel like you are always waiting on the tests to finish.

Does this resonate at all? Any way to run them faster I just don't know about? Any thoughts on other ideas? Happy to do some hacking myself.

when using mockito + powermock with testng, we need to set the "object-factory" attribute in the testng.xml, like this

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="ProjectCommon" parallel="classes" object-factory="org.powermock.modules.testng.PowerMockObjectFactory">
  <test name="common" preserve-order="false">
    <packages>
       <package name="xxx.yyy.*" />
    </packages>
  </test>
</suite>

when starting the testsuite using "mvn" from commandline everything works as expected.
when the tests are started using the eclipse-plugin i get a nullpointer-exception and the testrun fails.

seems to be some classloading issues, since the error happens while initializing our JAX-WS Clients.

Now I use testng-eclipse plugin 6.3.1 to run a testng xml file with a error argument, and the TestNG runner is hang.
-listener org.testng.demo.TestListener
org.testng.demo.TestListener class is not exist.

I think the reason is:

1. RemoteTestNG load listener failed and exit.
2. The method startTestRunListening of TestRunnerViewPart are called, and called BaseMessageSender initReceiver() method. In initReceiver method, use a endless loop trying to contact RemoteTestNG, and hanghere.

Hi,

I downloaded the latest version of testng elipse, and met a problem.

We use the testng eclipse api do some development, however, when I upgrade it into 6.4, I met some issues:

Access restriction: The type PreferenceStoreUtil is not accessible due to restriction on required project testng-eclipse
Access restriction: The type CommandLineArgs is not accessible due to restriction on required library testng-eclipse/lib/testng.jar
................

I am not sure if there are some license issue for this, very look forward for your feedback.

Thanks in advance.

Br,
Tim

Hi,

I found in testng eclipse 6.4, you only export those package:

Export-Package: org.testng,
org.testng.annotations,
org.testng.collections,
org.testng.eclipse,
org.testng.eclipse.launch,
org.testng.eclipse.util,
org.testng.internal,
org.testng.remote,
org.testng.xml

Could you add this in the next release:

org.testng.internal.annotations,

Really thanks.

Br,
Tim

If a DataProvider provides 100 scenarios for 1 test, and 99 scenarios pass but 1 fails, there's no way to rerun the failed scenario from Eclipse without rerunning the other 99 scenarios.

The same problem doesn't exist if testng-failed.xml is used to rerun the failing tests from command line. So this is a shortcoming of the Eclipse plugin, not TestNG.

I did not know where to report this bug: in testng-eclipse or testng.
I decided to report it in testn-eclipse because this is where I see the outcome of this bug.
Apologies if this was not the right place and feel free to move it to testng.

Consider the following test class:

package org.testng.sample;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class DataProviderReturnsArrayTestCase
{

  @DataProvider(name = "arrayInTheData")
  public Object[][] getArrayInTheData()
  {
    return new Object[][] { { new String[] { "a", "b", "c" } },
      { new String[] { "1", "2", "3" } } };
  }

  @Test(dataProvider = "arrayInTheData")
  public void testThatAcceptsAnArray(String[] strArray)
  {
    // test something...
  }
}

Here is the result of running this test in the TestNG view (it displays the result of calling toString() on an array):

I think this could be fixed by applying the following patch to testng:

diff --git a/src/main/java/org/testng/remote/strprotocol/TestResultMessage.java b/src/main/java/org/testng/remote/strprotocol/TestResultMessage.java
index 36397e6..6c9c323 100755
--- a/src/main/java/org/testng/remote/strprotocol/TestResultMessage.java
+++ b/src/main/java/org/testng/remote/strprotocol/TestResultMessage.java
@@ -309,6 +309,20 @@ public class TestResultMessage implements IStringMessage {
       if(null == o) {
         result.add("null");
       }
+      else if (o.getClass().isArray()) {
+        String[] strArray = toString((Object[]) o, null);
+        StringBuilder sb = new StringBuilder("[");
+        for (int i = 0; i < strArray.length; i++)
+        {
+          sb.append(strArray[i]);
+          if (i + 1 < strArray.length)
+          {
+            sb.append(",");
+          }
+        }
+        sb.append("]");
+        result.add(sb.toString());
+      }
       else {
         String tostring= o.toString();
         if(isStringEmpty(tostring)) {

You may freely reuse the test class and the patch file I published in this issue under the terms of the Apache License Version 2.0.

In test Run dialog, Arguments, VM Arguments text area
-Dlogistika.root.dir=${workspace_loc:gdit}/logistika-lib/work
In 5.14.6 as it was passed into test as W:\devel\project\gdit\trunk/logistika-lib/work
After upgrading to 5.14.10 it passes gdit/logistika-lib/work
It is not full path anymore and project name is used as prefix wrongly. Project is not stored inside workspace, but only referenced from there

Steps to reproduce:

1. run testgn test in Eclipse
2. wait for it to terminate
   • now you have results visible in the 'Results' view and console output visible in the 'Console' view
3. run or debug any launch in Eclipse (e.g. start an application server to launch selenium tests later, or just run an Ant build)

Observed

• Console output of previously terminated launch is forgotten (default Eclipse behaviour)
• So are the test results in the 'Results' view

Expected

• Console output of previously terminated launch is forgotten (default Eclipse behaviour)
• The test results are not cleared

When I launch a test-suite from an XML file, it ignores the dependency-methods annotations.
When I run the tests by a right-click on the class, it works.

It worked with 6.1.1.20110728_1012
It doesn't work anymore with 6.2.0.20110826_0903

Unable to resolve plug-in "platform:/plugin/org.testng.eclipse/icons/full/main16/testng_blur.gif". The icon is definitely there, I can't seem to find anything wrong with the plugin folder itself, but TestNG does not work. View is missing from "Show View" and plugin does not load. Running Eclipse 4.3.0 M2 with JDK 1.7.0.

Here is the stack trace:

java.io.IOException: Unable to resolve plug-in "platform:/plugin/org.testng.eclipse/icons/full/main16/testng_blur.gif".
    at org.eclipse.core.internal.runtime.PlatformURLPluginConnection.parse(PlatformURLPluginConnection.java:64)
    at org.eclipse.core.internal.runtime.PlatformURLPluginConnection.resolve(PlatformURLPluginConnection.java:75)
    at org.eclipse.core.internal.boot.PlatformURLHandler.openConnection(PlatformURLHandler.java:67)
    at org.eclipse.osgi.framework.internal.protocol.URLStreamHandlerProxy.openConnection(URLStreamHandlerProxy.java:112)
    at java.net.URL.openConnection(URL.java:969)
    at org.eclipse.core.internal.runtime.PlatformURLConverter.toFileURL(PlatformURLConverter.java:33)
    at org.eclipse.core.runtime.FileLocator.toFileURL(FileLocator.java:206)
    at org.eclipse.jface.resource.URLImageDescriptor.getFilePath(URLImageDescriptor.java:137)
    at org.eclipse.jface.resource.URLImageDescriptor.createImage(URLImageDescriptor.java:157)
    at org.eclipse.jface.resource.ImageDescriptor.createImage(ImageDescriptor.java:227)
    at org.eclipse.jface.resource.ImageDescriptor.createImage(ImageDescriptor.java:205)
    at org.eclipse.e4.ui.workbench.addons.minmax.TrimStack.getImage(TrimStack.java:652)
    at org.eclipse.e4.ui.workbench.addons.minmax.TrimStack.updateTrimStackItems(TrimStack.java:712)
    at org.eclipse.e4.ui.workbench.addons.minmax.TrimStack.createWidget(TrimStack.java:583)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:56)
    at org.eclipse.e4.core.internal.di.InjectorImpl.processAnnotated(InjectorImpl.java:859)
    at org.eclipse.e4.core.internal.di.InjectorImpl.inject(InjectorImpl.java:111)
    at org.eclipse.e4.core.internal.di.InjectorImpl.internalMake(InjectorImpl.java:319)
    at org.eclipse.e4.core.internal.di.InjectorImpl.make(InjectorImpl.java:253)
    at org.eclipse.e4.core.contexts.ContextInjectionFactory.make(ContextInjectionFactory.java:185)
    at org.eclipse.e4.ui.internal.workbench.ReflectionContributionFactory.createFromBundle(ReflectionContributionFactory.java:105)
    at org.eclipse.e4.ui.internal.workbench.ReflectionContributionFactory.doCreate(ReflectionContributionFactory.java:71)
    at org.eclipse.e4.ui.internal.workbench.ReflectionContributionFactory.create(ReflectionContributionFactory.java:49)
    at org.eclipse.e4.ui.workbench.renderers.swt.ToolControlRenderer.createWidget(ToolControlRenderer.java:75)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.createWidget(PartRenderingEngine.java:897)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.safeCreateGui(PartRenderingEngine.java:630)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.safeCreateGui(PartRenderingEngine.java:732)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.access$2(PartRenderingEngine.java:703)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$7.run(PartRenderingEngine.java:697)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.createGui(PartRenderingEngine.java:682)
    at org.eclipse.e4.ui.workbench.renderers.swt.SWTPartRenderer.processContents(SWTPartRenderer.java:59)
    at org.eclipse.e4.ui.workbench.renderers.swt.TrimBarRenderer.processContents(TrimBarRenderer.java:154)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.safeCreateGui(PartRenderingEngine.java:642)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$6.run(PartRenderingEngine.java:518)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.createGui(PartRenderingEngine.java:503)
    at org.eclipse.e4.ui.workbench.renderers.swt.WBWRenderer.processContents(WBWRenderer.java:660)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.safeCreateGui(PartRenderingEngine.java:642)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.safeCreateGui(PartRenderingEngine.java:732)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.access$2(PartRenderingEngine.java:703)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$7.run(PartRenderingEngine.java:697)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.createGui(PartRenderingEngine.java:682)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:969)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:924)
    at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:86)
    at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:588)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
    at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:543)
    at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
    at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:124)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1438)

hi,

in our dev environment, we stick with JDK1.5, can't upgrade to 1.6 or 1.7 for some reason

so i have to use testng-5.11-jdk15.jar since the latest testng 6.3.1 requires JDK1.6.
i installed org.testng.eclipse - 6.3.2.20111203_1323, and checked "UseProjectJar" in the project property page
when i run the test, the whole eclipse sucks, no response at all, i have to kill it with the task manager (under windows 7 x64)

Copying this from the Google Code issue tracker (http://code.google.com/p/testng-eclipse/issues/detail?id=63) as it doesn't seem to have been addressed but also doesn't seem to have been migrated to GitHub.

What steps will reproduce the problem?

1. create a base class
   import org.testng.annotations.Test;

@test
public class BaseClassTest {

}

1. create a sub-class

public class SomeTest extends BaseClassTest {

public void test1() {
    // test something
}

}
3. try to run/debug the sub-class SomeTest from any context fails,
Some examples:
a- in the editor for SomeTest. right-click and select Run or Debug as..., the TestNG menu item doesn't show up
b- from the Debug Configuration UI, "new" a TestNG configuration, set corresponding project, and try to browse for classes, "SomeClass" won't show up.
c- in the package explorer, right-click the Sometest.java node, and do the same as aboce, again the menu item to launch TestNG doesn't show up

What is the expected output? What do you see instead?

I would expect the TestNG menu-item to show up so that a test can be launched on SomeTest. Instead the testng plug-in seems to ignore that this class implements a valid test and thus doesn't provide a menu-item to launch it.

We're trying to build our tests with testng. And since our system is quiet complex we're trying to capture some of the complexity in a base class that all developers could simply inherit from.

What version of the product are you using? On what operating system?

TestNG 6.2.0.20110905_0934 org.testng.eclipse.feature.group Cedric Beust

Eclipse SDK
Version: 3.7.1
Build id: M20110909-1335

on Windows 7, 64 bits.

Please provide any additional information below.

I looked a bit at the plug-in source code. It looks like JDT core doesn't have a fast API to lookup the super types of a type (it seems you'd have to use an asynchronous search, which would probably be too slow for a UI gesture like this).
Of course there are workarounds, like putting @test on the sub-class, but at this point we're still trying to come up with a programming model as lightweight as possible.

Note the above test case is an over simplification. In the real implementation @test has more attributes (like dataProvider) that we'd prefer not to repeat for every single test.

1. In eclipse, selecting file -> new ->test NG Class...
2. Select an identical source folder, package name, and class name to an existing TestNG Test class.
3. Click "Finish," and watch as the wizard silently overwrites the existing test's .java file.

When right-clicking in any view a TestNG menu item appears, in some cases (for example the Remote System Explorer's 'Servers' view) even two entries appear (The second is TestNG->Convert to YAML). These menu items should be filtered out when the items in them make no sense. Please take a look at the handlers extension point. http://wiki.eclipse.org/Menu_Contributions/Problems_View_Example#Handlers

Hi,
My project use a more recent version of guice. (3.0-SNAPSHOT or sisu fork https://github.com/sonatype/sisu-guice).
Using maven surefire plugin correctly use the version from the pom.
But the testng eclipse plugin use the one "bundle" with the eclipse plugin

Hi,

We are using Eclipse 3.4.2 (Gallileo), and we tried to install TestNG and it cannot because of these dependencies:

Cannot complete the request.  See the details.
Unsatisfied dependency: [org.testng.eclipse 5.14.3.20101201_1526] requiredCapability: osgi.bundle/org.eclipse.ltk.ui.refactoring/[3.4.101,4.0.0)
Unsatisfied dependency: [org.testng.eclipse 5.14.3.20101201_1526] requiredCapability: osgi.bundle/org.eclipse.ltk.ui.refactoring/[3.4.101,4.0.0)
Unsatisfied dependency: [org.testng.eclipse.feature.group 5.14.3.20101201_1526] requiredCapability: org.eclipse.equinox.p2.iu/org.testng.eclipse/[5.14.3.20101201_1526,5.14.3.20101201_1526]

Is the plugin no longer supporting Eclipse 3.4? It works great in 3.6.

This is related to #47, I think, but not identical to.

Steps to Reproduce

1. Create a test class with a test method (marked with @Test)
2. Create a subclass of the class in step (1) with NO test methods
3. Try Run->Run As...

TestNG Test will not be there.

Sample Project

A sample project is available.

More Info

I think there might be a bug in TestSearchEngine.isTest(). I don't think it checks superclasses for any valid TestNG stuff. I'll be happy to write a patch if someone gives me some direction. =)

TestNG view is missing the history feature that allows to easily view results and re-run older runs.
Best definition of the feature would be "like in Eclipse JUnit view". There, test results history allows:

• to list a few last test suites executions along with date and status (all green or something failed)
• to view test suite results tree of any of those runs
• when viewing older run, buttons 'Rerun last test', 'Run failed tests' should rerun currently viewed test suite, not the last run suite

I have a somewhat large project that I've been testing with TestNG. The project has about 1000 source files (main source + tests) and 47 libraries linked. When I'm running a test and click Browse in the TestNG Run Configuration (for either Class, Method, Groups, or Package) it takes about a minute and a half to show a list.

I've messed around with the project setup to see if something in particular slowed it down. Each time I removed a few libraries, it sped up a little. Likewise, when I removed some source folders, that sped it up as well. I removed all libraries and it works in about 6 seconds. If instead I remove most of my source and leave the libraries, it takes about 4 seconds. It seems that the combination of a lot of source with several libraries cause it to slow way down.

Please help me... When I run my test suite I am getting this error.. I tried all ways listed under
http://code.google.com/p/testng-eclipse/issues/detail?id=43 and nothing worked for me..:( I am not using maven.

java.lang.NoClassDefFoundError: org.testng.internal.ClassImpl
at java.lang.Class.initializeClass(libgcj.so.90)
at org.testng.internal.BaseClassFinder.findOrCreateIClass(BaseClassFinder.java:48)
at org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:117)
at org.testng.TestRunner.initMethods(TestRunner.java:413)
at org.testng.TestRunner.init(TestRunner.java:235)
at org.testng.TestRunner.init(TestRunner.java:205)
at org.testng.TestRunner.(TestRunner.java:160)
at org.testng.remote.RemoteTestNG$1.newTestRunner(RemoteTestNG.java:142)
at org.testng.remote.RemoteTestNG$DelegatingTestRunnerFactory.newTestRunner(RemoteTestNG.java:271)
at org.testng.SuiteRunner$ProxyTestRunnerFactory.newTestRunner(SuiteRunner.java:561)
at org.testng.SuiteRunner.init(SuiteRunner.java:157)
at org.testng.SuiteRunner.(SuiteRunner.java:111)
at org.testng.TestNG.createSuiteRunner(TestNG.java:1278)
at org.testng.TestNG.createSuiteRunners(TestNG.java:1265)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1119)
at org.testng.TestNG.run(TestNG.java:1036)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
Caused by: java.lang.ClassNotFoundException: com.google.inject.Module not found in gnu.gcj.runtime.SystemClassLoader{urls=[file:/afs/.cpiv.com/techhome/shari.dh/.eclipse/org.eclipse.platform_3.5.0_2045279800/plugins/org.testng.eclipse_6.8.0.20121120_1820/lib/testng.jar,file:/afs/.cpiv.com/techhome/shari.dh/workspace/nstest/bin/,file:/afs/.cpiv.com/techhome/shari.dh/Desktop/selenium-server-standalone-2.28.0.jar,file:/afs/.cpiv.com/techhome/shari.dh/Desktop/selenium-java-2.28.0.zip,file:/afs/.cpiv.com/techhome/shari.dh/Desktop/org.testng.eclipse_6.8.0.20121120_1820.jar], parent=gnu.gcj.runtime.ExtensionClassLoader{urls=[], parent=null}}
at java.net.URLClassLoader.findClass(libgcj.so.90)
at gnu.gcj.runtime.SystemClassLoader.findClass(libgcj.so.90)
at java.lang.ClassLoader.loadClass(libgcj.so.90)
at java.lang.ClassLoader.loadClass(libgcj.so.90)
at java.lang.Class.initializeClass(libgcj.so.90)
...18 more

Thanks!

The Eclipse JUnit Runner has a feature where one can filter stacktrace elements from failed tests. See Eclipse Preferences => TreePath: Java - JUnit for filters.

It would be very nice to have the same feature in TestNG to minimize scrolling.

Default entries would be:
org.testng.internal.*
org.testng.TestRunner.*
sun.reflect.*
java.lang.*

Hello,

I tried to install the TestNG Eclipse plug-in using the update site, but it seems there's a problem with the update site.
Eclipse just can't find any update-stie at http://beust.com/eclipse.

The following error appears:

Unable to read repository at http://beust.com/eclipse/content.xml.
Connection reset

Could you please fix the update site and/or give zip files to install the plugin manually ?

Thanks for the amazing work on TestNG, it does exactly what I needed ;)

EDIT : might have been a proxy problem ...

In org/testng/eclipse/ui/TestRunnerViewPart.java, method showResultsInTree() does a System.out that appears in the console each time a test is run with TestNG Eclipse plugin.

See code below:

    postSyncRunnable(new Runnable() {
      public void run() {
        long start = System.currentTimeMillis();
        for (TestRunTab tab : ALL_TABS) {
          tab.updateTestResult(m_results);
        }
        System.out.println("Done updating tree:" + (System.currentTimeMillis() - start) + "ms");
      }
    });
  }

Can you please remove this call to System.out?

Thanks.

Currently, when you create a new test file from an existing Java class, it displays the list of methods you want to test. However, it does not show the method parameters. So if you have several methods like:
void set(String ...)
void set(Integer...)
They show up as set(), set().

It would be nice if it could generate tests similar to the Junit plugin like:
void set(String) --> setString()
void set(Integer) --> setInteger()

When running a group of tests, I want to see in the TestNG view:

• which tests have passed so far
• which one is currently running
• which tests are still to run

(i.e. similar to JUnit's view in Eclipse).

Currently, while the tests are running I can only see failures listed.