SBT interface to partest
Home Page: http://docs.scala-lang.org/scala
SBT interface to partest
This project is retired. See the README of https://github.com/scala/scala-partest for the new, simplified instructions for using partest from SBT.
While the Scala build has its main parttest suite in ./test/files, it has an additional suite under ./test/scaladoc/{run,scalacheck}. The command line partest script allows us to specify ./test/partest --srcpath test/scaladoc --run. My testing indicates that providing the option through partest-interface doesn't pick up these tests.
I suspect that the hardcoded "files" below is to blame.
def execute(eventHandler: EventHandler, loggers: Array[Logger]): Array[Task] = {
val forkedCp = scala.util.Properties.javaClassPath
val classLoader = new URLClassLoader(forkedCp.split(java.io.File.pathSeparator).map(new File(_).toURI.toURL))
val runner = SBTRunner(Framework.fingerprint, eventHandler, loggers, "files", classLoader, null, null, Array.empty[String], args)
I'm trying to test a macro with partest. I followed the instructions to set up my projects:
https://github.com/backuity/ansi-interpolator/blob/master/project/Build.scala
When I run test I get the following stack trace:
sbt.ForkMain$ForkError
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at scala.tools.partest.PartestTask.SBTRunner(TestingInterface.scala:91)
at scala.tools.partest.PartestTask.execute(TestingInterface.scala:70)
at sbt.ForkMain$Run$2.call(ForkMain.java:294)
at sbt.ForkMain$Run$2.call(ForkMain.java:284)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: sbt.ForkMain$ForkError: scala/tools/nsc/Properties$
at scala.tools.partest.nest.AntRunner.<init>(AntRunner.scala:20)
at scala.tools.partest.nest.SBTRunner.<init>(SBTRunner.scala:26)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at scala.tools.partest.PartestTask.SBTRunner(TestingInterface.scala:91)
at scala.tools.partest.PartestTask.execute(TestingInterface.scala:70)
at sbt.ForkMain$Run$2.call(ForkMain.java:294)
at sbt.ForkMain$Run$2.call(ForkMain.java:284)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: sbt.ForkMain$ForkError: scala.tools.nsc.Properties$
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at scala.tools.partest.nest.AntRunner.<init>(AntRunner.scala:20)
at scala.tools.partest.nest.SBTRunner.<init>(SBTRunner.scala:26)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at scala.tools.partest.PartestTask.SBTRunner(TestingInterface.scala:91)
at scala.tools.partest.PartestTask.execute(TestingInterface.scala:70)
at sbt.ForkMain$Run$2.call(ForkMain.java:294)
at sbt.ForkMain$Run$2.call(ForkMain.java:284)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
--verbose ought to print the transcripts, as it does in the commmand line runner. We shouldn't have to look at the individual test case log file manually.
> partest --show-diff --verbose --srcpath scaladoc --grep SI-191
[info] Packaging /Users/jason/code/scala2/build-sbt/pack/lib/scala-partest-javaagent.jar ...
[info] Done packaging.
Partest version:
Compiler under test: $baseDir/compiler
Scala version is: Scala compiler version 2.11.8-20160206-230105-fcaa3ab -- Copyright 2002-2016, LAMP/EPFL
Scalac options are:
Compilation Path: /Users/jason/code/scala2/target/test/it-classes:$baseDir/test:$baseDir/compiler:$baseDir/library:/Users/jason/code/scala2/build-sbt/libs/classes/forkjoin:$baseDir/reflect:$baseDir/interactive:$baseDir/actors:$baseDir/repl-jline-embedded:$baseDir/repl-jline:$baseDir/repl:$baseDir/scalap:$baseDir/partest-extras:/Users/jason/code/scala2/build-sbt/pack/lib/scala-partest-javaagent.jar:$baseDir/scaladoc:/Users/jason/code/scala2/test/files/lib/annotations.jar:/Users/jason/code/scala2/test/files/lib/enums.jar:/Users/jason/code/scala2/test/files/lib/genericNest.jar:/Users/jason/code/scala2/test/files/lib/jsoup-1.3.1.jar:/Users/jason/code/scala2/test/files/lib/macro210.jar:/Users/jason/code/scala2/test/files/lib/methvsfield.jar:/Users/jason/code/scala2/test/files/lib/nest.jar:/Users/jason/code/scala2/test/lib:/Users/jason/.ivy2/cache/org.apache.ant/ant/jars/ant-1.9.4.jar:/Users/jason/.ivy2/cache/org.apache.ant/ant-launcher/jars/ant-launcher-1.9.4.jar:/Users/jason/.ivy2/cache/org.scala-lang.modules/scala-asm/bundles/scala-asm-5.0.4-scala-3.jar:/Users/jason/.ivy2/cache/org.scala-lang.modules/scala-xml_2.11/bundles/scala-xml_2.11-1.0.4.jar:/Users/jason/.ivy2/cache/org.scala-lang.modules/scala-parser-combinators_2.11/bundles/scala-parser-combinators_2.11-1.0.4.jar:/Users/jason/.ivy2/cache/jline/jline/jars/jline-2.12.1.jar:/Users/jason/.ivy2/local/org.scala-lang.modules/scala-partest_2.11/1.0.12-SNAPSHOT/jars/scala-partest_2.11.jar:/Users/jason/.ivy2/cache/com.googlecode.java-diff-utils/diffutils/jars/diffutils-1.3.0.jar:/Users/jason/.ivy2/cache/org.scala-sbt/test-interface/jars/test-interface-1.0.jar:/Users/jason/.ivy2/cache/org.scala-lang.modules/scala-partest-interface_2.11/jars/scala-partest-interface_2.11-0.7.0.jar:/Users/jason/.ivy2/cache/org.scalacheck/scalacheck_2.11/jars/scalacheck_2.11-1.11.6.jar:/Users/jason/.sbt/boot/scala-2.10.5/org.scala-sbt/sbt/0.13.9/test-agent-0.13.9.jar:/Users/jason/.sbt/boot/scala-2.10.5/org.scala-sbt/sbt/0.13.9/test-interface-1.0.jar
Java binaries in: /Library/Java/JavaVirtualMachines/jdk1.8.0_71.jdk/Contents/Home/jre/bin
Java runtime is: Java HotSpot(TM) 64-Bit Server VM (build 25.71-b15, mixed mode)
Java options are: -Xmx1024M -Xms64M -XX:MaxPermSize=128M
baseDir: /Users/jason/code/scala2/build-sbt/quick/classes
sourceDir: /Users/jason/code/scala2/test/scaladoc
Selected 1 tests drawn from specified tests
# starting 1 test in run
% scalac run/SI-191.scala
!! 1 - run/SI-191.scala [non-zero exit code]
#0/1 passed, 1 failed in run
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] partest
[error] (test/it:testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 9 s, completed 07/02/2016 12:38:16 AM
> eval IO.read(file("test/scaladoc/run/SI-191-run.log"))
[info] ans: String = Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128M; support was removed in 8.0
[info] java.lang.AssertionError: assertion failed
[info] at scala.Predef$.assert(Predef.scala:156)
[info] at Test$.check$1(SI-191.scala:49)
[info] at Test$.testModel(SI-191.scala:52)
[info] at scala.tools.partest.ScaladocModelTest.show(ScaladocModelTest.scala:71)
[info] at scala.tools.partest.DirectTest.delayedEndpoint$scala$tools$partest$DirectTest$1(DirectTest.scala:92)
[info] at scala.tools.partest.DirectTest$delayedInit$body.apply(DirectTest.scala:18)
[info] at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
[info] at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
[info] at scala.App$$anonfun$main$1.apply(App.scala:76)
[info] at scala.App$$anonfun$main$1.apply(App.scala:76)
[info] at scala.collection.immutable.List.foreach(List.scala:381)
[info] at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
[info] at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
[info] at scala.App$class.main(App.scala:76)
[info] at scala.tools.partest.DirectTest.main(DirectTest.scala:18)
[info] at Test.main(SI-191.scala)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] at java.lang.reflect.Method.invoke(Method.java:497)
[info] at scala.reflect.internal.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:70)
[info] at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
[info] at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:101)
[info] at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:70)
[info] at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:101)
[info] at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:22)
[info] at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
[info] at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
[info] at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
[info] at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:65)
[info] at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:87)
[info] at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:98)
[info] at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
[info] at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
I attempted to thread this through. I got as far as:
diff --git a/src/main/scala/scala/tools/partest/nest/SBTRunner.scala b/src/main/scala/scala/tools/partest/nest/SBTRunner.scala
index acc956f..368e65f 100644
--- a/src/main/scala/scala/tools/partest/nest/SBTRunner.scala
+++ b/src/main/scala/scala/tools/partest/nest/SBTRunner.scala
@@ -62,7 +62,7 @@ class SBTRunner(partestFingerprint: Fingerprint, eventHandler: EventHandler, log
case Pass(_) => (Status.Success, new OptionalThrowable)
case Updated(_) => (Status.Success, new OptionalThrowable)
case Skip(_, _) => (Status.Skipped, new OptionalThrowable)
- case Fail(_, _, _) => (Status.Failure, new OptionalThrowable)
+ case Fail(_, reason, transcript) => (Status.Failure, new OptionalThrowable(new Throwable(reason + "\n\n" + transcript.mkString("\n"))))
case Crash(_, e, _) => (Status.Error, new OptionalThrowable(e))
}
}
Which gets it into target/partest/test-reports/partest.xml files (via JUnitXMLListener).
The test run below appeared to have two failed tests, but 3 failed tests were reported. Total and passed also appear to be incorrect.
[info]
!! 1 - presentation/doc [compilation failed]
ok 2 - presentation/ide-bug-1000469
ok 3 - presentation/completion-implicit-chained
ok 4 - presentation/forgotten-ask.scala
ok 5 - presentation/ide-bug-1000349
ok 6 - presentation/hyperlinks-macro
ok 7 - presentation/callcc-interpreter
ok 8 - presentation/hyperlinks
ok 9 - presentation/ide-bug-1000475
!! 10 - presentation/memory-leaks [compilation failed]
ok 11 - presentation/ide-bug-1000531
ok 12 - presentation/ide-t1000567
ok 13 - presentation/ide-t1000609
ok 14 - presentation/implicit-member
ok 15 - presentation/ide-t1001388
ok 16 - presentation/ide-t1000976
ok 17 - presentation/partial-fun
ok 18 - presentation/parse-invariants
ok 19 - presentation/ping-pong
ok 20 - presentation/random
ok 21 - presentation/recursive-ask
ok 22 - presentation/private-case-class-members
ok 23 - presentation/scope-completion-1
ok 24 - presentation/scope-completion-2
ok 25 - presentation/scope-completion-3
ok 26 - presentation/t4287
ok 27 - presentation/t4287b
ok 28 - presentation/t1207
ok 29 - presentation/scope-completion-import
ok 30 - presentation/scope-completion-4
ok 31 - presentation/t4287c
ok 32 - presentation/t5708
ok 33 - presentation/t7548
ok 34 - presentation/t7548b
ok 35 - presentation/t7678
ok 36 - presentation/t7915
ok 37 - presentation/t8459
ok 38 - presentation/t8085b
ok 39 - presentation/t8085
ok 40 - presentation/t8941
ok 41 - presentation/t8934
ok 42 - presentation/visibility
ok 43 - presentation/t8941b
[error] Failed: Total 44, Failed 3, Errors 0, Passed 41
[error] Failed tests:
[error] partest
[error] (test/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 53 s, completed Mar 11, 2015 4:06:27 PM
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.