hedgehogqa / scala-hedgehog Goto Github PK
View Code? Open in Web Editor NEWRelease with confidence, state-of-the-art property testing for Scala.
Home Page: https://hedgehogqa.github.io/scala-hedgehog
License: Other
Release with confidence, state-of-the-art property testing for Scala.
Home Page: https://hedgehogqa.github.io/scala-hedgehog
License: Other
The Haskell and F# version both use SplitMix, which is a clever way of doing PRNG.
Currently we have to use a more State-based PRNG.
The current choice was pure laziness in picking the first functional PRNG I could find on the JVM.
I'm not convinced that we need switch, but it would certainly make the code simpler not having to deal with the extra Seed
return value and re-implementing State
. That said, my vague understanding is that using SplitMix
gives rise to an implementation of Monad[GenT]
that break the monad laws, but may well be true of the current version as well (if someone knows please comment).
If we do switch we should be very careful to avoid weak gamma values.
I only want to do this if it's automated though. The easiest way would be to do another build/publish when a Git tag is added. That said, it would be great if we could add a PR process for adding a versioning so we can run mima.
Since Scala 3.0.0-M1 (Dotty) has been released, I think it's time to support Dotty.
I'm currently working on it but having an issue with macro used in Minitest.
Fortunately, there is an open PR to support Dotty on the Minitest repo so I'll finish it once that PR is merged and released.
I'm afraid to say that due to a few life developments I am unable to do any real maintenance or support for Scala Hedgehog for the foreseeable future. :(
While the details probably aren't all that interesting, I might just mention them briefly here. Firstly, and most importantly, my wife and I have just recently had our first child, and I'm committed to spending as much quality time with him as I can. Secondly, we have moved to Brisbane and I have started a new job, which is also a little more demanding than my previous one. I'm also restricting my work hours so that I can get home to be my son, so I don't feel comfortable squeezing in any time on Hedgehog at work.
My apologies to those who depends/relies on this library and who are inconvenienced by this change (I don't believe there are many, but it's hard to be sure). I will certainly be around to answer questions, but that's about all I have time for I'm afraid.
I don't know if this is a bug but it sure had me baffled for quite a while...
I was wondering why my generators never seemed to be generating their minimum values and it turns out that Range
's where x - y >= 100 && (origin == x || origin == y)
will never generate values at the origin
.
The reason is that the size from hedgehog.core.PropertyTReporting#report
will start at 1
(with the default configuration). I had always thought that the size started at 0
since that is the only value which ensures that hedgehog.Range#scaleLinear
will return z
unscaled.
Here is an example:
hedgehog.Range.scaleLinear(hedgehog.Size(0), 0, 100) == 0
hedgehog.Range.scaleLinear(hedgehog.Size(1), 0, 100) == 1
hedgehog.Range.scaleLinear(hedgehog.Size(100), 0, 100) == 100
I think this should be a bug. If the size only goes from 1 - 100 then the range only goes from 1 - 100 rather than 0 - 100.
def testMemory: Property =
Gen.string(Range.linear(100, 100))
.list(Range.linear(100, 100))
.forAll
.map(_ => Result.success)
With -Xmx1g
crashes with OOME. Identity
and Tree
are dominating. Hacking around with list
to be more strict improves things, but at the cost of shrinking correctness.
Yeah this is pretty embarrassing. :(
Would be nice to browse the API through scaladoc
https://github.com/sbt/test-interface
Because just using a main
function would be too easy.
Compilation fails in Java 9 or higher version due to deprecation of Class.newInstance().
Just change to JDK 9 or higher and sbt compile
then it shows the following error messages.
[error] /location/to/scala-hedgehog/sbt-test/src/main/scala/hedgehog/sbt/Framework.scala:69:11: method newInstance in class Class is deprecated: see corresponding Javadoc for more information.
[error] c.newInstance
[error] ^
[error] one error found
[error] (sbt-test / Compile / compileIncremental) Compilation failed
[error] Total time: 11 s, completed 8 Oct. 2018, 10:07:31 pm
I used Java 11 (OpenJDK 11) but the Class.newInstance
has been deprecated since Java 9 so it should result in the same issue using Java 9.
Use Class.getDeclaredConstructor().newInstance()
instead.
I've fixed and tested it already. The solution works. I'll create a PR soon. The fix works in both Java 8 and 11 so it should work in 9 and 10 as well although both 9 and 10 have been expired.
The test report XML file (JUnit test report format) generated by Hedgehog does not have any message from test failure when there's any test failure.
e.g.)
<?xml version='1.0' encoding='UTF-8'?>
<testsuite hostname="my.local" name="io.kevinlee.property_based_testing.MyAppSpecWithHedgehog" tests="1" errors="0" failures="1" skipped="0" time="0.033" timestamp="2019-07-13T16:01:42">
<properties>
<!-- omitted -->
</properties>
<testcase classname="io.kevinlee.property_based_testing.MyAppSpecWithHedgehog" name="add(a, b) should return a + b" time="0.033">
<failure message="No Exception or message provided"/>
</testcase>
<system-out><![CDATA[]]></system-out>
<system-err><![CDATA[]]></system-err>
</testsuite>
As it shows, <failure message="No Exception or message provided"/>
has no message from the test failure.
As @charleso pointed out already in a conversation with me, it might be caused by this line of code always passing None
for Option[Throwable]
argument which might be used for the failure message.
I'd like to see an integration between minitest and Hedgehog. They already have an integration module for ScalaCheck. I am not sure whether a Hedgehog integration should best be a module of minitest or of scala-hedgehog. Given that scala-hedgehog doesn't have proper releases yet, I think that having it in the scala-hedgehog project would work better.
I already implemented the integration. Given minitest's nature, the integration is minimal and requires only a single file with 30-40 lines of code. If there is interest in having this integration in a scala-hedgehog sbt sub project, I will open a PR.
There’s a @jlink repository targeted at comparing shrinking behaviour of various PBT libraries: https://github.com/jlink/shrinking-challenge
We're only using a call-by-need Identity
, which is lazy but not stack-safe. That means, among other things, that generating large lists will never work. We should use a stack-safe traverse
instead.
https://github.com/scalaz/scalaz/blob/v7.1.5/core/src/main/scala/scalaz/Traverse.scala#L80
Lazy approach would be to use travis' encrypted environment variables and something like bintray.
It would be nice to be able to support pattern matching with a for-comprehensions over Gen
.
What is required is an implementation of withFilter
.
The Gen
methods using Gen.integral
with a large range (more precisely too small min or too big max) generate too many duplicate numbers (99 same / only 1 different).
Most Gen.integral
methods namely short
, int
, long
and double
have this issue, yet Gen.char
seems fine.
I think it might have something to do with bit overflow.
10.14.1
1.8.0_181
2.12.6
1.2.6
Just run the following test
import hedgehog._
import hedgehog.Property._
import hedgehog.runner._
object MySpec extends Properties {
def tests: List[Test] = List(
property("testInt", testInt)
)
def testInt: Property = for {
x <- Gen.int(Range.linear(Int.MinValue, Int.MaxValue)).log("x")
} yield {
println(s"$x")
Result.assert(true)
}
}
Result:
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-2147483648
-367669218
As you can see, there are ninety-nine -2147483648
and one -367669218
.
Tests with all the other integral methods having the issues
import hedgehog._
import hedgehog.Property._
import hedgehog.runner._
object MySpec extends Properties {
def tests: List[Test] = List(
property("testShort", testShort)
, property("testInt", testInt)
, property("testLong", testLong)
, property("testDouble", testDouble)
)
def testShort: Property = for {
x <- Gen.short(Range.linear(Short.MinValue, Short.MaxValue)).log("x")
} yield {
println(s"$x")
Result.assert(true)
}
def testInt: Property = for {
x <- Gen.int(Range.linear(Int.MinValue, Int.MaxValue)).log("x")
} yield {
println(s"$x")
Result.assert(true)
}
def testLong: Property = for {
x <- Gen.long(Range.linear(Long.MinValue, Long.MaxValue)).log("x")
} yield {
println(s"$x")
Result.assert(true)
}
def testDouble: Property = for {
x <- Gen.double(Range.linearFrac(Double.MinValue, Double.MaxValue)).log("x")
} yield {
println(s"$x")
Result.assert(true)
}
}
Result:
Short
-32768
// ninety-nine -32768s
-15876
Int
-2147483648
// ninety-nine -2147483648
-1279861467
Long
-9223372036854775808
// ninety-nine -9223372036854775808
3983015563864400051
Double
NaN
Infinity
// ninety-nine Infinity
Gen.double
with Double.MinValue
and Double.MaxValue
may generate infinity. So far, I've seen only Double.PositiveInfinity
. I guess division by zero happens somewhere.
To reproduce it, you can simply run the following test.
object PropertyTest extends Properties {
def tests: List[Test] =
List(property("testGenDouble", testGenDouble))
def testGenDouble: Property = for {
n <- Gen.double(Range.linearFrac(Double.MinValue, Double.MaxValue)).log("n")
} yield Result.assert(!n.isInfinite)
}
The result might be like
testGenDouble: Falsified after 50 passed tests
> n: Infinity
Gen.double(Range.linearFracFrom(0D, Double.MinValue, Double.MaxValue))
has the same issue.
I developed these in order to port some ScalaCheck code. I think they would be handy to have:
def someOf[A](xs: List[A]): Gen[List[A]] = for {
n <- Gen.int(Range.constant(0, xs.length))
xs <- pick(n, xs)
} yield xs
def pick[A](n: Int, xs: List[A]): Gen[List[A]] = n match {
case 0 => Gen.constant(Nil)
case _ => for {
a <- Gen.elementUnsafe(xs)
(lhs, rhs) = xs.span(_ != a)
ys <- pick(n - 1, lhs ::: rhs.drop(1))
} yield a :: ys
}
Any objections to publishing to Maven Central and following semver? Git hash versioning and bintray make it more difficult to publish integration projects -- e.g., an munit + hedgehog bridge project.
Gen.integral(Range.linear)
with Long
range values which are greater than Int.MaxValue
can generate Long
values out of the range.
OS version: macOS High Sierra 10.13.6
JDK version: Oracle JDK 1.8.0_181
Scala version: 2.12.6
SBT version: 1.2.3
Hedgehog version:
d53add6001242fe005db51bdb3b1045cb358a54e
9fa11be968b5bce80cfdc7e46ef2c98ac06f81d2
e.g.) running the following test
import hedgehog._
import hedgehog.Property._
import hedgehog.runner._
object MySpec extends Properties {
def tests: List[Prop] = List(
Prop("test", test)
)
def test: Property[Unit] = for {
// FYI, without using automatic number promotion, e.g. (Int.MaxValue).toLong + 1L, doesn't make any difference.
x <- Gen.integral(Range.linear(Int.MaxValue + 1L, Long.MaxValue)).forAll
_ = println(s"x: $x / ${x.getClass}")
_ <- assert(true)
} yield ()
}
may generate Long
values like
x: -2147483648 / long
x: 939963596 / long
x: -2147483648 / long
x: 1558271342 / long
x: -2147483648 / long
x: -609689728 / long
x: -2147483648 / long
x: -453439177 / long
x: -2147483648 / long
x: -156076065 / long
x: -2147483648 / long
x: 1439518195 / long
x: -2147483648 / long
x: 657353906 / long
x: -2147483648 / long
x: -1808564068 / long
x: -2147483648 / long
x: 1922335041 / long
x: -2147483648 / long
x: -939539731 / long
x: -2147483648 / long
x: -277748096 / long
x: -2147483648 / long
x: 1594427233 / long
x: -2147483648 / long
x: 1887521431 / long
x: -2147483648 / long
x: 591692191 / long
x: -2147483648 / long
x: 346225587 / long
x: -2147483648 / long
x: -283745960 / long
x: -2147483648 / long
x: -919999642 / long
x: -2147483648 / long
x: 1531066437 / long
x: -2147483648 / long
x: -417881513 / long
x: -2147483648 / long
x: 1953755084 / long
x: -2147483648 / long
x: 1680720586 / long
x: -2147483648 / long
x: -167979916 / long
x: -2147483648 / long
x: -1194480696 / long
x: -2147483648 / long
x: -1049515392 / long
x: -2147483648 / long
x: 796188397 / long
x: -2147483648 / long
x: 1711529311 / long
x: -2147483648 / long
x: -1407235644 / long
x: -2147483648 / long
x: -182512082 / long
x: -2147483648 / long
x: -1405392926 / long
x: -2147483648 / long
x: -280871432 / long
x: -2147483648 / long
x: 521919314 / long
x: -2147483648 / long
x: -1200708761 / long
x: -2147483648 / long
x: 670834926 / long
x: -2147483648 / long
x: -1327842761 / long
x: -2147483648 / long
x: 283618817 / long
x: -2147483648 / long
x: 703023701 / long
x: -2147483648 / long
x: 105118359 / long
x: -2147483648 / long
x: 1675730596 / long
x: -2147483648 / long
x: -1862619281 / long
x: -2147483648 / long
x: -715241250 / long
x: -2147483648 / long
x: 683112162 / long
x: -2147483648 / long
x: -1686113508 / long
x: -2147483648 / long
x: 114941926 / long
x: -2147483648 / long
x: -674340250 / long
x: -2147483648 / long
x: -512874169 / long
x: -2147483648 / long
x: 1725665914 / long
x: -2147483648 / long
x: 1002672834 / long
x: -2147483648 / long
x: -769926961 / long
x: -2147483648 / long
x: -321988811 / long
x: -2147483648 / long
x: 2026678231 / long
Any Long
value less than Int.MaxValue
shouldn't be there, yet as you can see, there are even negative Long
values there.
Or test like this
def test: Property[Unit] = for {
x <- Gen.integral(Range.linear(Long.MaxValue >> 1, Long.MaxValue)).forAll
_ = println(s"x: $x / ${x.getClass}")
_ <- assert(true)
} yield ()
can generate values like
x: -1 / long
x: -1591335140 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 571504971 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1242519450 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1723595504 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1842561537 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 324575280 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -1070011865 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1669180956 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -2062390908 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 175779212 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1774467528 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 2014097179 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 147899542 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -1403977827 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -767724508 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1464276871 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1577008302 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -1287778900 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -2112858956 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 761675178 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -371500038 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1628998371 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -570559703 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: -1371590216 / long
x: -1 / long
x: -1 / long
x: -1 / long
x: 1876489010 / long
x: -1 / long
x: -1 / long
Currently the haskell version has:
newtype Gen m a =
Gen (Size -> Seed -> Tree (MaybeT m) a)
From a conversation with the author of the haskell implementation:
having 'MaybeT (Tree a)' means that you still get a list of children, even though the Nothing is supposed to mean "cut the tree off from this point"
which means you can't accidentally keep trying to shrink, you have no option but to discard
One option is to bake the Option
into the tree.
case class Tree[M[_], A](run: M[Option[Node[M, A]]])
This will then require the use of runDiscardEffect :: Monad m => Tree (MaybeT m) a -> Tree m (Maybe a)
in Property.report
Currently when a state based test fails (without shrinking), it will show all the commands that were going to run. It would be good if this list was trimmed to the point at which it failed so its clear what command to look at.
It would be great if we could see the seed that has been used during a test run, and if we could set the seed to a specific value read from an environment variable. This makes it a lot easier to analyse and reproduce failed tests.
I'm currently trying to figure out if it's possible to pull a value from Gen
? I looked at the tutorials, but it seems like it's not possible to get something else than Gen
/Property
- I think the value can only be accessed within a test?
I am benchmarking property-based test libraries, and one of the cases I want to cover is to see how fast scala-hedgehog can generate e.g. a Double.
Thanks for the effort put into this project by the way! :)
The following can generate 0.0 when shrunk
for {
d <- Gen.double(Range.linearFrac(Double.MinPositiveValue, 1)).forAll
_ = println(d == 0.0)
} yield Result.failure
x - diff
can outside of the range, which is missed by y != x
. We shouldn't just change that predicate though as the rest of the values after that can be fine.
This should be easy enough to fix though.
John Hughes did an excellent talk
https://www.youtube.com/watch?v=NcJOiQlzlXQ
Our Haskell sibling is about to get it so we should try to keep up... :)
Unfortunately when #63 was applied to "fix" the memory issues, I stupidly made all the result of evaluations strict, including all the shrink results at a given level. So once shrinking begun, for every shrink attempt all of the sibling shrink tests (potentially 10s to 100s) were run as well. For quick tests you wouldn't necessarily even notice, but for anything involving IO or that was slow this would result in shrinking that would never complete, making it pretty useless.
A test that captures this behaviour:
https://github.com/hedgehogqa/scala-hedgehog/pull/67/files#diff-02d4f55aa561899491b5b6e16c9fd9e6
I have used Scalacheck a lot in the past, and have gravitated over to the Scalatest flavour of it, using GeneratorDrivenPropertyChecks and native Scalatest assertions as opposed to Scalacheck properties.
This is all very nice, but I have the usual gripes about writing complex generators and finding that shrinkage doesn't preserve the test data invariants, so I end up disabling shrinkage and having to debug test failures with some quite complex failing test cases, or having to hand-roll shrinkers that I hope maintain the same invariants as the original generators.
Having heard of Hedgehog and ZIO Test, I've given both a spin, seen that they do indeed offer shrinkage that maintains test case invariants, and have plumped for Hedgehog after writing a nasty inefficient test case generator to stress both frameworks.
What I'd like is to have Hedgehog integrated into Scalatest for the one-stop shopping experience that I'm used to with Scalacheck & Scalatest. It doesn't have to be a verbatim translation of the original tests, but should only require a straightforward translation - maybe a few method name changes, extra arguments, but no big rearrangement of existing test code.
I've produced the following PR to show how a simple integration could work and to give a feel for the 'before and after' look of the tests (slightly complicated by one of the tests being in legacy Scalacheck style rather than in Scalatest style, but there is another that is a direct translation).
See here: sageserpent-open/americium#1. Sorry about the cheesy title, I was in a skittish mood.
If there is interest, I'd be happy to help moving this PR over into this project, or breaking it out on its own.
I'm going to continue working on the spike anyway for now as I wish to use it anyway on some other projects, so I shall be filling out some syntax helpers to provide generators matching the usual Scalacheck generators / arbitraries - think of 'nonEmptyList', arbInt etc.
Please comment in this issue / ping via GitHub if there is interest.
It doesn't look like haskell-hedgehog even has this! I would love to debug my generators and labelling with it.
I'm actually so keen that I might give a shot at implementing it, but I wouldn't how to do it and I can't even steal from haskell-hedgehog here... Any advice/tips/mentoring?
In my local sbt checkout, I have been seeing failures of sbt.ParseKey
that don't really provide any user feedback. The framework just reports that there was an error. I assume this is because the Test
object catches Exception
in its apply
method:
scala.util.control.NonFatal
instead.On a state based properties, when a command execution fail, the environment value is reset but the global state is kept. It leads to unexpected Environment error.
Here an example to reproduce :
import hedgehog.state.CommandIO
import hedgehog.state.Command
import hedgehog.core.GenT
import hedgehog.state.Environment
import hedgehog.state.Var
import hedgehog.core.Result
import hedgehog.state.Environment
import hedgehog.Gen
import hedgehog._
import hedgehog.runner._
import hedgehog.state._
object BugSpec extends Properties {
case class Token()
case class State(token: Option[Var[Token]])
def tests: List[Test] = {
List(
property("sequential", testSequential)
)
}
def testSequential: Property = {
sequential(
Range.linear(1, 100),
State(None),
commands,
() => ()
)
}
def commands: List[CommandIO[State]] =
List(
commandCreateToken,
commandUseToken
)
val commandCreateToken: CommandIO[State] = {
new Command[State, Unit, Token] {
def gen(s: State): Option[GenT[Unit]] =
s.token match {
case Some(_) => None
case None => Some(Gen.constant(()))
}
def execute(env: Environment, s: Unit): Either[String, Token] = {
println(s"create token with env $env")
Right(Token())
}
def update(s: State, i: Unit, o: Var[Token]): State = {
State(token = Some(o))
}
def ensure(
env: Environment,
before: State,
after: State,
i: Unit,
o: Token
): Result = {
Result.assert(after.token.isDefined)
}
}
}
val commandUseToken: CommandIO[State] = {
new Command[State, Var[Token], Unit] {
def gen(s: State): Option[GenT[Var[Token]]] =
for {
token <- s.token
} yield Gen.constant(token)
def execute(
env: Environment,
token: Var[Token]
): Either[String, Unit] = {
println(s"use token with env $env")
token.get(env)
Left("OH NO")
}
def update(state: State, i: Var[Token], o: Var[Unit]): State = state
def ensure(
env: Environment,
before: State,
after: State,
i: Input,
o: Unit
): Result = Result.success
}
}
}
Execution trace :
create token with env Environment(Map())
create token with env Environment(Map())
create token with env Environment(Map())
use token with env Environment(Map(Name(0) -> Token()))
use token with env Environment(Map())
[info] Using random seed: 33902690650767
[info] - BugSpec.sequential: Falsified after 2 passed tests
[info] > Var(Name(1)) = Var(Name(0))
[info] > Environment value not found for Var(0)
```
https://github.com/hedgehogqa/haskell-hedgehog/pull/272/files
This means that a generator like (,) <$> genA <*> genB will be able to alternate between shrinking genA and genB rather than doing one first and then the other.
This might not be possible with the State-based Seed rather than SplitMix:
This would include a Property.check(...): Future[Report]
function. Scalacheck doesn't support this but ScalaTest 3.2's built-in property testing support does. It's really useful when writing effectful tests that run on Scala.js, where you can't await futures inside the properties.
If Gen.frequency
isn't given any weights above zero, then it will blow up with Invariant: frequency hits an impossible code path.
Might be holding this tool wrong but when I run ./sbt build
I get an error:
Downloading sbt launcher for 1.1.6:
From http://repo.scala-sbt.org/scalasbt/maven-releases/org/scala-sbt/sbt-launch/1.1.6/sbt-launch.jar
To /Users/tim.mcgilchrist/.sbt/launchers/1.1.6/sbt-launch.jar
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /Users/tim.mcgilchrist/code/scala/scala-hedgehog/project
[info] Loading settings from build.sbt ...
[info] Loading settings from build.sbt ...
[info] Set current project to hedgehog (in build file:/Users/tim.mcgilchrist/code/scala/scala-hedgehog/)
[error] Not a valid command: build
[error] Not a valid project ID: build
[error] Expected ':'
[error] Not a valid key: build (similar: loadedBuild)
[error] build
[error] ^```
Hasn't been added due to time constraints. Mostly around printing the report, and more gen combinators
It looks like this problem is solved in Scala 3.0.0-M2. Because of this, there was no doc file generated for 0.6.0 and 0.6.0 could not be published to the Maven Central since Maven Central doesn't accept any artifacts without the doc file.
So now Hedgehog supporting Dotty (Scala 3) can be published. It will be done once the new minitest supporting Scala 3.0.0-M2 is released (monix/minitest#72).
Following on from #71, it would be nice to have hedgehogqa/haskell-hedgehog#288 ported.
Currently you need to specify the M
for most of the Gen
combinators (see the test. It actually works currently without the M
, and defaults to Id
. The problem then becomes that the performance seems to suffer, and my very rough guess is something to do with trampolining. It might be good to confirm this though.
One suggestion is to have a hard-coded IO
module (or whatever has the desired behaviour), but this will mean duplicating the API which is annoying.
I'm not sure exactly how this would work but it would be nice if the logs were available for a Success
result and not just for a Failure
.
The first case where I wanted them was when trying to integrate with ScalaMock. ScalaMock has a pretty awkward trait that has to be used to verify expectations. The only way to use this is to run the Hedgehog test to get the Report
and then ScalaMock verifies the expectations. If an expectation fails we fail the test but there is no way to show what the generated values were as the report does not contain them.
The second time I wanted them was when trying to write a helper method that would negate an arbitrary Result
. While that on its own is possible it won't be very good because the values are unknown. Even if they were it would be hard to write a very good log message but it would be possible to do something at least.
There is nothing preventing Size
from having a value over the max
of 100
and therefore percentage
returning a value higher than 1
.
Similarly the size can be negative which I don't think is valid (or is it?).
One confusing implication for this is when it comes to Range
's which scale with the size. I would expected Range.linear(1, 1000)
to never produce bounds outside of these limits but it can if given a bad size. This leads to a generator producing unexpected values and the whole thing becomes a bit of a nightmare to try and track down. Of course the original bug is probably a bad resize
or something but finding where that is is really tough since the error occurs when the generator is run not when it is constructed.
Is it intended for the Size
to only ever be between 0
and 100
? If so should we cap it?
not really an issue but I couldnt find any scala-hedgehog team's email to write this to :)
I've created this library https://github.com/svalaskevicius/hedgehog-arbitrary and thought you might be interested to have a look at it - maybe also share your feedback? :)
helps with generating input for big ADTs..
Regards,
Sarunas
cover
operates on PropertyT
, while the generators and commands for state machine testing are GenT
and Result
.
So as far as I can tell, there's nowhere to put coverage annotations.
Current Gen
does not have combinators for functions (e.g. Gen[A => B]
).
There are some cases where having function combinators is really handy (e.g. Functor laws, Monad laws, etc.) so can we have function combinators maybe like Gen.function1
, Gen.function2
, etc.?
As you know, there is a function generation project for Haskell Hedgehog.
http://hackage.haskell.org/package/hedgehog-fn
https://github.com/qfpl/hedgehog-fn
There a loss of precision when calculating the bounds of large ranges of long. In some cases this can actually result in an overflow and a range which actually shrinks as it grows.
For example:
> val positiveLongs = Range.linear(1, Long.MaxValue)
> positiveLongs.bounds(Size(100))
(1, 1)
Which should instead be:
(1, 9223372036854775807)
hedgehogqa/haskell-hedgehog#281
The F# version is apparently correct:
The Scala version might have the same problem:
hedgehogqa/haskell-hedgehog#313
scala-hedgehog/core/src/main/scala/hedgehog/Gen.scala
Lines 167 to 176 in 6d369c1
Please don't close this issue until Scala 3.0.0 is released and the version of Hedgehog supporting it is released.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.