hedgehogqa / fsharp-hedgehog-experimental Goto Github PK
View Code? Open in Web Editor NEWHedgehog with batteries included: Auto-generators, extra combinators, and more.
License: Other
Hedgehog with batteries included: Auto-generators, extra combinators, and more.
License: Other
It looks like the library consists of two parts:
Hedgehog
. These are all these sorted
, noNull
, shuffle
, uri
generators and operations on themAutoGenConfig
, GenX.auto
, etc.)These parts do not seem to depend on each other.
The questions are:
Gen
/Range
combinators and constructors? Some of them may even go to HH itself.Hedgehog.Autogen
can be born as its own library?It has been a while since I have been a maintainer of this package, but I just noticed that NuGet publishes are still done in my name, presumably because it's using my API key:
@TysonMN, I understand you to be the current primary maintainer of this package. I have just added you as an owner of the NuGet package. Whenever it's convenient for you, consider creating and using your own API key and removing me as a NuGet package owner.
Given that
I suggest that we
Advantages
Transferring this repository to @cmeeren can make it move a little bit faster, as right now the issues and the pull requests can be blocked several weeks waiting feedback from us.
Of course, I'm more than happy to help with any questions on issues and pull requests (given that I have the answer(s)), but I will no longer be the maintainer of this repository.
By not being the maintainer of fsharp-hedgehog-experimental, I may put my (limited) spare time into fsharp-hedgehog itself and even the Haskell version (where we're missing the diehard tests, for example).
Disadvantages
Are there any?
Practical issues
I'd really like this repository to be moved to @cmeeren's GitHub, (mine or some other's, if @cmeeren isn't willing to take over).
Moving fsharp-hedgehog-experimental outside @hedgehogqa amplifies that fact that this repository is all about taking great ideas from the community and grouping them altogether for trying them out, like:
I've discussed this with @jystic and he's positive, so we really look forward your decision @cmeeren ๐ฅ
@moodmosaic CI is not building. I think it's because AppVeyor doesn't have access to the hedgehogqa org. I have sent a request for access.
This build said it had an error, but the NuGet package was successfully published anyway.
@dharmaturtle, can you links/information you found about this issue?
It may help people discover it more easily! I think anyone using fsharp-hedgehog at least ought to know about this repo and its auto gen capability.
The only 'requirement' is that at least @jacobstanley and/or myself would need to have push access. But don't let that scare you, we work via pull requests all the time.
Hey!
I was hoping to create a generator that enforces a unique string is given to a certain field, is this something that already exists or do I need to write the logic?
Thanks :)
Why isn't there support when Array.Rank
is greater than 1
? Is there some fundamental technical difficulty or is it just that sufficient time/effort was never spent to implement that case?
fsharp-hedgehog-experimental/src/Hedgehog.Experimental/Gen.fs
Lines 421 to 431 in 439159c
In commit b52c847, I added [<CLIMutable>]
and [<Struct>]
to AutoGenConfig
, with the motivation to make it easier to use from C#. I can understand how CLIMutable
achieves that goal, but I can't figure out why Struct
should make it easier.
Furthermore, since AutoGenConfig
has so many fields, having it be a Struct
seems like it could be bad for performance (I have not measured).
Could anyone illuminate me? Is Struct
relevant for C# interop?
Hi - since I found no repo for the homepage, I will just post this here, in my favorite repo ;)
That link here directs to the old homepage, I imagine this is accidental?
@dharmaturtle, why did this build for a merge commit fail?
Just noticed (from static reading of the code) that the auto-generator seems to always generate DateTimeOffset
values with zero offset:
If that is the case, it is probably a good idea to generate various offsets. Perhaps using Gen.dateTimeOffset
?
This is not an important issue for me, and I can't prioritize making a PR right now, but I wanted to mention it since I saw it.
Do you think that support for Nullable<'T>
can be added, similarly to how Option<'T>
is now supported?
It is useful for property-testing with types that are defined in C#.
I am using collections from System.Collections.Immutable
(ImmutableDictionary
in my case) and am getting the following exception:
System.Exception: Class System.Collections.Immutable.ImmutableDictionary`2[System.String,System.String] lacking an appropriate ctor
System.Exception
Class System.Collections.Immutable.ImmutableDictionary`2[System.String,System.String] lacking an appropriate ctor
at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1433.Invoke(String message) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\printf.fs:line 1433
at Hedgehog.GenX.mkRandomMember@397-2.TypeShape-Core-Core-IMemberVisitor`2-Visit[b](ShapeMember`2 shape) in /home/runner/work/fsharp-hedgehog-experimental/fsharp-hedgehog-experimental/src/Hedgehog.Experimental/Gen.fs:line 400
at Hedgehog.GenX.autoInner[a](AutoGenConfig config, FSharpMap`2 recursionDepths)
at Hedgehog.GenX.autoInner@525-40.TypeShape-Core-Core-IConstructorVisitor`2-Visit[CtorParams](ShapeConstructor`2 ctor) in /home/runner/work/fsharp-hedgehog-experimental/fsharp-hedgehog-experimental/src/Hedgehog.Experimental/Gen.fs:line 528
at InternalLogic.genxAutoBoxWith[T](AutoGenConfig x)
which points to here
Since immutable collections are part of BCL, it would be beneficial to support them.
Would it be possible to do?
Currently, the build on Travis CI fails at:
$ ./validate.sh && ./build.sh
/home/travis/.travis/job_stages: line 57: ./validate.sh: Permission denied
As shown in https://travis-ci.org/hedgehogqa/fsharp-hedgehog-experimental#L1823/.
/cc @jystic
PR #21 added a configuration value called RecursionDepth
to prevent stack overflows when generating a recursive type (c.f. #19 (comment)).
FsCheck solves this problem by relating the recursion depth to the size parameter.
However, a recursive generator like this may fail to terminate with a
StackOverflowException
, or produce very large results. To avoid this, recursive generators should always use the size control mechanism
This seems like a better solution to me, but I am just getting seriously into property-based testing, so I am probably unaware of all the considerations.
What are the tradeoffs for this configuration value vs utilizing the size parameter?
Making a PR of the auto-generator now. Where do you want it placed? IMHO it should be accessible as Hedgehog.Gen.auto
, but the implementation is fairly substantial, so I'm wondering if it should be placed in its own file (and if so, how to do that considering that Hedgehog.Gen is already defined in Gen.fs) or if it's okay to have everything in Gen.fs (which might be the only option).
There needs to be a reference to the main Hedgehog library. I don't know what kind of constraints you want on the version number, so I'll leave it to you.
e.g.
let! foo = GenX.auto<uint64>
Fails with
System.Exception : Class System.UInt64 lacking an appropriate ctor
at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1647.Invoke(String message)
โฆ
at [email protected](Unit x)
at [email protected](Unit x)
at [email protected](Seed seed, Int32 size)
at Hedgehog.Property.loop@332-21(Int32 n, Random`1 random, Seed seed, Int32 size, Int32 tests, Int32 discards)
at Hedgehog.Property.Report(Int32 n, Property`1 p)
at Hedgehog.Property.Check(Property`1 p)
This also affects discriminated unions with uint64 cases.
I have an untracked file .paket\Paket.Restore.targets
. We should either check this in or add it to .gitignore (I messed up #24 because of it).
It would be great if the user could specify overrides for any complex type they desire. For example, say Customer
is a type that has a field of type Customer list
(sub-customers of the current customer). This is recursive and (currently) needs a specifically designed generator to avoid stack overflow. Say then that I want to generate an Order
type which, among other things, contains a Customer
, then I'd want to just say auto<Order>(...)
and somehow specify that I want Customer
objects to be generated using my specific generator, and otherwise the defaults are fine.
I'm not sure how to do this though, or if it is easy/feasible/possible.
We might want to convert this project to .NET Standard too, see hedgehogqa/fsharp-hedgehog#153
Similarly to what FSCheck is doing
Can we consider adding types like:
NegativeInt
NonNegativeInt
PositiveInt
NonZeroInt
NormalFloat
NonEmptyString
StringNoNulls
NonWhiteSpaceString
XmlEncodedString
UnicodeChar
UnicodeString
Interval
IntWithMinMax
NonEmptySet
NonEmptyArray
FixedLengthArray
to AutoGenConfig
so that they are available by default?
I know that they can be defined separately and configured via Property
attribute, but since they are very generic and very useful having them "out-of-the-box" would be extremely convenient...
Since auto<'a>()
has an explicit type parameter, I think it's possible to remove the unit parameter (if it's possible, then IMHO we should do it since it's just unnecessary noise). It compiled fine when I tried it, and all tests completed successfully.
Waiting with PR until after #21 is merged due to conflicts.
module HedgehogExperimentalRepro.Tests
open System
open Hedgehog
open FsUnitTyped
[<Test>]
let ``can make timespan`` () =
property {
let! (test : TimeSpan) = GenX.auto
test.Seconds |> shouldBeSmallerThan 100
} |> Property.check
fails with the exception
System.OverflowException : Value was either too large or too small for an Int64.
at System.Numerics.BigInteger.op_Explicit(BigInteger value)
at <StartupCode$Hedgehog>[email protected](BigInteger x)
at [email protected](Int32 sz)
at [email protected](Seed seed, Int32 size)
at [email protected](Seed seed, Int32 size)
at [email protected](Seed seed0, Int32 size)
at [email protected](Seed seed0, Int32 size)
at Hedgehog.Property.loop@332-21(Int32 n, Random`1 random, Seed seed, Int32 size, Int32 tests, Int32 discards)
at Hedgehog.Property.Report(Int32 n, Property`1 p)
at Hedgehog.Property.Check(Property`1 p)
at HedgehogExperimentalRepro.Tests.can make timespan()
Hi, I have found one of the edge cases in which GenX
fails to generate a correct instance, as well as the possible solution to the issue.
Assuming that we have a hierarchy of two types:
public abstract class A { public object Value { get; protected set; } }
public class B : A { public new string Value { get { return (string)base.Value; } set { base.Value = value; } } }
Notice that B
has a property with the same name as A
, but "hides" the base property with the new
keyword.
Now, when we ask GenX.auto<B>
then it generates an incorrect instance because Value
property gets obj
as a value and not string
.
It is happening because this code:
| Shape.CliMutable (:? ShapeCliMutable<'a> as shape) ->
let props = shape.Properties |> Seq.toList
gives us the list of two properties in this order:
Value
that is declared in B
Value
that is declared in A
and values for these properties are generated and set in this order.
So first, GenX
will generate a correct
(by intention) value and will set it to B.Value
, but then it will generate an empty object and pass it into the "hidden" property, which overwrites the correct value with an incorrect one.
The list of properties comes to us from TypeShape
, which just reflects all the instance properties in a given type:
and [<Sealed>] ShapeCliMutable<'Record> private (defaultCtor : ConstructorInfo) =
let properties = lazy(
typeof<'Record>.GetProperties(AllInstanceMembers)
|> Seq.filter (fun p -> p.CanRead && p.CanWrite && p.GetIndexParameters().Length = 0)
|> Seq.map (fun p -> mkWriteMemberUntyped<'Record> p.Name p [|p|])
|> Seq.toArray)
GetProperties
method explicitly does not specify the order in which properties are returned:
The GetProperties method does not return properties in a particular order, such as alphabetical or declaration order. Your code must not depend on the order in which properties are returned, because that order varies.
so that we cannot rely on "this instance" property being passed last (and it does, indeed, not happen).
Since we cannot rely on the order, one possible way of working around the issue would be to make sure that properties of "base instance" are set first, and properties of "this instance" are set last.
This would achieve that:
| Shape.CliMutable (:? ShapeCliMutable<'a> as shape) ->
let selfProps, baseProps =
shape.Properties
|> Seq.toList
|> List.partition (fun x -> x.MemberInfo.DeclaringType = typeof<'a>)
(baseProps @ selfProps) // <---- here is the change
|> Seq.toList
|> ListGen.traverse memberSetterGenerator
|> Gen.map (fun fs -> fs |> List.fold (|>) (shape.CreateUninitialized ()))
and this way GenX
will produce "correct by intention" value.
Of course, a "better" solution would be to detect new
and to remove "hidden" members from the list, but:
So I think that the solution with setting "this instance" properties first should be good enough, and should definitely be better to what we currently have.
Will you accept a PR that fixes this issue in this way? ;)
It'd be nice with generators that permute existing sequences/lists/strings:
/// Generates a permutation the specified list (shuffles its elements).
permutationOf (list : 'a list) : Gen<'a list>
/// Randomizes the case of the characters in the string.
casePermutationOf (str : string) : Gen<string> =
See the combinators removed in this commit. An open question is how to handle the randomness generation - System.Random might not be the correct choice.
I am running a test with a simple condition:
match message with
| Happy s when s.Reason.Contains('a') -> Nack
| _ -> Ack
I see in debug that it fails properly and the value is shrunk properly to the failure case of a single a
character (see the screenshot)
I also see in debug that this is the last shrink, i.e. the test is not getting called anymore.
However, the result of the test is printed as:
*** Failed! Falsifiable (after 68 tests and 6 shrinks):
[Happy { "reason": "`", "level": -287093 }]
Which is not a correct failing condition, it should say "reason": "a"
instead!
Interestingly, I also have a numeric field in that structure, and when I do something like
| Happy s when s.Level = 97 -> Nack
Then the failure is reported correctly. So it looks like it is working for numbers, but not for strings
Using HH
from C#
and this problem arises:
[Property]
public void SomeTest(List<int> values) { ... }
Then running this test fails with this exception:
System.ArgumentOutOfRangeException
capacity was less than the current size. (Parameter 'value')
at System.Collections.Generic.List`1.set_Capacity(Int32 value)
Perhaps a bug in a List
generator?
@cmeeren, do you think it'd be possible to use the new Hedgehog logo also in the Hedgehog.Experimental NuGet package?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.