franela / goblin Goto Github PK

Perhaps I am just not searching for the right terms, so I am going to try to explain what I am trying to accomplish...

I am currently adding tests to github.com/TJM/go-trello and I have hit a conditions multiple times where tests need to depend on the previous test. For example:

• Create Board
• Set Board to red
• Set Board Description
  (etc)

If "Create Board" fails, the other two need to "skip" or "fail fast" or something, but currently it just blows up and I get the go equivalent of a null pointer exception.

I was thinking of adding an assertion at the top, which should cause the test to fail at least (better than the crash I was getting before), but the failure state would look better if it showed them as dependent tests that were skipped :-/

// NOTE: "client" is setup in "init()" currently, but could probably be moved to a g.Before()?

		g.It("should create a board", func() {
			board, err = client.CreateBoard(testBoardName)
			Expect(err).To(BeNil())
			Expect(board).NotTo(BeNil())
			Expect(board.Name).To(Equal(testBoardName))
		})

		g.It("should get a board by ID", func() {
			Expect(board).NotTo(BeNil()) // Create Board needs to have succeeded
			board, err = client.Board(board.ID)
			Expect(err).To(BeNil())
			Expect(board).NotTo(BeNil())
			Expect(board.Name).To(Equal(testBoardName))
		})

		g.It("should change the board background to red", func() {
			Expect(board).NotTo(BeNil()) // Create Board needs to have succeeded
			err = board.SetBackground("red")
			Expect(err).To(BeNil())
			Expect(board.Prefs.Background).To(Equal("red"))
		})

		g.It("should change the description to something", func() {
			Expect(board).NotTo(BeNil()) // Create Board needs to have succeeded
			err = board.SetDescription("something")
			Expect(err).To(BeNil())
			Expect(board.Desc).To(Equal("something"))
		})

Today I shame myself by not writing properly testing. when the test succeeds everything is good, but when fails a nasty SIGSEGV: segmentation violation appear and there's no clue why this is happening

These are the two examples I just did the first one, later just checking if happen also with the second

1. Describe without It

func Test(t *testing.T) {
    g := goblin.Goblin(t)

    g.Describe("GetUploadS3Params", func() {        
        g.Assert("X").Equal("Y")
    })
}

the panic raises when goblin.go#L344 is executed, currentIt is nil

2. It without Describe

func Test(t *testing.T) {
    g := goblin.Goblin(t)

    g.It("GetUploadS3Params", func() {        
        g.Assert("X").Equal("Y")
    })
}

the panic raises when goblin.go#L279 is execute, parent is nil

I think would be very nice include some internal error handling when either currentIt or parent is nil

As PR #98 Says, concurrency race panic when running in parallel.

parseFlag() function only need do once, so use sync.Once to do init things.

This way it shows more information on where the test failed exactly and also when integrating with GoMega, it shows the exact line of the test where it failed.

about https://github.com/franela/goblin/blob/master/assertions.go#L23

I am finding some snippets about reflect ,I tried

package main

import (
    "fmt"
    "reflect"
)

func main() {
    fmt.Println(reflect.ValueOf(1) == reflect.ValueOf(1))
}

it shows "FALSE",
reflect.Value underlying structure is pointer,even they get same value,it points to different address.

package foo

import (
    "testing"
    . "github.com/onsi/gomega"
    . "github.com/franela/goblin"
)

trying to run go test produces the following error:

./foo_test.go:6:5: Assertion redeclared during import "github.com/franela/goblin"
        previous declaration during import "github.com/onsi/gomega"
FAIL    foo/foo [build failed]

go.mod

module foo/foo

go 1.14

require (
	github.com/franela/goblin v0.0.0-20200611003024-99f9a98191cf
	github.com/onsi/gomega v1.10.1
        ...
)

Hello. It would be nice if there was a neat, compact way of being able to print the expected and actual values in a font color in keeping with the rest of the printed output. Maybe it can already be done but it's beyond my skill level if it is!

I couldn't find anything about this in the README.md, but whenever I run my tests which all pass, Goblin actually won't output anything and I only see the standard go test output. Is this expected?

Will be cool if we can have the possibility to make it work with goblin gomock, mockery or others.

When executing go test against multiple packages, I am not getting any output other than the standard go test output. None of the Describe & It descriptions are being output to the terminal.

$ go test github.com/thedodd/<pkg> github.com/thedodd/<pkg>/lib
?   github.com/thedodd/<pkg>      [no test files]
ok  github.com/thedodd/<pkg>/lib  0.195s

There should be a way to ignore tests if necessary. A great use case for this is a situation where you need to account for a test in the future, but it doesn't apply now. Your choices at the moment are to either delete it or comment the test out. I would prefer to just be able to ignore it. Another great use-case is integration tests with third parties that you don't always want to run. You could implement it like so:

g.Ignore("Should add two numbers ", func() {
    g.Assert(1+1).Equal(2)
})

Testing frameworks in Scala have a similar feature.

On Debian amd64 TestResolver fails as follows with Golang-1.6:

=== RUN   TestResolver 

  Resolver 
    1) Should resolve the stack until the test 


 0 tests complete (0 ms) 
 1 tests failed:  

  1) Resolver Should resolve the stack until the test: 

    6 does not equal 3 
        /build/golang-github-franela-goblin-0.0.1+git20160123.62.889391d/obj-x86_64-linux-gnu/src/github.com/franela/goblin/resolver.go:9 +0x29 
        /build/golang-github-franela-goblin-0.0.1+git20160123.62.889391d/obj-x86_64-linux-gnu/src/github.com/franela/goblin/goblin.go:281 +0x2c 
        /build/golang-github-franela-goblin-0.0.1+git20160123.62.889391d/obj-x86_64-linux-gnu/src/github.com/franela/goblin/goblin.go:272 +0x34 
        /build/golang-github-franela-goblin-0.0.1+git20160123.62.889391d/obj-x86_64-linux-gnu/src/github.com/franela/goblin/assertions.go:43 +0x215 
        /build/golang-github-franela-goblin-0.0.1+git20160123.62.889391d/obj-x86_64-linux-gnu/src/github.com/franela/goblin/resolver_test.go:19 +0xec 
        /build/golang-github-franela-goblin-0.0.1+git20160123.62.889391d/obj-x86_64-linux-gnu/src/github.com/franela/goblin/resolver_test.go:12 +0x20 
        /build/golang-github-franela-goblin-0.0.1+git20160123.62.889391d/obj-x86_64-linux-gnu/src/github.com/franela/goblin/goblin.go:186 +0x1d 
        /build/golang-github-franela-goblin-0.0.1+git20160123.62.889391d/obj-x86_64-linux-gnu/src/github.com/franela/goblin/goblin.go:186 +0x175 
--- FAIL: TestResolver (0.00s) 

I realized after #85 that I made a false assumption: I though multiple messages could be displayed but as it turns ou only one failure can be stored for each test.

The issue is that gomock does that:

failures := ctrl.expectedCalls.Failures()
for _, call := range failures {
  ctrl.T.Errorf("missing call(s) to %v", call)
}
if len(failures) != 0 {
  ctrl.T.Fatalf("aborting test due to missing call(s)")
}

With the Errorf methods introduced by the original testify patch each successive call to Errorf and then Fatalf overwrite the previous one.

Adding support for multiple erros would allow logging non fatal errors like this:

g.Errrof("this should take less than 10ms")
g.Fatalf("X is different than 10")

I can give it a try but is that something that could be merged ?

I cannot see the regexp flag -goblin.run working unless I comment out the flag.Parsed() check in the Goblin constructor. In my tests it seems to always be true even though I do not see where else the flag.Parse() is called.

func Goblin(t *testing.T, arguments ...string) *G {
	// if !flag.Parsed() {
	parseFlags()
	// }

Removing the check seems to work well though.

The command I run is as follows:

$ go test ./pkg/... -v -goblin.run="regexp"

OR

$ go test ./pkg/... -v -args -goblin.run="regexp"

EDIT: I am using go1.11

When I run go test
goblin shows a nice output (along with all the passing tests)

But when there are multiple packages, I need to run go test <package-1> <package-2>
In this case, goblin does not show entire output & just shows
ok github.com/ameykpatil/<package-name> 22.832s

I don't even come to know how many total tests were there.

How can I get entire output in case of go test <package-1> <package-2>?
(I am a beginner in go world, I am not really sure if this is an issue with goblin or something else)

I use go1.3.3 windows/amd64.

go get github.com/franela/goblin output:

$GOPATH\src\github.com\franela\goblin\util.go:10: undefined: syscall.Termios
$GOPATH\src\github.com\franela\goblin\util.go:11: undefined: syscall.SYS_IOCTL
$GOPATH\src\github.com\franela\goblin\util.go:11: undefined: ioctlReadTermios
$GOPATH\src\github.com\franela\goblin\util.go:11: not enough arguments in call to syscall.Syscall6

So we can do:

  g.Assert(foo).Exists()

This should decide what to do for each type.

When using the testify framework, I would expect to see the output match the style in the readme (mocha-like) but instead there it kinda looks verbose, uncoloured, and doesn't register as an error in Goblin. Like this:

I was wondering if this is by design / not implemented / I am doing something very wrong.

Here is the code for reference:

func TestItem(t *testing.T) {
	g := Goblin(t)

	g.Describe("Item CRUD", func() {
		g.Describe("Should read", func() {
			g.It("An item by name ", func() {
				assert.Equal(t, "expected", "actual")
			})
		})
	})
}

Hello

adding goblin via dep cli dep ensure -add github.com/franela/goblin will install 0.0.1 tag version. One can change it by setting proper branch in Gopkg.toml to:

[[constraint]]
  name = "github.com/franela/goblin"
  branch = "master"

but it needs some thinking before finding out what's wrong.

From time to time go test -v command fails with

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: interface conversion: interface is runtime.errorString, not string [recovered]
    panic: interface conversion: interface is runtime.errorString, not string
[signal 0xb code=0x1 addr=0x0 pc=0x3b924]

goroutine 4 [running]:
testing.func·004()
    /usr/local/Cellar/go/1.1/src/pkg/testing/testing.go:348 +0xcd
github.com/franela/goblin.func·001()
    ~/src/github.com/franela/goblin/goblin.go:156 +0x9d

Unfortunately, I can't find appropriate scenario to reproduce this situation, but it happens quite often.

Use parameter provided by issue #8

fail:
g.Assert(bufHash32("")).Equal(0x0)

pass:
g.Assert(bufHash32("")).Equal(uint32(0x0))

plz add ability to use "0x%x" format. and view result in same format.

I found that this project already supports Go modules. But sadly, the tags doesn't follow Semantic Versioning, which means that all tags of this project will be ignored by Go modules and replaced by pseudo-versions, go get acts weirdly when tags are not in that form. It would be great to have the tagged release be named in the format vX.X.X format so that go mod can read it.

	github.com/franela/goblin v0.0.0-20200722185118-cb67619f1d10

Else the mod file shows something like github.com/franela/goblin v0.0.0-20200722185118-cb67619f1d10 which is not very readable and difficult to upgrade. It’s hard to verify which version is in use. This is not conducive to version control

So, I propose this project to follow Semantic Versioning in future versions. For example, v1.0.1, v2.0.0, v3.1.0-alpha, v3.1.0-beta.2etc.

Nice library, I'm loving the coloured output and the BDD style assertions.

Just wondering if there is a NotEqual assertion equivalent available or planned?

I'm running go test in this repo and the test suite reports a bunch of failures but I'm not sure if they are related to actual bugs or part of the testing.
I'd like to add a new simple feature but the fact that I'm not sure if the test suite is passing or failing is a bit worrying.

It would be great to have excluding of tests, for example, by renaming It to Xit:

    // This test will be excluded and not run
     g.Xit("Should add two numbers ", func() {
            g.Assert(1+1).Equal(2)
        })

And vice versa, force a single test or test suite to run by renaming It to Fit:

    // Only this tests will run 
     g.Fit("Should add two numbers ", func() {
            g.Assert(1+1).Equal(2)
        })

Thanks!

It will run all of the assertions and the last one is the one that will show up.

Hi Goblin,

Unsure what Before and After mean (after all or before all or after each or before each)? Can we add some documentation on the After block to provide that. Happy to submit a pull request, just need clarification myself.

Thanks for the great framework. Go was really missing a clean lightweight solution for BDD.

I'd like to run particular group of tests described via Describe(), just for developing purposes when I want to debug entire group, so i would really appreciate a separate flag, e.g -goblin.runFullName with the following behavior:

diff --git a/goblin.go b/goblin.go
index a4617db..078965c 100644
--- a/goblin.go
+++ b/goblin.go
@@ -284,8 +284,30 @@ func (g *G) SetReporter(r Reporter) {
        g.reporter = r
 }
 
+func (g *G) GetFullName(name string) string {
+       parent := g.parent
+       names := []string{name}
+
+       for parent != nil {
+               names = append(names, parent.name)
+               parent = parent.parent
+       }
+
+       result := ""
+       for _, n := range names {
+               if len(result) == 0 {
+                       result = n
+                       continue
+               }
+
+               result = n + "." + result
+       }
+
+       return result
+}
+
 func (g *G) It(name string, h ...interface{}) {
-       if matchesRegex(name) {
+       if matchesRegex(g.GetFullName(name)) {
                it := &It{name: name, parent: g.parent, reporter: g.reporter}
                if g.parent == nil {
                        panic(fmt.Sprintf("It(\"%s\") block should be written inside Describe() block.", name))

This can be done using func(done Done)
When Goblin detects that the body is of this type it will wait for an exception or the call to done, or timeout.

I don't have a reproduction for this bug yet, but here is a quick explanation that might make sense:

I have a nested multi describe test, in the last test of the tree, I do a for loop.

type expectation struct {
 Id int
 List []string
}

// construct an array of expectations

for i, exp := range expectations {
   g.It("each thing must be correct", func() {
    thing := things[i]
    t.Log(len(thing.List)) // always prints 0
    t.Log(len(exp.List)) // always print 0
  })
}

I'm very confused why as to why even tho I construct my array with filled up lists, I never get the content inside the scope of the test.
However if I move the test function like that:

g.It("each thing must be correct", func() {
  for i, exp := range expectations {
    thing := things[i]
    t.Log(len(thing.List)) // prints the right value
    t.Log(len(exp.List)) // prints the right value
  }
})

I'm sure this is something dumb I should know about golang, but that's a very surprising behavior. Any idea what's going on?

Running table-driven tests with proper descriptions seems buggy (if it's even supported). If you range over test data within a Describe block, it runs each test with the very last data set in the range, giving invalid test results.

// Example of how it has to be done, but requires less-than optimal scoping
func TestFoo(t *testing.T) {
	g := Goblin(t)

	tests := []struct {
		Description string
		Input       bool
		Output      bool
	}{
		{
			Description: `should fail`,
			Input:       false,
			Output:      false,
		},
		{
			Description: `should pass`,
			Input:       false,
			Output:      true,
		},
	}

	for _, t := range tests {
		g.Describe(`foo()`, func() {
			g.It(t.Description, func() {
				out := foo(t.Input)
				g.Assert(out).Equal(t.Output)
			})
		})
	}
}

The above example is how you're required to write table-driven tests, which makes test output ugly by outputting the describe block for every iteration, and also makes scoping of tests more difficult (in terms of After, Before, etc). In this case, the first test will fail & the second test will pass, as expected.

// Example of how describe blocks should work, but breaks range
func TestFoo(t *testing.T) {
	g := Goblin(t)

	g.Describe(`foo()`, func() {
		tests := []struct {
			Description string
			Input       bool
			Output      bool
		}{
			{
				Description: `should fail`,
				Input:       false,
				Output:      false,
			},
			{
				Description: `should pass`,
				Input:       false,
				Output:      true,
			},
		}

		for _, t := range tests {
			g.It(t.Description, func() {
				out := foo(t.Input)
				g.Assert(out).Equal(t.Output)
			})
		}
	})
}

The second case is how I would expect the nesting of Describe blocks to work, but in this case, the tests will not run properly. Both tests will pass, running both iterations of the test with the last item in the data set. This is not a valid test result, as we're expecting the first test to fail.

The foo function in this example just accepts a boolean and inverts it.

func foo(input bool) bool {
	return !input
}

Is there any reason why tests using data ranges within a Describe block should break?

Implement something like g.Time(int) that will override the default max execution time for a specific test.

This feature depends on #8

Trying the below code. It should actually fail on first It as it has expected and actual values different.

import (
	"testing"

	. "github.com/franela/goblin"
)

func Test(t *testing.T) {
	g := Goblin(t)
	// RegisterFailHandler(func(m string, _ ...int) { g.Fail(m) })
	g.Describe("Holly tessst", func() {
		loopableLoop := []struct {
			expected string
			actual   string
		}{{
			expected: "1",
			actual:   "2",
		},
			{
				expected: "2",
				actual:   "2",
			},
		}

		for _, te := range loopableLoop {
			g.It(te.expected, func() {
				g.Assert(te.expected).Equal(te.actual)
			})
		}
	})
}

I'd first like to say thanks for all the efforts and time spent putting together such a useful project.

I use Goblin in almost every single Go project I ever write, and have noticed every time that I vendor my projects there is always the Goblin logo somewhere in that PR. It finally lead me to general curiosity of wondering how many instances of the goblin_logo.jpg and goblin_output.png are in my stuff.

I see that you've done a great job of reducing the size of those files, however it eventually add up. A look across my system right now with find $GOPATH/src -name "goblin_output.png" | wc -l says 78 of each. Napkin math says thats 4,251 KB of images vendored on my system right now.

If you were to move the images to a Github Pages repo like franela.github.io/goblin/images, and link it here that would be an easy win to remove the needless duplication of those images as Goblin is vendored across the Go landscape. All in all, I understand it's not terribly important and you may not wish to bother taking the time. I only wanted to point it out as a possibility to consider.

Gomega handles assertion failures exactly like Goblin's internal assertion library handles assertion failures: by calling a fail method that is provided to it (in Goblin's case it is g.fail which does not panic).

It would be incorrect for Gomega to panic because there is no way for the caller to rescue a panic emitted in a called go routine. This would make your asynchronous test example impossible:

  g.Describe("Numbers", func() {
      g.It("Should add two numbers asynchronously", func(done Done) {
          go func() {
              g.Assert(1+1).Equal(2) //If this line panics there's no way for the test framework to catch it!
              done()
          }()
      })
  })

Would you mind changing the language in the Readme to indicate that this really is not an open issue in Gomega any more? Thanks!

total golang noob here. How do you compare type equality w goblin?

var i int = 42
var f float64 = float64(i)
g.Assert(reflect.ValueOf(f).Kind()).Equal("float64")
// FAILS with "float64 does not equal float64"

As per title. Example:

func TestWeirdness(t *testing.T) {
	test := Goblin(t)

	test.Describe("Pointer Weirdness", func() {

		test.It("Should PASS but FAILS", func() {
			var ptr *[]int
			test.Assert(ptr).IsNil()
		})

		test.It("Should PASS but FAILS", func() {
			var ptr *[2]int
			test.Assert(ptr).IsNil()
		})

		test.It("Should PASS but FAILS", func() {
			var ptr *time.Time
			test.Assert(ptr).IsNil()
		})

		test.It("Should FAIL but PASSES", func() {
			var ptr *[]int
			test.Assert(ptr).IsNotNil()
		})

		test.It("Should FAIL but PASSES", func() {
			var ptr *[2]int
			test.Assert(ptr).IsNotNil()
		})

		test.It("Should FAIL but PASSES", func() {
			var ptr *time.Time
			test.Assert(ptr).IsNotNil()
		})
	})
}

For example:

func TestSkipped(t *testing.T) {
    g := goblin.Goblin(t)
    g.Before(func () {
        panic("Tests are skipped, this shouldn't happen")
    })
    g.Xit(func (){ 
        g.Assert(false)
    })
}

This test suite should be a no-op, but will take action in the hooks (Before, After, etc.) even so.

This would be parity with the behavior when no tests are declared but there are hooks present.

My use case is using Before hooks to set up external services for the tests. We also allow a "fast" test mode when the services aren't present (by wrapping It and Xit from an environment flag).

Alternatively: Add a g.SkipIf(bool) to skip entire test suites if the passed value is true/false.