Giter VIP home page Giter VIP logo

Comments (1)

onsi avatar onsi commented on July 3, 2024

thanks @ivelichkovich

the simplest way to do this would be to orchestrate (i.e. shell out to) the ginkgo cli to have it run suites for you on demand. this gives you clean isolation, will support parallelization, and allows you to write ginkgo suites as you're used to. of course the downside is the need to distribute your binary with additional test binaries (you could include precompiled test binaries, it wouldn't have to be source code, but I get that this is ugly).

If you want a way to invoke ginkgo specs in-process as code this is possible but we'd need to add some additional tooling to make it a decent experience. The only thing that would be very challenging to do would be to run the specs in parallel. This is because Ginkgo's shared memory model (the different closures in a container hierarchy communicate via shared variables) doesn't translate directly to an in-process parallelization model. But if you're ok running specs in serial then read on.

Today, when call Describe or BeforeEach or It, Ginkgo creates a node and attaches it to the current testing tree in a single global Suite object. This is all done under the hood for you so you don't have to bother passing a Suite instance around. Since Ginkgo's primary mode of operation is as a test runner of go test suites this limitation works just fine: each test package is assumed to exist in one singular suite so globbing stuff onto the share global variable as the code is initialized makes sense.

This gets tricky, however, if you want to use Ginkgo as a library and (potentially) have multiple suites - or be able to run a single suite multiple times.

Ginkgo's own internal_integration suite solves this by (swapping out the global suite)[https://github.com/onsi/ginkgo/blob/master/internal/internal_integration/internal_integration_suite_test.go#L75-L82] object on-demand (look here for an example).

Along these lines I could imagine a new package under ginkgo/extensions that would let you do stuff like:

import "github.com/onsi/ginkgo/extensions/suite"
...

// the entry point for our code - this will construct
// a suite, configure it to use the passed-in label filter
// and return a report
func RunSpecs(labelFilter string) (spec.Report, error) {
    s := suite.New()
    gomega.RegisterFailHandler(ginkgo.Fail) // register Gomega's global fail handler

    //build the test tree - you can imagine pulling the callback into a separate package
    // you could call AppendSpecs multiple times
    s.AppendSpecs(func() {
        Describe("specs go here", func() {
            It("...", func() {
                  Expect("foo").To(Equal("bar"))
            })
        })
    })

    // grab and reconfigure the suite configuration    
    conf :=  s.SuiteConfiguration()
    conf.LabelFilter = labelFilter
    
     // run the suite
     passed, report, err := s.Run(conf)
     return report, err
}

There are some important caveats here - since AppendSpecs must swap out Ginkgo's global suite object you must not call AppendSpecs in multiple goroutines at the same time. Also since Gomega's assertions play a similar global trick you must not call s.Run in multiple goroutines at the same time.

We could implement workarounds for both of these limitations but they would require you to pass the suite object and a particular gomega instance around. I think this could get ugly quickly (but it's the sort of pass-things-around boilerplate that Go users are used to):

func MySpecs(s suite.Suite, g gomega.Gomega) {
    s.Describe("specs go here", func() {
        s.It("...", func() {
             g.Expect("foo").To(Equal("bar"))
        })
    })
}

As you can see the caveats start to pile up and I worry that a user new to all this will struggle to make sense of it.


So i wonder if we should explore some alternatives. Perhaps a better approach would be tooling that can handle calling ginkgo for you and returning the report. This would allow you to run things in parallel and to reuse suites in various contexts. It doesn't solve the packaging and distribution problem - but perhaps there are some creative options there? How crazy would it be to precompile the test binaries and the ginkgo cli and embed them into your executable (answer: probably very crazy).

from ginkgo.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.