Comments (1)
thanks @ivelichkovich
the simplest way to do this would be to orchestrate (i.e. shell out to) the ginkgo cli to have it run suites for you on demand. this gives you clean isolation, will support parallelization, and allows you to write ginkgo suites as you're used to. of course the downside is the need to distribute your binary with additional test binaries (you could include precompiled test binaries, it wouldn't have to be source code, but I get that this is ugly).
If you want a way to invoke ginkgo specs in-process as code this is possible but we'd need to add some additional tooling to make it a decent experience. The only thing that would be very challenging to do would be to run the specs in parallel. This is because Ginkgo's shared memory model (the different closures in a container hierarchy communicate via shared variables) doesn't translate directly to an in-process parallelization model. But if you're ok running specs in serial then read on.
Today, when call Describe
or BeforeEach
or It
, Ginkgo creates a node and attaches it to the current testing tree in a single global Suite
object. This is all done under the hood for you so you don't have to bother passing a Suite
instance around. Since Ginkgo's primary mode of operation is as a test runner of go test suites this limitation works just fine: each test package is assumed to exist in one singular suite so globbing stuff onto the share global variable as the code is initialized makes sense.
This gets tricky, however, if you want to use Ginkgo as a library and (potentially) have multiple suites - or be able to run a single suite multiple times.
Ginkgo's own internal_integration suite solves this by (swapping out the global suite)[https://github.com/onsi/ginkgo/blob/master/internal/internal_integration/internal_integration_suite_test.go#L75-L82] object on-demand (look here for an example).
Along these lines I could imagine a new package under ginkgo/extensions
that would let you do stuff like:
import "github.com/onsi/ginkgo/extensions/suite"
...
// the entry point for our code - this will construct
// a suite, configure it to use the passed-in label filter
// and return a report
func RunSpecs(labelFilter string) (spec.Report, error) {
s := suite.New()
gomega.RegisterFailHandler(ginkgo.Fail) // register Gomega's global fail handler
//build the test tree - you can imagine pulling the callback into a separate package
// you could call AppendSpecs multiple times
s.AppendSpecs(func() {
Describe("specs go here", func() {
It("...", func() {
Expect("foo").To(Equal("bar"))
})
})
})
// grab and reconfigure the suite configuration
conf := s.SuiteConfiguration()
conf.LabelFilter = labelFilter
// run the suite
passed, report, err := s.Run(conf)
return report, err
}
There are some important caveats here - since AppendSpecs
must swap out Ginkgo's global suite object you must not call AppendSpecs
in multiple goroutines at the same time. Also since Gomega's assertions play a similar global trick you must not call s.Run
in multiple goroutines at the same time.
We could implement workarounds for both of these limitations but they would require you to pass the suite object and a particular gomega instance around. I think this could get ugly quickly (but it's the sort of pass-things-around boilerplate that Go users are used to):
func MySpecs(s suite.Suite, g gomega.Gomega) {
s.Describe("specs go here", func() {
s.It("...", func() {
g.Expect("foo").To(Equal("bar"))
})
})
}
As you can see the caveats start to pile up and I worry that a user new to all this will struggle to make sense of it.
So i wonder if we should explore some alternatives. Perhaps a better approach would be tooling that can handle calling ginkgo
for you and returning the report. This would allow you to run things in parallel and to reuse suites in various contexts. It doesn't solve the packaging and distribution problem - but perhaps there are some creative options there? How crazy would it be to precompile the test binaries and the ginkgo cli and embed them into your executable (answer: probably very crazy).
from ginkgo.
Related Issues (20)
- Junit report should have system-out HOT 1
- could not log in Eventually with fmt.Fprintf(GinkgoWriter) HOT 3
- Question: One Suite to rule them all HOT 2
- when using concurrency, a test case failure in a background go routine does not show which test case failed HOT 3
- Add cli flag to fail when no tests were run HOT 1
- `ginko -p 2` does not complain about having an extra argument
- How to integrate with xgo? HOT 7
- when watching, add ability to ignore hidden files (including tests!) HOT 5
- Report Portal and Gingko integration plugin
- Implement generic transformer decorator: ginkgo.ContextTransformer HOT 3
- why can't generate html report? HOT 1
- CLI: add long aliases for short options
- --focus-file should work with failing test location HOT 6
- panic when passing context AND nil in a table Entry HOT 2
- Add support for Label Sets
- support outputting the time spent on each spec HOT 1
- [Documentation] Fix the wrong variable call in Section "Reporting Nodes - ReportBeforeSuite and ReportAfterSuite"
- It function use only check last test case param HOT 1
- Run tests in individual process HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ginkgo.