serpapi / turbo_tests Goto Github PK

Run RSpec tests on multiple cores. Like parallel_tests but with incremental summarized output. Originally extracted from the Discourse and Rubygems source code.

Home Page: https://rubygems.org/gems/turbo_tests

License: MIT License

Ruby 100.00%

rspec parallel-tests ruby

turbo_tests's Introduction

TurboTests

turbo_tests is a drop-in replacement for grosser/parallel_tests with incremental summarized output. Source code of this gem is based on Discourse and RubyGems work in this area.

Incremental summarized output doesn't fit vision of parallel_tests author and RSpec doesn't support built-in parallel testing yet. This gem will not be useful once one of the issues above will be implemented.

Why incremental output?

parallel_tests is great, but it messes output:

$ bundle exec rake parallel_tests:spec[^spec/search]
.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 1 minute 6.92 seconds (files took 6.95 seconds to load)
2616 examples, 0 failures

.........................................................................................................................................F........................................................................................................................................F..............................................................................................................................................................................................................................................................................................

Finished in 1 minute 35.05 seconds (files took 6.26 seconds to load)
2158 examples, 2 failures

.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 1 minute 35.05 seconds (files took 6.26 seconds to load)
2158 examples, 0 failures

turbo_tests output looks like regular rspec:

$ bundle exec turbo_tests
..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F........................................................................................................................................F..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 2 minute 25.15 seconds (files took 0 seconds to load)
6873 examples, 2 failures

Installation

Add this line to your application's Gemfile:

gem 'turbo_tests'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install turbo_tests

Usage

Execute tests:

$ bundle exec turbo_tests

Show help:

$ bundle exec turbo_tests -h
Usage: turbo_tests [options]

[optional] Only selected files & folders:
  turbo_tests spec/bar spec/baz/xxx_spec.rb

Options:
    -n [PROCESSES]                   How many processes to use, default: available CPUs
    -r, --require PATH               Require a file.
    -f, --format FORMATTER           Choose a formatter. Available formatters: progress (p), documentation (d). Default: progress
    -t, --tag TAG                    Run examples with the specified tag.
    -o, --out FILE                   Write output to a file instead of $stdout
        --runtime-log FILE           Location of previously recorded test runtimes
    -v, --verbose                    More output
        --fail-fast=[N]
        --seed SEED                  Seed for rspec

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/serpapi/turbo_tests. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the TurboTests project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

turbo_tests's People

Contributors

Stargazers

Watchers

turbo_tests's Issues

Automatically create a database similar to Rails

It'd be good to automatically set up a database similar to Rails. Rails docs.

In Rails, ActiveRecord::Tasks::DatabaseTasks.reconstruct_from_schema is called inside the ActiveSupport::Testing::Parallelization#after_fork_hook. Ref.

TODO

Clarify documentation about DB creation (related to #12)
Execute code or command before executing rspec in a subprocess. First guess is to prepend commands to the command variable.

Originally posted by @pirj in rspec/rspec-rails#2104 (comment):

What was missing (and what Rails provides with its default testing framework):

Active Record automatically handles creating a database

Without createdb myproject2 etc it didn't work. I've seen rails db:test:prepare, but db:create wasn't apparently called.

undefined method `logger' for Async:Module

When running bundle exec turbo_tests.
Ruby 3.1, Async 2.0
Async.logger throws error also in irb console

Capture time to load files

The summary contains files took 0 seconds to load instead of actual time. The goal is to capture it properly.

$ bundle exec turbo_spec
...........................................................................................................................................................................................
...........................................................................................................................................................................................
...........................................................................................................................................................................................
.........................................................................................................................................F.................................................
.......................................................................................F...................................................................................................
...........................................................................................................................................................................................
...........................................................................................................................................................................................
...........................................................................................................................................................................................
...........................................................................................................................................................................................

Finished in 2 minutes 25.15 seconds (files took 0 seconds to load)
6873 examples, 2 failures

Support --only-group command-line option

parallel_tests has an option to distribute tests across multiple CI workers (eg., strategy.matrix for GitHub Actions.)

For instance, bundle exec parallel_test test -n 6 --only-group 5,6.

I want to be able to do the same with turbo_tests: bundle exec turbo_tests -n6 --only-group 6.

Compatibility with parallel_tests v4

Can we get a new release compatible with version 4 of parallel_tests which was released last November? 🙏🏼

Try to replace subprocesses with DRb similar to Rails

Rails uses DRb when tests are parallelized with processes. Ref.

Try to replace Open3.popen3 with DRb and share results here. Probably it'll require a complete rewrite but there's not that much code.

I'm most interested in comparing file-based IPC & JSON-formatted messages with what DRb provides.

First runner process doesn't report (or is skipped entirely)

I've noticed that the results seem to be short of my total suite number when I run with rspec, or parallel_tests, directly. Running in verbose mode, I've tracked it down to the first process, which doesn't seem to report its results (I'm unsure whether it's even running all of the tests).

I've done this by running in verbose mode, and comparing the number of specs run by the first runner with the number that are missing in my output - they always match, regardless of the number of processes I specify.

$ rake parallel:spec
...
87 examples, 0 failures
Took 139 seconds (2:19)

$ bundle exec turbo_tests -v
12 processes for 16 specs, ~ 1 specs per process
...
Finished in 2 minutes 16.4 seconds (files took 0 seconds to load)
68 examples, 0 failures

$ bundle exec turbo_tests -n 2
...
Finished in 39.83 seconds (files took 0 seconds to load)
34 examples, 0 failures

$ bundle exec turbo_tests -n 3
...
Finished in 44.92 seconds (files took 0 seconds to load)
64 examples, 0 failures

$ bundle exec turbo_tests -n 4
...
Finished in 49.09 seconds (files took 0 seconds to load)
69 examples, 0 failures

Simplecov support

SimpleCov tries to automatically determine the test framework / tools being used based on process name and environment variables.

Since this project is based on parallel_tests, for which SimpleCov claims support, I was expecting this to "just work" as well.

Apparently, all that needs to be done is exposing two environment variables to the sub processes, as explained here. Would that be feasible? (Happy to send a PR if you approve.)

Support passing extra options to rspec

I'd like to be able to use this with Guard, like it's possible with to do with parallel_rspec. However this is not currently possible with because guard passes specific options to rspec. Running guard fails with:

[1] guard(main)>
16:04:03 - INFO - Run all
16:04:03 - INFO - Running all specs
/Users/louim/.asdf/installs/ruby/2.6.8/lib/ruby/gems/2.6.0/gems/turbo_tests-1.2.4/lib/turbo_tests/cli.rb:72:in `run': invalid option: --failure-exit-code (OptionParser::InvalidOption)
	from /Users/louim/.asdf/installs/ruby/2.6.8/lib/ruby/gems/2.6.0/gems/turbo_tests-1.2.4/bin/turbo_tests:11:in `<top (required)>'
	from bin/turbo_tests:29:in `load'
	from bin/turbo_tests:29:in `<main>'
16:04:03 - ERROR - Failed: "bin/turbo_tests -f progress -r /Users/louim/.asdf/installs/ruby/2.6.8/lib/ruby/gems/2.6.0/gems/guard-rspec-4.7.3/lib/guard/rspec_formatter.rb -f Guard::RSpecFormatter --failure-exit-code 2  spec" (exit code: 1)

Because turbo_tests choke on the options passed. It would be useful to support possing down options to Rspec via a specific flag.

turbo_tests is not working with Windows

see https://github.com/rubygems/rubygems/actions/runs/5286199021/jobs/9568614121?pr=6739#step:8:27

#<Thread:0x0000026a04cdca10 C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:174 run> terminated with exception (report_on_exception is true):
C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:175:in `initialize': No such file or directory @ rb_sysopen - tmp/test-pipes/subprocess-1 (Errno::ENOENT)
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:175:in `open'
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:175:in `block in start_subprocess'
#<Thread:0x00000[26](https://github.com/rubygems/rubygems/actions/runs/5286199021/jobs/9568614121?pr=6739#step:8:27)a05473df0 C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:174 run> terminated with exception (report_on_exception is true):
C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:175:in `initialize': No such file or directory @ rb_sysopen - tmp/test-pipes/subprocess-2 (Errno::ENOENT)
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:175:in `open'
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:175:in `block in start_subprocess'
C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:217:in `pop': No live threads left. Deadlock? (fatal)
1 threads, 1 sleeps current:0x0000026a04fb9f20 main thread:0x0000026a7eb19e60
* #<Thread:0x0000026a7f01c4[30](https://github.com/rubygems/rubygems/actions/runs/5286199021/jobs/9568614121?pr=6739#step:8:31) sleep_forever>
   rb_thread_t:0x0000026a7eb19e60 native:0x000000000000016c int:0
   C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:217:in `pop'
   C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:217:in `block in handle_messages'
   C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:216:in `loop'
   C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:216:in `handle_messages'
   C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:93:in `run'
   C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:[38](https://github.com/rubygems/rubygems/actions/runs/5286199021/jobs/9568614121?pr=6739#step:8:39):in `run'
   C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/cli.rb:96:in `run'
   C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/bin/turbo_tests:11:in `<top (required)>'
   D:/a/rubygems/rubygems/bundler/spec/support/rubygems_ext.rb:103:in `load'
   D:/a/rubygems/rubygems/bundler/spec/support/rubygems_ext.rb:103:in `gem_load_and_activate'
   D:/a/rubygems/rubygems/bundler/spec/support/rubygems_ext.rb:18:in `gem_load'
   bin/parallel_rspec:17:in `<main>'
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:217:in `block in handle_messages'
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:216:in `loop'
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:216:in `handle_messages'
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:93:in `run'
2 processes for 170 specs, ~ 85 specs per process
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/runner.rb:38:in `run'
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/lib/turbo_tests/cli.rb:96:in `run'
	from C:/hostedtoolcache/windows/Ruby/2.7.7/x64/lib/ruby/gems/2.7.0/gems/turbo_tests-2.1.0/bin/turbo_tests:11:in `<top (required)>'
	from D:/a/rubygems/rubygems/bundler/spec/support/rubygems_ext.rb:103:in `load'
	from D:/a/rubygems/rubygems/bundler/spec/support/rubygems_ext.rb:103:in `gem_load_and_activate'
	from D:/a/rubygems/rubygems/bundler/spec/support/rubygems_ext.rb:18:in `gem_load'
	from bin/parallel_rspec:17:in `<main>'

I applied another workaround of rubygems/rubygems#6739 and run RubyGems/Bundler with Windows platform. turbo_tests is still not working.

Rails.application is nil in spec_helper.rb

Hi! This is probably some quirk of our Rails setup, but for some reason when using turbo_tests Rails.application is nil inside our spec_helper.rb file. We normally do a Rails.application.eager_load! to help with simplecov code coverage stats, but this throws a nil error while using turbo_tests.

NoMethodError:
  undefined method `eager_load!' for nil:NilClass

I need to do more investigation of our init process to try and figure out why this is happening, but I wanted to ask here to see if anyone else has run into this or might have an idea why this would be different with turbo_tests vs a normal rspec run. Thank you!

I did see grosser/parallel_tests#360 and so I tried running with DISABLE_SPRING=true but that doesn't seem to have helped.

Oh whoops! I realized now that the Rails application is not loaded in spec_helper but rails_helper in our application. Moving the eager load there fixed the problem! But I don't understand why it works when not using turbo_tests (or parallel_test also).

Add Changelog

Please add a changelog so we can see whats changed between versions.

Seed option is not passed to underlying rspec runners

turbo_tests -n 8 --seed 51251

Documentation formatter is missing nesting/context lines

Interestingly, I had to run the entire spec/models folder because running a single file didn't work, unlike using parallel_tests directly.

$ rake parallel:spec['spec/models/user_spec']
Using recorded test runtime
1 processes for 1 specs, ~ 1 specs per process

Randomized with seed 3971

User
  #max_pending_project_status
    is nil with no projects
    when a freelancer with projects
      is farthest pending status
      is pending even with projects in a non-pending status
  #local_time
    handles a user without a time_zone
    handles a user with a time_zone
  defaults
    sets a random password when provided value is nil
    does not override a provided password
    sets a random password when provided value is blank
  .invite
    does not send invitation email

Finished in 4.9 seconds (files took 11.38 seconds to load)
9 examples, 0 failures

$ be turbo_tests spec/models/user_spec.rb -f d
Using recorded test runtime
Using recorded test runtime
1 processes for 1 specs, ~ 1 specs per process

Finished in 12.11 seconds (files took 0 seconds to load)
0 examples, 0 failures

$ be turbo_tests spec/models/ -n 2 -f d
2 processes for 10 specs, ~ 5 specs per process
does not send invitation email
handles a user with a time_zone
handles a user without a time_zone
is nil with no projects
is farthest pending status
is pending even with projects in a non-pending status
sets a random password when provided value is nil
sets a random password when provided value is blank
does not override a provided password
<snipped output from other files>

Finished in 27.3 seconds (files took 0 seconds to load)
28 examples, 0 failures

When a exception is raised out of specs or code, tests are marked as successful

For example you can raise an exception in spec/rails_helper.rb with raise 'foo', eg:

# This file is copied to spec/ when you run 'rails generate rspec:install'
require 'spec_helper'
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../config/environment', __dir__)
# Prevent database truncation if the environment is production
abort("The Rails environment is running in production mode!") if Rails.env.production?
require 'rspec/rails'

# RAISE ERROR
raise "foo"

begin
  ActiveRecord::Migration.maintain_test_schema!
rescue ActiveRecord::PendingMigrationError => e
  puts e.to_s.strip
  exit 1
end
RSpec.configure do |config|

  config.fixture_path = "#{::Rails.root}/spec/fixtures"
  config.use_transactional_fixtures = true

  config.infer_spec_type_from_file_location!

  config.filter_rails_from_backtraceend
end

Then I run bundle exec turbo_tests And the output wil be:

8 processes for 8 specs, ~ 1 specs per process


Finished in 1.47 seconds (files took 0 seconds to load)
0 examples, 0 failures

I expect the tests to fail, just like bundle exec rspec would fail:

An error occurred while loading ./spec/helpers/widgets_helper_spec.rb.
Failure/Error: raise "foo"

RuntimeError:
  foo
# ./spec/rails_helper.rb:25:in `<top (required)>'
# ./spec/helpers/widgets_helper_spec.rb:1:in `require'
# ./spec/helpers/widgets_helper_spec.rb:1:in `<top (required)>'
...

I'm using rails 6.0.3.4, rspec-rails 4.0.2 and turbo_tests 1.2.

TurboTests spawns several sub-processes even no tests match the filters

TurboTests, grosser/parallel_tests, and briandunn/flatware spawn several sub-processes even no tests match filters and outputs Run options: include {:not_exists=>true} multiple times. RSpec filters tests correctly in this case.

It works this way because it's required to load test files with RSpec to filter the tests. I'll try to find a work-around.

TurboTests command:

bundle exec turbo_tests ./spec/ --tag not_exists

TurboTests output:

3 processes for 3 specs, ~ 1 specs per process
Run options: include {:not_exists=>true}

All examples were filtered out
Run options: include {:not_exists=>true}

All examples were filtered out
Run options: include {:not_exists=>true}

All examples were filtered out


Finished in 3.67 seconds (files took 0.00396 seconds to load)

ParallelTests command:

bundle exec parallel_rspec -- --tag not_exists -r ./spec/spec_helper.rb -r ./lib/turbo_tests.rb -- ./spec/

ParallelTests output:

3 processes for 3 specs, ~ 1 spec per process
Run options: include {:"not_exists"=>true}

All examples were filtered out


Finished in 0.00026 seconds (files took 0.20091 seconds to load)
0 examples, 0 failures

Run options: include {:"not_exists"=>true}

All examples were filtered out


Finished in 0.0003 seconds (files took 0.21081 seconds to load)
0 examples, 0 failures

Run options: include {:"not_exists"=>true}

All examples were filtered out


Finished in 0.00066 seconds (files took 3.25 seconds to load)
0 examples, 0 failures


0 examples, 0 failures

Took 3 seconds

Flatware command:

bundle exec flatware rspec --tag non_exists

Flatware output:

Run options: include {:non_exists=>true}

All examples were filtered out
Run options: include {:non_exists=>true}

All examples were filtered out
Run options: include {:non_exists=>true}

All examples were filtered out

Finished in 0.00239 seconds (files took 3.52 seconds to load)

RSpec command:

bundle exec rspec ./spec/ --tag not_exists

RSpec output:

Run options: include {:not_exists=>true}

All examples were filtered out

Finished in 0.00212 seconds (files took 3.21 seconds to load)
0 examples, 0 failures

`turbo_tests` output missed one error compared and failed lines (compared to RSpec output)

turbo_tests output missed one error compared and failed lines (compared to RSpec output). It could be related to the aggregate_failures option.

RSpec output

RSpec listed the failed spec, NoMethodError, and pointed on the file name and line number.

Failures:

  1) Google Organic Results API Organic Results for Coffee First Wikipedia result has inline sitelinks
     Got 1 failure and 1 other error:

     1.1) Failure/Error: @result["sitelinks"].should be_a(Hash)
            expected nil to be a kind of Hash

          # ./spec/search/google/organic_results/organic_results_coffee_spec.rb:328:in `block (4 levels) in <main>'

     1.2) Failure/Error: @result["sitelinks"]["inline"].should be_a(Array)

          NoMethodError:
            undefined method `[]' for nil:NilClass

          # ./spec/search/google/organic_results/organic_results_coffee_spec.rb:329:in `block (4 levels) in <main>'

Finished in 5.49 seconds (files took 4.36 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/search/google/organic_results/organic_results_coffee_spec.rb:327 # Google Organic Results API Organic Results for Coffee First Wikipedia result has inline sitelinks

`turbo_tests` output

turbo_tests output contained only the error messages.

Failures:

  1) Google Organic Results API Organic Results for Coffee First Wikipedia result has inline sitelinks                                                                       
     Failure/Error: Unable to find matching line from backtrace

       expected nil to be a kind of Hash

       undefined method `[]' for nil:NilClass

Finished in 18.97 seconds (files took 3.88 seconds to load)
185 examples, 1 failure

Failed examples:

rspec ./spec/search/google/organic_results/organic_results_coffee_spec.rb:327 # Google Organic Results API Organic Results for Coffee First Wikipedia result has inline sitelinks

Code

it "has inline sitelinks" do
  @result["sitelinks"].should be_a(Hash)
  @result["sitelinks"]["inline"].should be_a(Array)
end

TEST_ENV_NUMBER value incorrect for first process

The TEST_ENV_NUMBER env var is being set to 1 for the first process when parallel_tests expects it to be blank. This is causing issues because parallel_tests doesn't create a database whereTEST_ENV_NUMBER=1 when you run rake parallel:setup.

Here's where TEST_ENV_NUMBER is being set to 1: https://github.com/serpapi/turbo_tests/blob/master/lib/turbo_tests/runner.rb#L112

Here's the docs for parallel_tests: https://github.com/grosser/parallel_tests#add-to-configdatabaseyml

I'd be happy to address this but I'm curious if we think turbo_tests should be changed to set the value to empty string like parallel_tests expects or if we should figure out if we can configure parallel_tests to use the value 1 for the first process and just add some documentation on how to do that?

NoMethodError: undefined method `pending_exception' for TurboTests::FakeExecutionResult

Cool project, thought I'd give it a try. Getting the following error below. I'm not sure which spec line is producing the error as I haven't spent time digging. If you want me to try something to figure it out let me know.

Extract from Gemfile.lock:

    parallel (1.20.1)
    parallel_tests (3.4.0)
      parallel
...
    rails (6.0.3.4)
...
    rspec (3.10.0)
      rspec-core (~> 3.10.0)
      rspec-expectations (~> 3.10.0)
      rspec-mocks (~> 3.10.0)
    rspec-cells (0.3.5)
      cells (>= 4.0.0, < 6.0.0)
      rspec-rails (< 5.0)
    rspec-core (3.10.0)
      rspec-support (~> 3.10.0)
    rspec-expectations (3.10.0)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (~> 3.10.0)
    rspec-mocks (3.10.0)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (~> 3.10.0)
    rspec-rails (4.0.1)
      actionpack (>= 4.2)
      activesupport (>= 4.2)
      railties (>= 4.2)
      rspec-core (~> 3.9)
      rspec-expectations (~> 3.9)
      rspec-mocks (~> 3.9)
      rspec-support (~> 3.9)
    rspec-support (3.10.0)
...
    turbo_tests (1.2.0)
      bundler
      parallel_tests (~> 3.3)
      rspec (~> 3.10.0)
...
RUBY VERSION
   ruby 2.7.2p137

BUNDLED WITH
   2.1.4

Exception:

root@864e496ab0b5:/usr/local/app# bundle exec turbo_tests

[TRUNCATED]

FFF.F.F.F.F.F...bundler: failed to load command: turbo_tests (/usr/local/gemcache/ruby/2.7.0/bin/turbo_tests)
NoMethodError: undefined method `pending_exception' for #<TurboTests::FakeExecutionResult:0x000055f5244afed8>
Did you mean?  pending_fixed?
  /usr/local/gemcache/ruby/2.7.0/gems/rspec-core-3.10.0/lib/rspec/core/formatters/exception_presenter.rb:291:in `initialize'
  /usr/local/gemcache/ruby/2.7.0/gems/rspec-core-3.10.0/lib/rspec/core/notifications.rb:211:in `new'
  /usr/local/gemcache/ruby/2.7.0/gems/rspec-core-3.10.0/lib/rspec/core/notifications.rb:211:in `initialize'
  /usr/local/gemcache/ruby/2.7.0/gems/rspec-core-3.10.0/lib/rspec/core/notifications.rb:55:in `new'
  /usr/local/gemcache/ruby/2.7.0/gems/rspec-core-3.10.0/lib/rspec/core/notifications.rb:55:in `for'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/lib/turbo_tests.rb:77:in `notification'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/lib/turbo_tests/reporter.rb:55:in `example_pending'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/lib/turbo_tests/runner.rb:162:in `block in handle_messages'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/lib/turbo_tests/runner.rb:154:in `loop'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/lib/turbo_tests/runner.rb:154:in `handle_messages'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/lib/turbo_tests/runner.rb:63:in `run'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/lib/turbo_tests/runner.rb:26:in `run'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/lib/turbo_tests/cli.rb:89:in `run'
  /usr/local/gemcache/ruby/2.7.0/gems/turbo_tests-1.2.0/bin/turbo_tests:11:in `<top (required)>'
  /usr/local/gemcache/ruby/2.7.0/bin/turbo_tests:23:in `load'
  /usr/local/gemcache/ruby/2.7.0/bin/turbo_tests:23:in `<top (required)>'

In Reporter#add, add the following hack:

       ...
        @formatters << formatter_class.new(output)

        if name == "Fuubar"
          output = `bundle exec rspec --dry-run | grep 'examples, 0 failures'`
          example_count = output.split(" ").first.to_i
          delegate_to_formatters(:start, RSpec::Core::Notifications::StartNotification.new(example_count))
        end

        @formatters
        ...

This will send the required StartNotification to fuubar.

Now we can run the specs with bundle exec turbo_tests -f Fuubar -r 'fuubar' to get nicely formatted output.