Giter VIP home page Giter VIP logo

connorshea / vscode-ruby-test-adapter Goto Github PK

View Code? Open in Web Editor NEW
83.0 5.0 50.0 1.95 MB

A Ruby test adapter extension for the VS Code Test Explorer

Home Page: https://marketplace.visualstudio.com/items?itemName=connorshea.vscode-ruby-test-adapter

License: MIT License

TypeScript 77.14% Ruby 22.63% Shell 0.23%
vscode vscode-extension vscode-test-explorer vscode-test-adapter ruby rspec vscode-ruby minitest automated-testing

vscode-ruby-test-adapter's Introduction

Ruby Test Explorer

Install it from the VS Code Marketplace.

This is a Ruby Test Explorer extension for the VS Code Test Explorer extension.

An example screenshot of the extension in use

The extension supports the RSpec and Minitest test frameworks.

Setup

RSpec

The extension needs Ruby and the rspec-core gem installed (and any other dependencies required by your test suite). It's been tested with Ruby 2.6 and Rspec 3.8, but it should work with most recent versions of Ruby and all versions of Rspec 3.x above 3.6.0 (versions before 3.6.0 do not currently work because they don't expose an id property for tests in the JSON formatter).

By default, you need to have rspec installed via Bundler with a Gemfile and bundle install, otherwise bundle exec rspec won't work. If you want to run your Rspec tests with a command other than bundle exec rspec, you can configure the command with the rubyTestExplorer.rspecCommand setting.

Minitest

The extension needs Ruby and the minitest gem installed (and any other dependencies required by your test suite). It's been tested with Ruby 2.5 and 2.6, and Minitest 5.x. It should work with most recent versions of Ruby and Minitest.

Features

Currently supported:

  • Support for RSpec and Minitest suites.
  • Automatic detection of test framework (based on gems listed by bundle list), as well as manual override if necessary.
  • Running individual tests.
  • Running full test suite.
  • Running tests for a specific file.
  • Viewing test output for failed tests (click the test in the test explorer sidebar to open the Output view).
  • Line decorations in test files when a test fails.
  • Displaying test statuses. Success, failure, and pending (called 'skipped' in the extension).
  • Live test status updates as the test suite runs.
  • File locations for each test.
  • Configurable RSpec command.
  • Configurable RSpec spec/ directory.
  • Configurable Minitest command.
  • Configurable Minitest test/ directory.
  • Test hierarchy information.
  • Automatic reloading of test suite info when a file in the test directory changes.
  • Multi-root workspaces.

Configuration

The following configuration options are available:

Property Description
rubyTestExplorer.logpanel Whether to write diagnotic logs to an output panel.
rubyTestExplorer.logfile Write diagnostic logs to the given file.
rubyTestExplorer.testFramework none, auto, rspec, or minitest. auto by default, which automatically detects the test framework based on the gems listed by Bundler. Can disable the extension functionality with none or set the test framework explicitly, if auto-detect isn't working properly.
rubyTestExplorer.filePattern Define the pattern to match test files by, for example ["*_test.rb", "test_*.rb", "*_spec.rb"].
rubyTestExplorer.debuggerHost Define the host to connect the debugger to, for example 127.0.0.1.
rubyTestExplorer.debuggerPort Define the port to connect the debugger to, for example 1234.
rubyTestExplorer.debugCommand Define how to run rdebug-ide, for example rdebug-ide or bundle exec rdebug-ide.
rubyTestExplorer.rspecCommand Define the command to run RSpec tests with, for example bundle exec rspec, spring rspec, or rspec.
rubyTestExplorer.rspecDirectory Define the relative directory of the specs in a given workspace, for example ./spec/.
rubyTestExplorer.minitestCommand Define how to run Minitest with Rake, for example ./bin/rake, bundle exec rake or rake. Must be a Rake command.
rubyTestExplorer.minitestDirectory Define the relative location of your test directory, for example ./test/.

Troubleshooting

If the extension doesn't work for you, here are a few things you can try:

  • Make sure you've run bundle install and that any gems specified in your Gemfile.lock have been installed (assuming you're using Bundler).
  • Disable parallel tests if you are using minitest. To do this set parallelize(workers: 1) in test_helper.rb, or add PARALLEL_WORKERS=1 to the rubyTestExplorer.minitestCommand.
  • Enable the rubyTestExplorer.logpanel config setting and take a look at the output in Output > Ruby Test Explorer Log. This should show what the extension is doing and provide more context on what's happening behind the scenes. (You can alternatively use rubyTestExplorer.logfile to log to a specific file instead).
  • Check the VS Code Developer Tools (Command Palette > 'Developer: Toggle Developer Tools') for any JSON parsing errors, or anything else that looks like it might come from the extension. That could be a bug in the extension, or a problem with your setup.
  • If you're using RSpec, make sure you're using a recent version of the rspec-core gem. If you're on a version prior to 3.6.0, the extension may not work.
  • If you're using RSpec, make sure that the RSpec command and spec directory are configured correctly. By default, tests are run with bundle exec rspec and the tests are assumed to be in the ./spec/ directory. You can configure these with rubyTestExplorer.rspecCommand and rubyTestExplorer.rspecDirectory respectively.
  • If the test suite info isn't loading, your testFramework config may be set to none or the auto-detect may be failing to determine the test framework. Try setting the testFramework config to rspec or minitest depending on what you want to use.

If all else fails or you suspect something is broken with the extension, please feel free to open an issue! :)

Contributing

You'll need VS Code, Node (any version >= 12 should probably work), and Ruby installed.

  • Clone the repository: git clone https://github.com/connorshea/vscode-ruby-test-adapter
  • Run bin/setup to install dependencies.
  • Open the directory in VS Code.
  • Run npm run watch or start the watch Task in VS Code to get the TypeScript compiler running.
  • Go to the Debug section in the sidebar and run "Ruby adapter". This will start a separate VS Code instance for testing the extension in. It gets updated code whenever "Reload Window" is run in the Command Palette.
    • You'll need a Ruby project if you want to actually use the extension to run tests, I generally use my project vglist for testing, but any Ruby project with RSpec or Minitest tests will work.

This extension is based on the example test adapter, it may be useful to check that repository for more information. Test adapters for other languages may also be useful references.

Running tests

There are two groups of tests included in the repository.

  • Tests for Ruby scripts to collect test information and run tests. Run with bundle exec rake in ruby directory.
  • Tests for VS Code extension which invokes the Ruby scripts. Run from VS Code's debug panel with the "Run tests for" configurations.
    • There are separate debug configurations for each supported test framework.
    • Note that you'll need to run npm run build && npm run package before you'll be able to successfully run the extension tests. You'll also need to re-run these every time you make changes to the extension code or your tests.

You can see .github/workflows/test.yml for CI configurations.

Publishing a new version

See the VS Code extension docs for more info.

Before publishing, make sure to update the CHANGELOG.md file. You also need to be logged in to vsce. When creating a Personal Access Token to log in, make sure to give it access to all organizations in your Azure DevOps account. Otherwise, it won't work correctly.

vsce publish VERSION, e.g. vsce publish 1.0.0 will automatically handle creating the git commit and git tag, updating the package.json, and publishing the new version to the Visual Studio Marketplace. You'll need to manually run git push and git push --tags after publishing.

Alternatively, you can bump the extension version with vsce publish major, vsce publish minor, or vsce publish patch.

vscode-ruby-test-adapter's People

Contributors

agilbert201 avatar andyw8 avatar apolzon avatar baelter avatar blowmage avatar cezarygapinski avatar connorshea avatar cristianbica avatar cscorley avatar dependabot[bot] avatar dwarburt avatar gring2 avatar hbenl avatar jochenseeber avatar juice10 avatar markatlish avatar maryamkaka avatar naveg avatar noniq avatar panissupraomnia avatar soutaro avatar ttilberg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

vscode-ruby-test-adapter's Issues

Update to new VS Code packages

The vscode package is deprecated in favor of two other separate packages, should upgrade to those.

npm WARN deprecated [email protected]: This package is deprecated in favor of @types/vscode and vscode-test. For more information please read: https://code.visualstudio.com/updates/v1_36#_splitting-vscode-package-into-typesvscode-and-vscodetest

Broken behavior when opening from UI vs opening from terminal

I am using rvm for different ruby versions with different projects, and I'm using a mac.

When I open up vs code from the terminal using code ., everything works perfectly.

However, when I open up vs code from the spotlight/dock, the test explorer doesn't work. I've checked and in vs code integrated terminal all the environment variables and ruby version are setup correctly. Somehow it seems though that the extension is looking in the wrong place for things. This is the error I get:

[2019-10-04 17:15:18.812] [ERROR] Error while finding RSpec test suite: Command failed: bundle exec rspec --require /Users/<user>/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/spec_set.rb:87:in `block in materialize': Could not find nokogiri-1.10.3 in any of the sources (Bundler::GemNotFound)
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/spec_set.rb:81:in `map!'
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/spec_set.rb:81:in `materialize'
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/definition.rb:170:in `specs'
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/definition.rb:237:in `specs_for'
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/definition.rb:226:in `requested_specs'
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/runtime.rb:108:in `block in definition_method'
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/runtime.rb:20:in `setup'
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler.rb:107:in `setup'
	from /Users/<user>/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler/setup.rb:20:in `<top (required)>'
	from /Users/<user>/.rvm/rubies/ruby-2.6.3/lib/ruby/site_ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /Users/<user>/.rvm/rubies/ruby-2.6.3/lib/ruby/site_ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'

(It should be noted that the nokogiri gem is one that my project depends on, but the extension is looking in the wrong place for it.)

Here's the output of my gem env:

RubyGems Environment:
  - RUBYGEMS VERSION: 3.0.6
  - RUBY VERSION: 2.6.2 (2019-03-13 patchlevel 47) [x86_64-darwin18]
  - INSTALLATION DIRECTORY: /Users/<user>/.rvm/gems/ruby-2.6.2@mayhem
  - USER INSTALLATION DIRECTORY: /Users/<user>/.gem/ruby/2.6.0
  - RUBY EXECUTABLE: /Users/<user>/.rvm/rubies/ruby-2.6.2/bin/ruby
  - GIT EXECUTABLE: /usr/local/bin/git
  - EXECUTABLE DIRECTORY: /Users/<user>/.rvm/gems/ruby-2.6.2@mayhem/bin
  - SPEC CACHE DIRECTORY: /Users/<user>/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /Users/<user>/.rvm/rubies/ruby-2.6.2/etc
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86_64-darwin-18
  - GEM PATHS:
     - /Users/<user>/.rvm/gems/ruby-2.6.2@mayhem
     - /Users/<user>/.rvm/rubies/ruby-2.6.2/lib/ruby/gems/2.6.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /Users/<user>/.rvm/gems/ruby-2.6.2@mayhem/bin
     - /Users/<user>/.rvm/gems/ruby-2.6.2@global/bin
     - /Users/<user>/.rvm/rubies/ruby-2.6.2/bin
     - /Users/<user>/.rvm/bin
     - /usr/local/bin
     - /usr/bin
     - /bin
     - /usr/sbin
     - /sbin
     - /usr/local/opt/protobuf/bin
     - /usr/local/share/android-sdk/platform-tools
     - /usr/local/share/android-sdk/build-tools/29.0.2
     - /usr/local/share/android-sdk/tools

So for some reason, the extension is looking in 2.6.3, even though in the terminal it confirms that it should be using 2.6.2 for this project.

I even tried setting my default rvm to use 2.6.2, but I still get the same error when launching from the UI/spotlight/dock (this time it is looking in the 2.6.2 section, but it still doesn't find what it needs).

I'm super new to Ruby and rvm, but it seems like the extension isn't getting the right info from the rvm magic before it starts loading.

Container Support

Your environment

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: 2.5.7
  • VS Code version: 1.39.2
  • Operating System: Linux
  • RSpec version: 3.9.0

Expected behavior

My development environment uses containers so I set rubyTestExplorer.rspecCommand to something like docker-compose run --rm frontend bundle exec rspec and the vscode-ruby-test-adapter works

Actual behavior

There are a couple of problems with this:

  1. Somehow the CustomFormatter needs to get into my container. This can be work arounded by making getCustomFormatterLocation configurable and putting the file into the workspace somehow (like copy it to .vscode/custom_formatter.rb)
  2. Inside the container the files will not be in the same absolute path than the workspace is. So rubyTestExplorer.*Directory somehow needs to be not relative to the workspace.

invalid option: -R

Your environment

  • vscode-ruby-test-adapter version: 0.8.0
  • Ruby version: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
  • VS Code version: Version: 1.52.0 (user setup)
  • Operating System: OS: Windows_NT x64 10.0.19041 (Win 10), but running in Docker WSL2 inside the container. uname -a command: Linux 9fe4659d85f3 4.19.104-microsoft-standard #1 SMP Wed Feb 19 06:37:35 UTC 2020 x86_64 GNU/Linux
  • RSpec version: (just using Minitest, no RSpec)

Expected behavior

The list of tests should be populated

Actual behavior

With the default configuration, the output is:

[2020-12-17 02:17:10.861] [INFO] Loading Ruby tests...
[2020-12-17 02:17:10.862] [INFO] Loading Minitest tests...
[2020-12-17 02:17:10.862] [INFO] Getting a list of Minitest tests in suite with the following command: bundle exec rake -R $EXT_DIR vscode:minitest:list
[2020-12-17 02:17:12.123] [ERROR] Error while finding Minitest test suite: Command failed: bundle exec rake -R $EXT_DIR vscode:minitest:list

[2020-12-17 02:17:12.123] [ERROR] Output: START_OF_TEST_JSON{"version":"5.14.1","examples":[....  alot of data..  redacted ....]"]}END_OF_TEST_JSON

invalid option: -R  

minitest options:
    -h, --help                       Display this help.
        --no-plugins                 Bypass minitest plugin auto-loading (or set $MT_NO_PLUGINS).
    -s, --seed SEED                  Sets random seed. Also via env. Eg: SEED=n rake
    -v, --verbose                    Verbose. Show progress processing files.
    -n, --name PATTERN               Filter run on /regexp/ or string.
        --exclude PATTERN            Exclude /regexp/ or string from run.

Known extensions: guard_minitest, rails, pride
        --guard                      Run with guard plugin enabled
    -w, --warnings                   Run with Ruby warnings enabled
    -e, --environment ENV            Run tests in the ENV environment
    -b, --backtrace                  Show the complete backtrace
    -d, --defer-output               Output test failures and errors after the test run
    -f, --fail-fast                  Abort test run on first failure or error
    -c, --[no-]color                 Enable color in the output
    -p, --pride                      Pride. Show your testing pride!

Plugin is not working

Having an issue with the plugin finding the tests. It's complaining about the rakefile missing from the $EXT_DIR which is set to /root/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.7.1/ruby/.

VSCode Environment

Plugin is installed remotely into dev container running on Windows with Docker.

  • VSCode 1.49.2
  • Ruby Test Explorer 0.7.1
  • Test Explorer UI 2.19.1
  • Ruby 0.27.0

Ruby Test Explorer Log

[2020-09-26 22:21:45.171] [INFO] Loading Ruby tests...
[2020-09-26 22:21:45.171] [INFO] Loading Minitest tests...
[2020-09-26 22:21:45.171] [INFO] Getting a list of Minitest tests in suite with the following command: bundle exec rake --trace -R $EXT_DIR vscode:minitest:list
[2020-09-26 22:21:45.485] [ERROR] Error while finding Minitest test suite: Command failed: bundle exec rake --trace -R $EXT_DIR vscode:minitest:list
rake aborted!
No Rakefile found (looking for: rakefile, Rakefile, rakefile.rb, Rakefile.rb)
/usr/local/lib/ruby/gems/2.7.0/gems/rake-13.0.1/lib/rake/application.rb:698:in `raw_load_rakefile'
/usr/local/lib/ruby/gems/2.7.0/gems/rake-13.0.1/lib/rake/application.rb:104:in `block in load_rakefile'
/usr/local/lib/ruby/gems/2.7.0/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
/usr/local/lib/ruby/gems/2.7.0/gems/rake-13.0.1/lib/rake/application.rb:103:in `load_rakefile'
/usr/local/lib/ruby/gems/2.7.0/gems/rake-13.0.1/lib/rake/application.rb:82:in `block in run'
/usr/local/lib/ruby/gems/2.7.0/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
/usr/local/lib/ruby/gems/2.7.0/gems/rake-13.0.1/lib/rake/application.rb:80:in `run'
/usr/local/lib/ruby/gems/2.7.0/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
/usr/local/bundle/bin/rake:23:in `load'
/usr/local/bundle/bin/rake:23:in `<top (required)>'
/usr/local/lib/ruby/2.7.0/bundler/cli/exec.rb:63:in `load'
/usr/local/lib/ruby/2.7.0/bundler/cli/exec.rb:63:in `kernel_load'
/usr/local/lib/ruby/2.7.0/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/2.7.0/bundler/cli.rb:476:in `exec'
/usr/local/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/usr/local/lib/ruby/2.7.0/bundler/cli.rb:30:in `dispatch'
/usr/local/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/usr/local/lib/ruby/2.7.0/bundler/cli.rb:24:in `start'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/libexec/bundle:46:in `block in <top (required)>'
/usr/local/lib/ruby/2.7.0/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/libexec/bundle:34:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'

Ruby version and gems installed

$ ruby --version
ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]

$ bundle list | grep -e rake -e test-unit -e minitest -e rspec
  * minitest (5.14.2)
  * rake (13.0.1)
  * rspec-core (3.8.2)
  * rspec-support (3.8.3)
  * test-unit (3.3.6)

Listing from the plugin's ruby directory

$ ls /root/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.7.1/ruby/
Gemfile  Rakefile  specs  vscode  vscode.rake

Output pane does not show the complete error log from bundle exec rspec

Your environment

  • vscode-ruby-test-adapter version: 0.9.0
  • Ruby version: 2.5.1p57
  • VS Code version: 1.56.2 (connected to WSL: Ubuntu)
  • Operating System: Ubuntu 20.04.2 LTS on WSL2 (Windows 10)
  • RSpec or Minitest version: RSpec 3.10

Steps to Reproduce

  1. Install the extension in VS Code.
  2. Load a Ruby project with RSpec which doesn't have the test database.
  3. Click "View error message" on the following notification message.
    image
    "Ruby Test Explorer failed to find an RSpec test suite. Make sure RSpec is installed and your configured RSpec command is correct."

Expected behavior

The full error log must be shown in the Output pane.

Actual behavior

image

Only the message "No examples found." is shown in the Output pane. When running bundle exec rspec spec in terminal, the following log output is produced:

$ bundle exec rspec spec
An error occurred while loading rails_helper.
Failure/Error: ActiveRecord::Migration.maintain_test_schema!

ActiveRecord::NoDatabaseError:
  FATAL:  database "my_test_db" does not exist
# ./spec/rails_helper.rb:29:in `<top (required)>'
# ------------------
# --- Caused by: ---
# PG::ConnectionBad:
#   FATAL:  database "my_test_db" does not exist
#   ./spec/rails_helper.rb:29:in `<top (required)>'
No examples found.
No examples found.
 0/0 |====================================================== 100 ======================================================>| Time: 00:00:00


Finished in 0.00103 seconds (files took 6.71 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

Finished in 0.00103 seconds (files took 6.71 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

Coverage report generated for RSpec to ./coverage. 736 / 29444 LOC (2.5%) covered.
Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected

Minitest nested test classes not working

Your environment

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: 2.5.1
  • VS Code version: 1.36.0
  • Operating System: Ubuntu 18.04
  • Minitest version: 5.11.3
  • Rails version: 5.2.3

The app and all tests run in a docker container. The vscode-ruby-test-adapter also communicates via docker-compose commands

specific vscode settings

{
  "rubyTestExplorer.testFramework": "minitest",
  "rubyTestExplorer.minitestCommand": "docker-compose run --rm -e RAILS_ENV=test -e VSCODE=true api bundle exec rake",
}

Test Setup

require 'test_helper'

class MyTest < ActionController::TestCase
  setup do
    do_something
  end

  test 'test something' do
    assert something
  end

  class MyNestedTest < ActionController::TestCase
    setup do
      do_something_else
    end

    test 'test something else' do
      assert something_else
    end
  end
end

Expected behavior

  • the tests are listed and marked in the actual test file at the correct lines
  • the tests can be run and report back

Actual behavior

  • the tests will be listed, but have a warning sign associated with them
    image
  • running a test in the parent test class will run the correct test, report back, but not update the UI. Instead it shows a warning sign
    image
    Log Output:
    [2019-07-09 06:23:16.036] [INFO] Running Ruby tests ["./test/controllers/facility/versions_controller_test.rb[1:1:1]"]
    [2019-07-09 06:23:16.038] [INFO] Running single test: /home/[USER]/projects/[PROJECT]/test/controllers/facility/versions_controller_test.rb:14
    [2019-07-09 06:23:16.038] [INFO] Running command: docker-compose run --rm -e RAILS_ENV=test -e VSCODE=true api bundle exec rake -R $EXT_DIR vscode:minitest:run test/controllers/facility/versions_controller_test.rb:14
    [2019-07-09 06:23:20.357] [DEBUG] [CHILD PROCESS OUTPUT] 
    [2019-07-09 06:23:20.357] [DEBUG] [CHILD PROCESS OUTPUT] RUNNING: ./test/controllers/facility/versions_controller_test.rb[1:1:1]
    [2019-07-09 06:23:20.966] [DEBUG] [CHILD PROCESS OUTPUT] 
    [2019-07-09 06:23:20.966] [DEBUG] [CHILD PROCESS OUTPUT] PASSED: ./test/controllers/facility/versions_controller_test.rb[1:1:1]
    [2019-07-09 06:23:20.967] [DEBUG] [CHILD PROCESS OUTPUT] START_OF_TEST_JSON{"version":"5.11.3","summary":{"duration":2.12,"example_count":3,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 2.12, Runs: 1, Assertions: 3, Failures: 0, Errors: 0, Skips: 0","examples":[{"description":"should download file","full_description":"should download file","file_path":"./test/controllers/facility/versions_controller_test.rb","full_path":"/app/test/controllers/facility/versions_controller_test.rb","line_number":14,"klass":"Facility::VersionsControllerTest","method":"test_should_download_file","id":"./test/controllers/facility/versions_controller_test.rb[1:1:1]","status":"passed"}]}END_OF_TEST_JSON
    [2019-07-09 06:23:20.967] [DEBUG] Parsing the below JSON:
    [2019-07-09 06:23:20.967] [DEBUG] {"version":"5.11.3","summary":{"duration":2.12,"example_count":3,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 2.12, Runs: 1, Assertions: 3, Failures: 0, Errors: 0, Skips: 0","examples":[{"description":"should download file","full_description":"should download file","file_path":"./test/controllers/facility/versions_controller_test.rb","full_path":"/app/test/controllers/facility/versions_controller_test.rb","line_number":14,"klass":"Facility::VersionsControllerTest","method":"test_should_download_file","id":"./test/controllers/facility/versions_controller_test.rb[1:1:1]","status":"passed"}]}
    [2019-07-09 06:23:20.967] [DEBUG] Handling status of test: {"description":"should download file","full_description":"should download file","file_path":"./test/controllers/facility/versions_controller_test.rb","full_path":"/app/test/controllers/facility/versions_controller_test.rb","line_number":14,"klass":"Facility::VersionsControllerTest","method":"test_should_download_file","id":"./test/controllers/facility/versions_controller_test.rb[1:1:1]","status":"passed"}
    [2019-07-09 06:23:21.426] [INFO] Child process has exited. Sending test run finish event.
  • running a test in the nested class will run the equivalent nth test from the parent class (the first nested test will run the first test from the parent class)
    image
    Log Output:
    [2019-07-09 06:30:03.703] [INFO] Running Ruby tests ["./test/controllers/facility/versions_controller_test.rb[1:1:1]"]
    [2019-07-09 06:30:03.703] [INFO] Running single test: /home/[USER]/projects/[PROJECT]/test/controllers/facility/versions_controller_test.rb:14
    [2019-07-09 06:30:03.704] [INFO] Running command: docker-compose run --rm -e RAILS_ENV=test -e VSCODE=true api bundle exec rake -R $EXT_DIR vscode:minitest:run test/controllers/facility/versions_controller_test.rb:14
    [2019-07-09 06:30:08.220] [DEBUG] [CHILD PROCESS OUTPUT] 
    [2019-07-09 06:30:08.220] [DEBUG] [CHILD PROCESS OUTPUT] RUNNING: ./test/controllers/facility/versions_controller_test.rb[1:1:1]
    [2019-07-09 06:30:08.902] [DEBUG] [CHILD PROCESS OUTPUT] 
    [2019-07-09 06:30:08.902] [DEBUG] [CHILD PROCESS OUTPUT] PASSED: ./test/controllers/facility/versions_controller_test.rb[1:1:1]
    [2019-07-09 06:30:08.903] [DEBUG] [CHILD PROCESS OUTPUT] START_OF_TEST_JSON{"version":"5.11.3","summary":{"duration":2.21,"example_count":3,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 2.21, Runs: 1, Assertions: 3, Failures: 0, Errors: 0, Skips: 0","examples":[{"description":"should download file","full_description":"should download file","file_path":"./test/controllers/facility/versions_controller_test.rb","full_path":"/app/test/controllers/facility/versions_controller_test.rb","line_number":14,"klass":"Facility::VersionsControllerTest","method":"test_should_download_file","id":"./test/controllers/facility/versions_controller_test.rb[1:1:1]","status":"passed"}]}END_OF_TEST_JSON
    [2019-07-09 06:30:08.903] [DEBUG] Parsing the below JSON:
    [2019-07-09 06:30:08.903] [DEBUG] {"version":"5.11.3","summary":{"duration":2.21,"example_count":3,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 2.21, Runs: 1, Assertions: 3, Failures: 0, Errors: 0, Skips: 0","examples":[{"description":"should download file","full_description":"should download file","file_path":"./test/controllers/facility/versions_controller_test.rb","full_path":"/app/test/controllers/facility/versions_controller_test.rb","line_number":14,"klass":"Facility::VersionsControllerTest","method":"test_should_download_file","id":"./test/controllers/facility/versions_controller_test.rb[1:1:1]","status":"passed"}]}
    [2019-07-09 06:30:08.904] [DEBUG] Handling status of test: {"description":"should download file","full_description":"should download file","file_path":"./test/controllers/facility/versions_controller_test.rb","full_path":"/app/test/controllers/facility/versions_controller_test.rb","line_number":14,"klass":"Facility::VersionsControllerTest","method":"test_should_download_file","id":"./test/controllers/facility/versions_controller_test.rb[1:1:1]","status":"passed"}
    [2019-07-09 06:30:09.340] [INFO] Child process has exited. Sending test run finish event.
  • reporting on nested tests will again only show a warning sign
  • a nested test that has no nth equivalent in the parent test class will run and report as usual (the parent test class has 3 tests; the 4th nested test will run normal and report correctly)

Error: Don't know how to build task 'vscode:minitest:list'

Your environment

  • vscode-ruby-test-adapter version: 0.7.1
  • Ruby version: ruby 2.5.7p206 (2019-10-01 revision 67816) [x64-mingw32]
  • VS Code version: 1.42.1
  • Operating System: Windows 10 (1909)
  • Minitest version: 5.14.0

Expected behavior

Expected the extension to detect and display the Minitest tests of the project.

Actual behavior

Getting error message notification:

Command failed: bundle exec rake -R $EXT_DIR vscode:minitest:list
rake aborted!
Don't know how to build task 'vscode:minitest:list' (See the list of available tasks with `rake --tasks`)

(See full trace by running task with --trace)

Am I reading the message right in that the VSCode extension injecting a rake task to collect info? And it's somehow failing to load/register this Rake task?

Any suggestions on what I can do to further track this down?

Unable to get it to work

Your environment

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: 2.4.7
  • VS Code version: 1.40.0
  • Operating System: Mac OS Mojave
  • RSpec version: 3.7

Expected behavior

Actual behavior

When I click on this tab on the side panel in VS Code it loads for some time and fails with an error saying.

Command failed: bundle exec rspec --require /Users/username/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run
warning: constant ::Fixnum is deprecated
warning: constant ::Fixnum is deprecated
:name is duplicated and overwritten on line 147

I feel it's taking the warning rspecs throws and treats them as an error.

I also get a warning saying

Ruby Test Explorer failed to find an RSpec test suite. Make sure RSpec is installed and your configured RSpec command is correct.

I'm not sure if this will work in ruby-2.4.7. If there is something that can be done to make it work that would be great.

Palette command to run spec file from app source file

Hey,

First off, great extension!

I think the usability of the extension would be helped by adding a command to the command palette:

Open rspec/test file from app source file

For example, when viewing app/controllers/users_controller.rb, open command palette and select "Open associated test file in new tab" and it should open spec/controllers/users_controller_spec.rb in a new tab and auto-run the tests in that file.

The inverse (open source file from test file) would also be helpful.

Cheers,
Georges

NameError when attempting to run tests (passes if outside VS Code)

Your environment

  • vscode-ruby-test-adapter version: 0.8.0
  • Ruby version: 2.5.1
  • VS Code version: 1.52.1
  • Operating System: Mac
  • RSpec version: not RSpec - minitest (5.14.2)

Expected behavior

Expecting a pass when running the tests for https://github.com/Shopify/shopify-app-cli (public repo).

Actual behavior

It fails with:

[2020-12-31 03:00:46.335] [ERROR] Error while finding Minitest test suite: Command failed: bundle exec rake -R $EXT_DIR vscode:minitest:list
rake aborted!
NameError: uninitialized constant Extension::Models::Type
Did you mean?  Extension::Models::Types
/Users/andyw8/src/github.com/Shopify/shopify-app-cli/test/project_types/extension/extension_test_helpers/test_extension.rb:5:in `<module:ExtensionTestHelpers>'
/Users/andyw8/src/github.com/Shopify/shopify-app-cli/test/project_types/extension/extension_test_helpers/test_extension.rb:4:in `<module:Extension>'
/Users/andyw8/src/github.com/Shopify/shopify-app-cli/test/project_types/extension/extension_test_helpers/test_extension.rb:3:in `<top (required)>'
/Users/andyw8/src/github.com/andyw8/dotfiles/vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/ruby/vscode/minitest/tests.rb:27:in `require'
/Users/andyw8/src/github.com/andyw8/dotfiles/vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/ruby/vscode/minitest/tests.rb:27:in `block in load_files'
/Users/andyw8/src/github.com/andyw8/dotfiles/vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/ruby/vscode/minitest/tests.rb:27:in `load_files'
/Users/andyw8/src/github.com/andyw8/dotfiles/vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/ruby/vscode/minitest/tests.rb:8:in `all'
/Users/andyw8/src/github.com/andyw8/dotfiles/vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/ruby/vscode/minitest.rb:25:in `list'
/Users/andyw8/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/ruby/vscode.rake:8:in `block (3 levels) in <top (required)>'
/Users/andyw8/.gem/ruby/2.5.1/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
/Users/andyw8/.gem/ruby/2.5.1/bin/bundle:23:in `load'
/Users/andyw8/.gem/ruby/2.5.1/bin/bundle:23:in `<main>'
Tasks: TOP => vscode:minitest:list
(See full trace by running task with --trace)

The tests pass when running bundle exec rake outside of VS Code.

Can't get it to work :(

Hi @connorshea

Firstly - thank you so much for building a test explorer for Ruby. I had given up hope of finding one for VSCode, and then got wind of yours. 1000.times { puts "THANK YOU!!!" }

Only problem is, I can't seem to get it to work...

When I first installed it, I got errors about something being out of bounds. Can't 100% remember, as they don't display anymore after restarting vscode.

What happens now, on any rspec tested project is that the test explorer refresh spinner just spins infinitely, and nothing loads.

My versions are:

  • VSCode: 1.34.0
  • Test Explorer UI: 2.10.0
  • Ruby Test Explorer: 0.4.2

I'm using Fish as my shell, and asdf as my ruby version manager.

Please let me know if I can give you any further debug data... I would love to start using this.

Support test-unit

Hi! I want to support test-unit tests. I have added some tests to understand how it is working.

So, this is the outline of my plan:

  1. Add vscode:test-unit:list and vscode:test-unit:run rake tasks for test-unit tests, which works exactly like minitest version.
  2. Add debug_test-unit.rb file.
  3. Rename MinitestTests class to something like RakeBasedTests class, and add options to specify test framework name.

Any feedback is welcome.

Doesn't work with VSCodium / Theia

Your environment

  • vscode-ruby-test-adapter version: 0.8.0
  • Ruby version: 3.0.0
  • VS Code version: Version: 1.53.2 Commit: 622cb03f7e070a9670c94bae1a45d78d7181fbd4 (VS Codium)
  • Operating System: Mac OS Big Sur
  • RSpec or Minitest version: RSpec 3.10.0

Expected behavior

Right clicking on a spec file and clicking to "run all tests in this file" should try to run them and display the results on the test panel interface

Actual behavior

Nothing happens. Nothing is printed to the console, no error appears anywhere, it is as if the menu item was not triggering anything.

This may be due to some incompatibility with VSCodium, or due to the Test Explorer api relying into some code that is shipped by vscode, but not being shipped elsewhere due to not being opensource.

I would appreciate any suggestion on how to debug this. :)

A similar behavior can be seen with Theia editor in gitpod. You can try with this repository : https://gitlab.com/gitlab-org/gitlab_kramdown (if you have gitpod integration enabled in your profile, all you have to do is click the GitPod button)

Workspace shows tests twice

Your environment

  • vscode-ruby-test-adapter version: 0.5.5
  • Ruby version: 2.6.3
  • VS Code version: 1.35.1
  • Operating System: macOS 10.14.4
  • RSpec version: 3.8 (see below)
RSpec 3.8
  - rspec-core 3.8.0
  - rspec-expectations 3.8.3
  - rspec-mocks 3.8.0
  - rspec-rails 4.0.0.beta2
  - rspec-support 3.8.0

Expected behavior

Ruby Test Explorer shows tests once when in a workspace.

Actual behavior

Ruby Test Explorer shows tests twice when in a workspace [1]. The workspace contains two projects: A rails 6 API only application & a VueJS app. When I open the API folder directly without the workspace, the test suite is shown correctly [2].

Both tests suites shown in [1] work correctly and execute the same command.
The content of the workspace file:

{
  "folders": [
    {
      "path": "api"
    },
    {
      "path": "web"
    }
  ],
  "settings": {
    "solargraph.useBundler": true,
    "solargraph.diagnostics": true
  }
}

I tried to provide as much information as possible :)

Screenshots

[1] (Workspace)
image

[2](Project opened by folder /api)
image

Update test statuses as they're run

Right now when you run the full test suite, all the tests are set to a Pending state until the suite is finished running.

This will most likely require a custom Rspec formatter. Formatters can 'subscribe' to notifications, I think?

'Open in file' doesn't work properly for file groups

Your environment

  • vscode-ruby-test-adapter version: 0.5.3
  • Ruby version: 2.6.3
  • VS Code version: 1.34
  • Operating System: macOS
  • RSpec version: 3.8.0

Expected behavior

When clicking the "Open in file" icon for a file group, it should open the file represented by the group.

Actual behavior

When clicking the "Open in file" icon for a file group, it doesn't do anything.

Screen Shot 2019-06-04 at 10 06 40 PM

Failed to run specs with rspec 3.5.4 and ruby 2.3.8p459

Your environment

  • vscode-ruby-test-adapter version:
  • Ruby version: 2.3.8p459
  • VS Code version: 0.7.1
  • Operating System: OS X Catalina (v10.15.1)
  • RSpec version: 3.5.4

Expected behavior

Extension should load and run the specs.

Actual behavior

It fails to load with the following output in the Ruby Test Explorer Log panel.

[2020-02-19 02:27:36.552] [ERROR] Error while finding RSpec test suite: Command failed: bundle exec rspec --require /Users/chanmyaesanhlaing/.vscode-insiders/extensions/connorshea.vscode-ruby-test-adapter-0.7.1/custom_formatter.rb --format CustomFormatter --order defined --dry-run

NoMethodError: undefined method `errors_outside_of_examples_count' for #<RSpec::Core::Notifications::SummaryNotification:0x00007f9a52c3dae0>
  /Users/chanmyaesanhlaing/.vscode-insiders/extensions/connorshea.vscode-ruby-test-adapter-0.7.1/custom_formatter.rb:37:in `dump_summary'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/reporter.rb:201:in `block in notify'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/reporter.rb:200:in `each'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/reporter.rb:200:in `notify'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/reporter.rb:179:in `block in finish'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/reporter.rb:187:in `close_after'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/reporter.rb:168:in `finish'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/reporter.rb:79:in `report'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:111:in `run_specs'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:87:in `run'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:71:in `run'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:45:in `invoke'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.4/exe/rspec:4:in `<top (required)>'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/bin/rspec:22:in `load'
  /Users/chanmyaesanhlaing/.rbenv/versions/2.3.8/bin/rspec:22:in `<top (required)>'

Minitest fails to report back

Your environment

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: 2.6.3
  • VS Code version: 1.36.0
  • Operating System: Windows 10 (using WSL)
  • Minitest version: 5.11.3
  • Rails version: 6.0.0.rc1
  • VS Code: Using Remote WSL

Actual behavior

  • Tests successfully populate in sidebar
  • Click on the run button
  • Button turns blue but fails to report back

Ruby Test Explorer Output Log

[2019-07-08 17:43:37.038] [INFO] Running Ruby tests ["./test/controllers/health_checks_controller_test.rb"]
[2019-07-08 17:43:37.038] [INFO] Running test file: /mnt/c/Users/curre/Code/haven/test/controllers/health_checks_controller_test.rb
[2019-07-08 17:43:37.038] [INFO] Running command: bundle exec rake -R $EXT_DIR vscode:minitest:run test/controllers/health_checks_controller_test.rb
[2019-07-08 17:44:00.786] [DEBUG] [CHILD PROCESS OUTPUT] START_OF_TEST_JSON{"version":"5.11.3","summary":{"duration":3.63,"example_count":0,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 3.63, Runs: 0, Assertions: 0, Failures: 0, Errors: 0, Skips: 0","examples":[]}END_OF_TEST_JSON
[2019-07-08 17:44:00.786] [DEBUG] Parsing the below JSON:
[2019-07-08 17:44:00.786] [DEBUG] {"version":"5.11.3","summary":{"duration":3.63,"example_count":0,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 3.63, Runs: 0, Assertions: 0, Failures: 0, Errors: 0, Skips: 0","examples":[]}
[2019-07-08 17:44:00.794] [INFO] Child process has exited. Sending test run finish event.

Developer Tools Console Output

  • When running all tests from ui, no output in console
  • When running single file from ui, no output in console
  • When running single method from ui:
[Extension Host] stack trace: TypeError: Cannot read property 'status' of undefined	at MinitestTests.handleStatus (/home/curreta/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/out/minitestTests.js:110:18)	at MinitestTests.<anonymous> (/home/curreta/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/out/tests.js:282:26)	at Generator.next (<anonymous>)	at fulfilled (/home/curreta/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/node_modules/tslib/tslib.js:104:62)	at process._tickCallback (internal/process/next_tick.js:68:7)

TypeError: Cannot read property 'status' of undefined
	at MinitestTests.handleStatus (/home/curreta/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/out/minitestTests.js:110)
	at MinitestTests.<anonymous> (/home/curreta/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/out/tests.js:282)
	at Generator.next (<anonymous>)
	at fulfilled (/home/curreta/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/node_modules/tslib/tslib.js:104)
	at process._tickCallback (internal/process/next_tick.js:68)

Debug attempted

  • I extracted the ruby rake task into my application to run independently
  • The runner successfully adds the files, and extracts out the proper runnable classes
  • The Reporter#prerecord never gets run (as apparent in logs with no "RUNNING:...")

Feature request: configure debug command

When running debug mode I stumble into differing installed gems (see error below). It would be great if we could configure our debug command. In my case I would love to prepend bundle exec in front of it.

Running command: rdebug-ide --host 127.0.0.1 --port 1234 -- $EXT_DIR/debug_minitest.rb 'test/system/audio_recordings_test.rb:6'
[2021-02-03 21:14:35.696] [DEBUG] [CHILD PROCESS OUTPUT] Fast Debugger (ruby-debug-ide 0.7.2, debase 0.2.4.1, file filtering is supported) listens on 127.0.0.1:1234
[2021-02-03 21:14:37.668] [DEBUG] [CHILD PROCESS OUTPUT] Uncaught exception: You have already activated minitest 5.14.3, but your Gemfile requires minitest 5.13.0. Prepending bundle exec to your command may solve this.

Minitest tests run but show no result – empty JSON

Your environment

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
  • VS Code version: 1.39.2
  • Operating System: Ubuntu 19.04
  • Minitest version: minitest (5.12.2, 5.11.3) according to "gem list minitest"
  • Ruby manager: rvm
  • Context: minimal Padrino web application

Expected behavior

I expected the two tests to show whether they succeeded or failed.

Actual behavior

The tests run but after a second the status icon becomes a gray square again.

Screenshot of the tests view:

grafik

Screenshot of the developer toolbar:

grafik

Running "rake" or "rake test" shows this output:

/home/chaas/.rvm/rubies/ruby-2.6.3/bin/ruby -w -I"lib" -I"/home/chaas/.rvm/gems/ruby-2.6.3/gems/rake-13.0.0/lib" "/home/chaas/.rvm/gems/ruby-2.6.3/gems/rake-13.0.0/lib/rake/rake_test_loader.rb" "test/models/feedconfig_test.rb" 
/home/chaas/.rvm/gems/ruby-2.6.3/gems/sequel-5.25.0/lib/sequel/dataset/query.rb:84: warning: statement not reached
/home/chaas/.rvm/gems/ruby-2.6.3/gems/moneta-1.0.0/lib/moneta/expires.rb:31: warning: assigned but unused variable - expires
/home/chaas/.rvm/gems/ruby-2.6.3/gems/padrino-gen-0.14.4/lib/padrino-gen/command.rb:16: warning: `*' interpreted as argument prefix
Run options: --seed 429

# Running:

.F

Finished in 0.002195s, 910.9571 runs/s, 1366.4357 assertions/s.

  1) Failure:
Feedconfig Model#test_0002_fails [/home/chaas/projekte/retadapi_padrino/test/models/feedconfig_test.rb:11]:
Expected false to be truthy.

2 runs, 3 assertions, 1 failures, 0 errors, 0 skips
rake aborted!
Command failed with status (1): [ruby -w -I"lib" -I"/home/chaas/.rvm/gems/ruby-2.6.3/gems/rake-13.0.0/lib" "/home/chaas/.rvm/gems/ruby-2.6.3/gems/rake-13.0.0/lib/rake/rake_test_loader.rb" "test/models/feedconfig_test.rb" ]
/home/chaas/.rvm/gems/ruby-2.6.3/gems/rake-13.0.0/exe/rake:27:in `<top (required)>'
/home/chaas/.rvm/gems/ruby-2.6.3/bin/ruby_executable_hooks:24:in `eval'
/home/chaas/.rvm/gems/ruby-2.6.3/bin/ruby_executable_hooks:24:in `<main>'
Tasks: TOP => default => test => test:models
(See full trace by running task with --trace)

Display nested describes & contexts as subtrees

Your environment

  • vscode-ruby-test-adapter version: 0.6.1
  • Ruby version: ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-darwin15]
  • VS Code version: 1.41.1
  • Operating System: Mac OS X El Capitan (10.11.5)
  • RSpec version: 3.8.0

Expected behavior

When I use nested describes or contexts, I want them to be displayed in the Test Explorer as subtrees that reflect how I've organized the code.

describe "Foo" do
    it "CORRECT: nested 1 level deep, directly under filename"

    context "Bar" do
        it "REQUEST: should be nested 2 levels deep"
    end
end
> example_spec.rb
  - Foo CORRECT: nested 1 level deep, directly under filename
  > Foo Bar
    - REQUEST: should be nested 2 levels deep

Actual behavior

Screen Shot 2019-12-24 at 12 03 33 PM

Tests generated by fresh Rails 6.1.4 app using scaffolding do not return any results

Your environment

  • vscode-ruby-test-adapter version: v 0.9.0
  • Ruby version: 3.0.2p107
  • VS Code version: Version: 1.58.2 (Universal)
  • Operating System: Mac OS (ARM - m1 chip)
  • Minitest version: 5.14.4 (As included with Rails 6.1.4 )

Expected behavior

When running tests scaffolded by Rails, the results of the tests show up in green or red, both when running Rails and running a simpler Sinatra app.

Actual behavior

When running the tests for a Rails app, the results never show. Instead the tests remain as they were before running. Everything works as expected in the simpler Sinatra app.

127498818-9f0686a2-9170-470c-bed8-0ae9f1a4cc74

Setting the debug output flag, this is the output I get:

[2021-07-29 13:16:06.226] [INFO] Running Ruby tests ["root"]
[2021-07-29 13:16:06.228] [INFO] Running full test suite.
[2021-07-29 13:16:06.228] [INFO] Running command: bundle exec rake -R $EXT_DIR vscode:minitest:run
[2021-07-29 13:16:07.192] [DEBUG] [CHILD PROCESS OUTPUT] START_OF_TEST_JSON{"version":"5.14.4","summary":{"duration":0.28,"example_count":0,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 0.28, Runs: 0, Assertions: 0, Failures: 0, Errors: 0, Skips: 0","examples":[]}END_OF_TEST_JSON
[2021-07-29 13:16:07.192] [DEBUG] Parsing the below JSON:
[2021-07-29 13:16:07.192] [DEBUG] {"version":"5.14.4","summary":{"duration":0.28,"example_count":0,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 0.28, Runs: 0, Assertions: 0, Failures: 0, Errors: 0, Skips: 0","examples":[]}
[2021-07-29 13:16:07.202] [INFO] Child process has exited. Sending test run finish event.

When I try running bundle exec rake -R $EXT_DIR vscode:minitest:run in terminal, the tests exit correctly, with the following output:

rails_hello_world_app git:(main) ✗ bundle exec rake -R $EXT_DIR vscode:minitest:run
Run options: --seed 29657

# Running:

.......

Finished in 0.323463s, 21.6408 runs/s, 27.8239 assertions/s.
7 runs, 9 assertions, 0 failures, 0 errors, 0 skips

It seems to me like the JSON results are either parsed incorrectly, or passed incorrectly.

Steps to reproduce

  1. Set up Rails (See Gemfile below) rails new test, bundle install, etc.
  2. Scaffold a primitive model with rails g scaffold Test test:string and migrate the db with rails db:migrate
  3. Run the tests (I had to restart vscode to get the tests recognized)

Gemfile:

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '3.0.2'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails', branch: 'main'
gem 'rails', '~> 6.1.4'
# Use Puma as the app server
gem 'puma', '~> 5.0'
# Use SCSS for stylesheets
gem 'sass-rails', '>= 6'
# Transpile app-like JavaScript. Read more: https://github.com/rails/webpacker
gem 'webpacker', '~> 5.0'
# Turbolinks makes navigating your web application faster. Read more: https://github.com/turbolinks/turbolinks
gem 'turbolinks', '~> 5'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 2.7'
# Use Redis adapter to run Action Cable in production
# gem 'redis', '~> 4.0'
# Use Active Model has_secure_password
# gem 'bcrypt', '~> 3.1.7'


# Needed for Rails with Ruby 3:
gem 'rexml', '~> 3.2', '>= 3.2.4'

# Use Active Storage variant
# gem 'image_processing', '~> 1.2'

# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.4.4', require: false

group :production do
  gem 'pg', '~> 1.2.3'
end

group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
  # Use sqlite3 as the database for Active Record for testing and dev
  gem 'sqlite3', '~> 1.4'
end

group :development do
  # Access an interactive console on exception pages or by calling 'console' anywhere in the code.
  gem 'web-console', '>= 4.1.0'
  # Display performance information such as SQL time and flame graphs for each request in your browser.
  # Can be configured to work on production as well see: https://github.com/MiniProfiler/rack-mini-profiler/blob/master/README.md
  gem 'rack-mini-profiler', '~> 2.0'
  gem 'listen', '~> 3.3'
  # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
  gem 'spring'
end

group :test do
  # Adds support for Capybara system testing and selenium driver
  gem 'capybara', '>= 3.26'
  gem 'selenium-webdriver'
  # Easy installation and use of web drivers to run system tests with browsers
  gem 'webdrivers'
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
# gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

Steps I've tried that didn't work:

  1. Adding minitest to the gem file explicitly. AFAICT it did no difference.

Better organization of tests

Ideally we would follow the same hierarchy as the tests, but I'm not sure how to get an Rspec custom formatter to output the right information to make that possible.

It's not really possible to derive the information from the test names because tests sharing the beginning of their names isn't indicative of whether they belong in the same group. e.g. "ApplicationHelper flash class sets the correct class for a notice" and "ApplicationHelper flash class sets the correct class for a warning" would match up until the last word, even though that's not how the test hierarchy should be represented.

Cannot find module out/main.js

Your environment

  • vscode-ruby-test-adapter version:
  • Ruby version: 2.7.2
  • VS Code version: 1.51.0
  • Operating System: macOS 10.14.6
  • Minitest version: 5.14.2

Expected behavior

The test suite should be detected.

Actual behavior

The developer tools log the following:

abstractExtensionService.ts:717
Activating extension 'connorshea.vscode-ruby-test-adapter' failed: Cannot find module '/Users/steve/.vscode-oss/extensions/connorshea.vscode-ruby-test-adapter-0.7.1/out/main.js'

No output in the logpanel.

Maybe Relevant Details

I'm writing spec-like tests as per MiniTest::Spec. Therefore, the filesystem looks as follows:

spec/
  |
  +-- spec_helper.rb
  |
  +-- lib/

I've tried most of the settings for this extension with zero effect.

Returned JSON does not contain nested tests result.

Your environment

  • vscode-ruby-test-adapter version: 0.6.1
  • Ruby version: 2.6.5
  • VS Code version: 1.41
  • Operating System: OSX
  • Minitest version: 5.11.3

Expected behavior

Expect the returning JSON to contain results from all tests, including nested tests.

describe ClassToTest do
  describe 'method' do
    it 'returns JSON as expected' do
   
    end
    describe 'another context in method' do
      it 'returns JSON as expected' do

      end
    end
  end
end

Actual behavior

describe ClassToTest do
  describe 'method' do
    it 'returns JSON as expected' do
   
    end
    describe 'another context in method' do
      it 'Test runs successfully but does not report the results to JSON' do

      end
    end
  end
end

Plugin bails on first folder without rake gem/Rakefile in multi-folder Workspace

Your environment

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: 2.6.2
  • VS Code version: 1.38.0
  • Operating System: macOS
  • RSpec version: -

Expected behavior

In a workspace with more than one folder, but one without tests (and no rake gem in the Gemfile, and on Rakefile), the plugin should find the tests in the folder with tests (and a rake gem and a Rakefile). It should skip folders with problems (perhaps report the error in the "problems" output though. It should not stop and not resolve other folders' tests.

Actual behavior

There seems to be a problem when a folder has no Rakefile and/or no 'rake' gem in the Gemfile. It will bails out with:

Command failed: bundle exec rake -R $EXT_DIR vscode:minitest:list Gem::Exception: can't find executable rake for gem rake. rake is not currently included in the bundle, perhaps you meant to add it to your Gemfile?

Support multiple spec directories in settings

Your environment

  • vscode-ruby-test-adapter version: 0.7.1
  • Ruby version: 2.6.3
  • VS Code version: 1.45.1
  • Operating System: Mac OS Mojave 10.14.6
  • RSpec version: 3.9.0

Desired Behavior

This a request for enhancement, not a problem, as the extension works beautifully.

Our two primary apps use components and engines, respectively, to house different functionality and consequently also have their own spec directories. Typically models and other top level classes have specs in ./spec/ but for controllers, presenters, helpers, etc. we often house specs inside of the respective engine or component directory, e.g. ./engines/admin/spec

I often work out of the top level directory and would love for ruby test explorer to support this, perhaps allow "rubyTestExplorer.rspecDirectory" to accept an array of directories, or better yet, grep entire work space for directories named spec and pull those into the explorer view. Not sure if this is specific to to this extension or a limitation of the overall vscode test explorer. Will also look at the extension code to see if this is doable but wanted to get this on your radar.

Actual behavior

Only able to run and view specs in test explorer when they live in the directory defined by "rubyTestExplorer.rspecDirectory" in settings.

Organize tests based on their type

Right now the top-level organization of tests is based on the test files. Instead, it should be based on the type of test and then the file.

This would make it much easier to find tests in the sidebar.

Can maybe do this by figuring out the structure of the spec/ directory from the file names?

plugin is not working

Hello everybody!
I installed Ruby Test Explorer but nothing is working. When I start VS Code I get next error:

Command failed: rspec --require c:\Users\shredder.vscode\extensions\connorshea.vscode-ruby-test-adapter-0.6.0\custom_formatter.rb --format CustomFormatter --order defined --dry-run

Source: Ruby Test Explorer

And this warning:
Ruby Test Explorer failed to find an RSpec test suite. Make sure RSpec is installed and your configured RSpec command is correct.

Settings of Ruby Test Explorer

"rubyTestExplorer.rspecCommand": "rspec",
"rubyTestExplorer.rspecDirectory": 
"C:\\Users\\shredder\\VisualStudioCodeProjects\\<project_name\\spec",
"rubyTestExplorer.logpanel": true

And there is nothing in Test menu
image

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: 2.6.3
  • VS Code version: 1.37.0
  • Operating System: Windows 10
  • RSpec version: rspec-core 3.8.2

Can you help me, please?

Not working on remote SSH

  • vscode-ruby-test-adapter version: 0.8.0
  • Ruby version: 2.7.2
  • VS Code version: 1.52.1
  • Operating System: Ubuntu 20
  • RSpec or Minitest version: rspec

If I use the remote SSH https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh I get the error

Error while finding RSpec test suite: 
Command failed: bundle exec rspec --pattern './spec//**/*_test.rb,./spec//**/test_*.rb,./spec//**/*_spec.rb' --require /LOCAL_COMPUTER_PATH/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/custom_formatter.rb --format 

It's trying to use a local path, not on a ssh server

Unable to debug specs

Your environment

  • vscode-ruby-test-adapter version: 0.8.0
  • Ruby version: 2.3.8
  • VS Code version: 1.51.1
  • Operating System: Ubuntu 18.04 via WSL2 (Windows 10)
  • RSpec version: 3.6.0

Expected behavior

Selecting a rspec example group from the test explorer panel and clicking the debug icon should run the example group with debugging enabled.

Actual behavior

I get an popup with the error:

Error while debugging test: SyntaxError: Unexpected end of JSON input
Source: Test Explorer UI (Exension)

In the output panel, if I select Ruby Test Explorer Log I see the following when trying to debug geolocation_utils_spec.rb

[2020-11-23 09:02:23.505] [INFO] Debugging test(s) ["./spec/lib/geolocation_utils_spec.rb"] of /home/cec/core/ruby-projects/agendalugano
[2020-11-23 09:02:23.505] [INFO] Running Ruby tests ["./spec/lib/geolocation_utils_spec.rb"]
[2020-11-23 09:02:23.506] [INFO] Starting the debug session
[2020-11-23 09:02:23.506] [INFO] Running test file: /home/cec/core/ruby-projects/agendalugano/spec/lib/geolocation_utils_spec.rb
[2020-11-23 09:02:23.506] [INFO] Running command: rdebug-ide --host 127.0.0.1 --port 1234 -- $EXT_DIR/debug_rspec.rb --require /home/cec/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/custom_formatter.rb --format CustomFormatter '/home/cec/core/ruby-projects/agendalugano/spec/lib/geolocation_utils_spec.rb'
[2020-11-23 09:02:23.990] [DEBUG] [CHILD PROCESS OUTPUT] Fast Debugger (ruby-debug-ide 0.7.2, debase 0.2.4.1, file filtering is supported) listens on 127.0.0.1:1234
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] Uncaught exception: cannot load such file -- /debug_rspec.rb
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	/home/cec/.rvm/gems/ruby-2.3.8@agenda/bin/rdebug-ide:23:in `load'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	/home/cec/.rvm/gems/ruby-2.3.8@agenda/bin/rdebug-ide:23:in `<main>'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	/home/cec/.rvm/gems/ruby-2.3.8@agenda/bin/ruby_executable_hooks:24:in `eval'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	/home/cec/.rvm/gems/ruby-2.3.8@agenda/bin/ruby_executable_hooks:24:in `<main>'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] /home/cec/.rvm/gems/ruby-2.3.8@agenda/gems/ruby-debug-ide-0.7.2/lib/ruby-debug-ide.rb:106:in `debug_load': cannot load such file -- /debug_rspec.rb (LoadError)
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	from /home/cec/.rvm/gems/ruby-2.3.8@agenda/gems/ruby-debug-ide-0.7.2/lib/ruby-debug-ide.rb:106:in `debug_program'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	from /home/cec/.rvm/gems/ruby-2.3.8@agenda/gems/ruby-debug-ide-0.7.2/bin/rdebug-ide:193:in `<top (required)>'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	from /home/cec/.rvm/gems/ruby-2.3.8@agenda/bin/rdebug-ide:23:in `load'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	from /home/cec/.rvm/gems/ruby-2.3.8@agenda/bin/rdebug-ide:23:in `<main>'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	from /home/cec/.rvm/gems/ruby-2.3.8@agenda/bin/ruby_executable_hooks:24:in `eval'
[2020-11-23 09:02:24.368] [DEBUG] [CHILD PROCESS OUTPUT] 	from /home/cec/.rvm/gems/ruby-2.3.8@agenda/bin/ruby_executable_hooks:24:in `<main>'
[2020-11-23 09:02:24.372] [INFO] Child process has exited. Sending test run finish event.
[2020-11-23 09:02:24.403] [DEBUG] Parsing the below JSON:
[2020-11-23 09:02:24.403] [DEBUG] 
[2020-11-23 09:02:25.047] [INFO] Debug session ended
[2020-11-23 09:02:25.047] [INFO] Killing currently-running tests.

I apologize in advance if this is a stupid setup mistake on my end, but I'm new to VS Code as a IDE, since before I could work on a linux desktop and I used RubyMine.

This extensions is absolutely amazing! I can't thank you enough for it.

Implement a test definitions watcher

Instead of needing to reload the test explorer definitions manually, it'd be nice to be able to reload the tests whenever a file in the spec/ directory is changed.

It may be a good idea to make the watched directory configurable.

When Rspec test loading fails, adequate error information is not surfaced

Your environment

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
  • VS Code version: 1.39.2
  • Operating System: Debian GNU/Linux 10 (buster) (Docker container)
  • RSpec version: 3.9.0

Expected behavior

Errors in encountered by rspec when loading tests (rspec --dry-run) should be surfaced in the IDE.

Actual behavior

Ruby test explorer shows a generic failure message.
For example, when using a variable defined in a let(){} function outside of a describe or context block, I see the following:

A vscode notification, that has some of the output, but not the relevant error message:

image

Copying that message and pasting it elsewhere to get the full message does not show the error either.

The Ruby Test Explorer Log shows the same output as the message:

[2019-11-07 03:43:13.674] [INFO] Running dry-run of RSpec test suite with the following command: bundle exec rspec --require /home/baw_web/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run
[2019-11-07 03:43:13.688] [INFO] Sending autorun event
[2019-11-07 03:44:00.321] [ERROR] Error while finding RSpec test suite: Command failed: bundle exec rspec --require /home/baw_web/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run
/usr/local/bundle/gems/activesupport-4.2.11.1/lib/active_support/core_ext/object/duplicable.rb:111: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
The client method is deprecated as of redis-rb 4.0.0, please use the new _clientmethod instead. Support for the old method will be removed in redis-namespace 2.0.
The PGconn, PGresult, and PGError constants are deprecated, and will be
removed as of version 1.0.

You should use PG::Connection, PG::Result, and PG::Error instead, respectively.

Called from /usr/local/bundle/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'
/usr/local/bundle/gems/shoulda-matchers-2.8.0/lib/shoulda/matchers/active_model/validate_inclusion_of_matcher.rb:251: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.

There's a lot of rubbish in there but none of it is the error.

If I run the command myself in another console:

baw_web@38f11388d3f9:~/baw-server$  bundle exec rspec --require /home/baw_web/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.6.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run

***
Tests must be run in the test envrionment.
The current envrionment `development` has been changed to `test`.
See rails_helper.rb to disable this check
***
/usr/local/bundle/gems/activesupport-4.2.11.1/lib/active_support/core_ext/object/duplicable.rb:111: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
===> Settings: 'test' loaded from /home/baw_web/baw-server/config/settings/default.yml.
===> BawWorkers::Settings: 'test' loaded from /home/baw_web/baw-server/config/settings/default.yml.
===> Settings: 'test' merged from /home/baw_web/baw-server/config/settings/test.yml.
===> BawWorkers::Settings: 'test' merged from /home/baw_web/baw-server/config/settings/test.yml.
===> Configuration passed validation.
The client method is deprecated as of redis-rb 4.0.0, please use the new _clientmethod instead. Support for the old method will be removed in redis-namespace 2.0.
The PGconn, PGresult, and PGError constants are deprecated, and will be
removed as of version 1.0.

You should use PG::Connection, PG::Result, and PG::Error instead, respectively.

Called from /usr/local/bundle/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'
[Zonebie] Setting timezone: ZONEBIE_TZ="Kyiv"
===> Time zone offset is 7200.
/usr/local/bundle/gems/shoulda-matchers-2.8.0/lib/shoulda/matchers/active_model/validate_inclusion_of_matcher.rb:251: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
START_OF_TEST_JSON{"version":"3.9.0","messages":["\nAn error occurred while loading ./spec/acceptance/media/media_access_spec.rb.\n\u001B[31mFailure/Error: \u001B[0mprepare_project_logged_in(site)\u001B[0m\n\u001B[31m  `site` is not available on an example group (e.g. a `describe` or `context` block). It is only available from within individual examples (e.g. `it` blocks) or from constructs that run in the scope of an example (e.g. `before`, `let`, etc).\u001B[0m\n\u001B[36m# ./spec/acceptance/media/media_access_spec.rb:19:in `block in \u003Ctop (required)\u003E'\u001B[0m\n\u001B[36m# /usr/local/bundle/gems/rspec_api_documentation-4.8.0/lib/rspec_api_documentation/dsl.rb:26:in `resource'\u001B[0m\n\u001B[36m# ./spec/acceptance/media/media_access_spec.rb:8:in `\u003Ctop (required)\u003E'\u001B[0m\n\u001B[36m# /usr/local/bundle/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:268:in `load'\u001B[0m\n\u001B[36m# /usr/local/bundle/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:268:in `block in load'\u001B[0m\n\u001B[36m# /usr/local/bundle/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'\u001B[0m\n\u001B[36m# /usr/local/bundle/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:268:in `load'\u001B[0m\n"],"examples":[],"summary":{"duration":0.0001013,"example_count":0,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":1},"summary_line":"0 examples, 0 failures, 1 error occurred outside of examples"}END_OF_TEST_JSON

Bingo! In between the *_OF_TEST_JSON markers, the error from rspec exists.

The error reported from rspec is totally user error. This bug is rather that the user error is squashed and not shown anywhere in vscode (as far as I can tell).

I believe the issue is here:

childProcess.exec(cmd, execArgs, (err, stdout) => {
if (err) {
this.log.error(`Error while finding RSpec test suite: ${err.message}`);
// Show an error message.
vscode.window.showWarningMessage("Ruby Test Explorer failed to find an RSpec test suite. Make sure RSpec is installed and your configured RSpec command is correct.");
vscode.window.showErrorMessage(err.message);
throw err;
}
resolve(stdout);

i think the standard output should probably be parsed before the exception is thrown.

Command failed: bundle exec rails test -R $EXT_DIR vscode:minitest:list

Your environment

  • vscode-ruby-test-adapter version: 0.9.0
  • Ruby version: 2.6.6
  • VS Code version: 1.59.0
  • Operating System: Linux Mint 20.2
  • RSpec or Minitest version: 5.14.4
  • Application: Rails 6.1
  • rbenv 1.1.2

Expected behavior

See the list of tests.

Actual behavior

Error message on the bottom right:

Command failed: bundle exec rails test -R $EXT_DIR vscode:minitest:list /home/chaas/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bootsnap-1.7.7/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:34:in require': cannot load such file -- rspec (LoadError) from /home/chaas/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.9.0/ruby/rspecs/unit_test.rb:1:in

' from /home/chaas/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bootsnap-1.7.7/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in require' from /home/chaas/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bootsnap-1.7.7/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in block in require_with_bootsnap_lfi' from /home/chaas/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bootsnap-1.7.7/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register' from /home/chaas/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bootsnap-1.7.7/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22...

…and…

Ruby Test Explorer failed to find a Minitest test suite. Make sure Minitest is installed and your configured Minitest command is correct.

Trying vscode:minitest:list in the shell

EXT_DIR=~/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.9.0/ruby
bundle exec rake -R $EXT_DIR vscode:minitest:list
rake aborted!
KeyError: key not found: "TESTS_PATTERN"
/home/chaas/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.9.0/ruby/vscode/minitest/tests.rb:25:in fetch' /home/chaas/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.9.0/ruby/vscode/minitest/tests.rb:25:in load_files'
/home/chaas/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.9.0/ruby/vscode/minitest/tests.rb:8:in all' /home/chaas/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.9.0/ruby/vscode/minitest.rb:25:in list'
/home/chaas/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.9.0/ruby/vscode.rake:8:in block (3 levels) in <main>' /home/chaas/.rbenv/versions/2.6.6/bin/bundle:23:in load'
/home/chaas/.rbenv/versions/2.6.6/bin/bundle:23:in `

'
Tasks: TOP => vscode:minitest:list
(See full trace by running task with --trace)

Trying to set TESTS_PATTERN into the environment before starting VScode

export EXT_DIR=~/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.9.0/ruby ; export TESTS_PATTERN='*_test.rb'
rake -R $EXT_DIR vscode:minitest:list

START_OF_TEST_JSON{"version":"5.14.4","examples":[{"description":"sanity","full_description":"sanity","file_path":"./test/helpers/logs_helper_test.rb","full_path":"/home/chaas/projekte/debshots2/test/helpers/logs_helper_test.rb","line_number":4,"klass":"LogsHelperTest","method":"test_sanity","runnable":"LogsHelperTest","id":"./test/helpers/logs_helper_test.rb[4]"},{"description":"should detect the proper order of package versions","full_description":"should detect the proper order of package versions","file_path":"./test/lib/deb_importer_test.rb","full_path":"/home/chaas/projekte/debshots2/test/lib/deb_importer_test.rb","line_number":11,"klass":"PackagesHelperTest","method":"test_should_detect_the_proper_order_of_package_versions","runnable":"PackagesHelperTest","id":"./test/lib/deb_importer_test.rb[11]"},{"description":"should split version files into epoch, upstream and revision","full_description":"should split version files into epoch, upstream and revision","file_path":"./test/lib/deb_importer_test.rb","full_path":"/home/chaas/projekte/debshots2/test/lib/deb_importer_test.rb","line_number":21,"klass":"PackagesHelperTest","method":"test_should_split_version_files_into_epoch,_upstream_and_revision","runnable":"PackagesHelperTest","id":"./test/lib/deb_importer_test.rb[21]"},{"description":"should be able to parse local Debian repository test files","full_description":"should be able to parse local Debian repository test files","file_path":"./test/lib/deb_importer_test.rb","full_path":"/home/chaas/projekte/debshots2/test/lib/deb_importer_test.rb","line_number":48,"klass":"PackagesHelperTest","method":"test_should_be_able_to_parse_local_Debian_repository_test_files","runnable":"PackagesHelperTest","id":"./test/lib/deb_importer_test.rb[48]"},{"description":"visit home page","full_description":"visit home page","file_path":"./test/integration/homepage_test.rb","full_path":"/home/chaas/projekte/debshots2/test/integration/homepage_test.rb","line_number":8,"klass":"AdvancedTest","method":"test_visit_home_page","runnable":"AdvancedTest","id":"./test/integration/homepage_test.rb[8]"},{"description":"valid","full_description":"valid","file_path":"./test/models/user_test.rb","full_path":"/home/chaas/projekte/debshots2/test/models/user_test.rb","line_number":8,"klass":"UserTest","method":"test_valid","runnable":"UserTest","id":"./test/models/user_test.rb[8]"},{"description":"user can see home page","full_description":"user can see home page","file_path":"./test/system/browses_test.rb","full_path":"/home/chaas/projekte/debshots2/test/system/browses_test.rb","line_number":10,"klass":"BrowsesTest","method":"test_user_can_see_home_page","runnable":"BrowsesTest","id":"./test/system/browses_test.rb[10]"},{"description":"user can browse packages","full_description":"user can browse packages","file_path":"./test/system/browses_test.rb","full_path":"/home/chaas/projekte/debshots2/test/system/browses_test.rb","line_number":15,"klass":"BrowsesTest","method":"test_user_can_browse_packages","runnable":"BrowsesTest","id":"./test/system/browses_test.rb[15]"},{"description":"upload broken screenshot anonymously and get error message","full_description":"upload broken screenshot anonymously and get error message","file_path":"./test/system/uploads_test.rb","full_path":"/home/chaas/projekte/debshots2/test/system/uploads_test.rb","line_number":4,"klass":"UploadsTest","method":"test_upload_broken_screenshot_anonymously_and_get_error_message","runnable":"UploadsTest","id":"./test/system/uploads_test.rb[4]"},{"description":"upload png screenshot anonymously and await moderation","full_description":"upload png screenshot anonymously and await moderation","file_path":"./test/system/uploads_test.rb","full_path":"/home/chaas/projekte/debshots2/test/system/uploads_test.rb","line_number":13,"klass":"UploadsTest","method":"test_upload_png_screenshot_anonymously_and_await_moderation","runnable":"UploadsTest","id":"./test/system/uploads_test.rb[13]"},{"description":"upload png screenshot as moderator and expect auto-approval","full_description":"upload png screenshot as moderator and expect auto-approval","file_path":"./test/system/uploads_test.rb","full_path":"/home/chaas/projekte/debshots2/test/system/uploads_test.rb","line_number":59,"klass":"UploadsTest","method":"test_upload_png_screenshot_as_moderator_and_expect_auto-approval","runnable":"UploadsTest","id":"./test/system/uploads_test.rb[59]"},{"description":"go to login page","full_description":"go to login page","file_path":"./test/system/users_test.rb","full_path":"/home/chaas/projekte/debshots2/test/system/users_test.rb","line_number":4,"klass":"UsersTest","method":"test_go_to_login_page","runnable":"UsersTest","id":"./test/system/users_test.rb[4]"}]}END_OF_TEST_JSON

Running rake test works

ake test
Started with run options --seed 21853
[…]
Finished in 0.11691s
6 tests, 30 assertions, 0 failures, 0 errors, 0 skips

SimpleCov rspec output breaks test detection

Your environment

  • vscode-ruby-test-adapter version: 0.7.1
  • Ruby version: 2.5.5
  • VS Code version: 1.46.0-insider
  • Operating System: Windows 10
  • RSpec version: 3.8

Expected behavior

With SimpleCov enabled, the extension should detect the specs

Actual behavior

The spec detection step fails silently. SimpleCov adds a few lines after the dry-run output, which may cause this behavior. Disabling the coverage report fixes it at least

Slow refresh of tests and high CPU usage while editing a test file

Your environment

  • vscode-ruby-test-adapter version: 0.7.1
  • Ruby version: ruby 2.5.7p206 (2019-10-01 revision 67816) [x86_64-darwin18]
  • VS Code version: 1.46.0
  • Operating System: OSX
  • RSpec version: N.A.
  • TestExplorer: Autorun Disabled (via UI menu option 'Disable Autorun', couldn't find a sticky setting for it.)

Expected behavior

The tests should not continuously reload while editing a test file. Or the reload needs to be super fast.

Actual behavior

The list of tests is continuously reloaded while editing a test file (even when Autorun is disabled). The reload is slow and causes high CPU usage.

Every time a *test.rb file is saved the plugin triggers an autorun event (see log below) and lists the tests. Since i have auto save enabled, this triggers a new run every few seconds while editing / fixing tests. These saves end up triggering multiple "Getting a list of Minitest tests in suite.." events. Which get queued up and cause high CPU load for an extended period of time.

Example Ruby Test Explorer log with just a few seconds of file edits triggering multiple runs:

[2020-06-16 22:10:36.450] [INFO] /Users/path/to/some_test.rb was saved - checking if this effects /Users/path/to/rails/project
[2020-06-16 22:10:36.450] [INFO] A test file has been edited, reloading tests.
[2020-06-16 22:10:36.450] [INFO] Loading Ruby tests...
[2020-06-16 22:10:36.450] [INFO] Loading Minitest tests...
[2020-06-16 22:10:36.450] [INFO] Getting a list of Minitest tests in suite with the following command: bundle exec rake -R $EXT_DIR vscode:minitest:list
[2020-06-16 22:10:36.457] [INFO] Sending autorun event
[2020-06-16 22:10:38.451] [INFO] /Users/path/to/some_test.rb was saved - checking if this effects /Users/path/to/rails/project
[2020-06-16 22:10:38.452] [INFO] A test file has been edited, reloading tests.
[2020-06-16 22:10:38.452] [INFO] Loading Ruby tests...
[2020-06-16 22:10:38.452] [INFO] Loading Minitest tests...
[2020-06-16 22:10:38.452] [INFO] Getting a list of Minitest tests in suite with the following command: bundle exec rake -R $EXT_DIR vscode:minitest:list
[2020-06-16 22:10:38.457] [INFO] Sending autorun event
[2020-06-16 22:10:40.058] [INFO] /Users/path/to/some_test.rb was saved - checking if this effects /Users/path/to/rails/project
[2020-06-16 22:10:40.058] [INFO] A test file has been edited, reloading tests.
[2020-06-16 22:10:40.058] [INFO] Loading Ruby tests...
[2020-06-16 22:10:40.058] [INFO] Loading Minitest tests...
[2020-06-16 22:10:40.059] [INFO] Getting a list of Minitest tests in suite with the following command: bundle exec rake -R $EXT_DIR vscode:minitest:list
[2020-06-16 22:10:40.064] [INFO] Sending autorun event
[2020-06-16 22:10:45.598] [INFO] /Users/path/to/some_test.rb was saved - checking if this effects /Users/path/to/rails/project
[2020-06-16 22:10:45.598] [INFO] A test file has been edited, reloading tests.
[2020-06-16 22:10:45.598] [INFO] Loading Ruby tests...
[2020-06-16 22:10:45.598] [INFO] Loading Minitest tests...
[2020-06-16 22:10:45.598] [INFO] Getting a list of Minitest tests in suite with the following command: bundle exec rake -R $EXT_DIR vscode:minitest:list
[2020-06-16 22:10:45.607] [INFO] Sending autorun event
[2020-06-16 22:10:48.158] [INFO] /Users/path/to/some_test.rb was saved - checking if this effects /Users/path/to/rails/project
[2020-06-16 22:10:48.158] [INFO] A test file has been edited, reloading tests.
[2020-06-16 22:10:48.158] [INFO] Loading Ruby tests...
[2020-06-16 22:10:48.158] [INFO] Loading Minitest tests...
[2020-06-16 22:10:48.158] [INFO] Getting a list of Minitest tests in suite with the following command: bundle exec rake -R $EXT_DIR vscode:minitest:list
[2020-06-16 22:10:48.165] [INFO] Sending autorun event

Explorer hangs in refresh after running spec with syntax errors

Your environment

  • vscode-ruby-test-adapter version: v0.1.3
  • Ruby version: ruby 2.7.2p137
  • VS Code version: 1.52.1
  • Operating System: Mac OS Catalina
  • RSpec or Minitest version:
    • RSpec 3.10
      • rspec-core 3.10.1
      • rspec-expectations 3.10.1
      • rspec-mocks 3.10.1
      • rspec-rails 4.0.2
      • rspec-support 3.10.1

Expected behavior

Tests with ruby syntax errors should present as test failures in the sidebar.

Actual behavior

The test refresh indicator spins constantly, other buttons in the extension are unresponsive. Restart of the editor is required to address the issue. Fixing the test without restart does not help.

Screen Shot 2021-01-26 at 12 20 47 PM

Movie:

Screen.Recording.2021-01-26.at.12.51.09.PM.mov

Any help appreciated!

This section should include any relevant screenshots, code samples, console output, etc. The more information we have to reproduce the better!

Input Field to filter tests

I would really like to have an input field to be able to filter the list of tests.

Will have a look at this soon.

Fails to work

Your environment

  • vscode-ruby-test-adapter version: 0.6.0
  • Ruby version: 2.6.3
  • VS Code version: 1.38.1
  • Operating System: Darwin x64 18.7.0
  • RSpec version: 3.8.2

Expected behavior

The plugin should work.

Actual behavior

The plugin does not work. and trying to change its settings yields the following notification:

Unable to write to User Settings because rubyTestExplorer.rspecCommand is not a registered configuration.

Developer tools log:

 ERR Unable to write to User Settings because rubyTestExplorer.rspecCommand is not a registered configuration.: Error: Unable to write to User Settings because rubyTestExplorer.rspecCommand is not a registered configuration.
    at k.reject (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:4448:395)
    at k.resolveAndValidate (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:4450:706)
    at k.doWriteConfiguration (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:4445:819)
    at Object.Promise.resolve.queue.queue [as factory] (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:4445:612)
    at v.consume (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:132:609)
    at Promise (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:132:424)
    at new Promise (<anonymous>)
    at v.queue (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:132:348)
    at k.writeConfiguration (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:4445:597)
    at C.writeConfigurationValue (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:4470:836)
    at cyclicDependency.then (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:4460:594)

This section should include any relevant screenshots, code samples, console output, etc. The more information we have to reproduce the better!

Running a single test / single test file fails if the test file’s path contains spaces

Your environment

  • vscode-ruby-test-adapter version: 0.6.1
  • Ruby version: 2.5.3
  • VS Code version: 1.41.1
  • Operating System: macOS
  • RSpec version: 3.9.0

Expected behavior

Running a single test / single test file works.

Actual behavior

Running a single test / single test file fails if the test file’s path contains spaces.

Cause

When constructing the command for running a single test, the test file’s path is not escaped, see

let testCommand = `${this.getTestCommand()} --require ${this.getCustomFormatterLocation()} --format CustomFormatter ${testLocation}`;

A quick workaround is to add quotes around ${testLocation} like this:

 let testCommand = `${this.getTestCommand()} --require ${this.getCustomFormatterLocation()} --format CustomFormatter "${testLocation}"`;

The proper fix would be to use shell escaping for all the arguments (including the test comannd and the formatter location).

Running a single test file suffers from the same issue, see

let testCommand = `${this.getTestCommand()} --require ${this.getCustomFormatterLocation()} --format CustomFormatter ${testFile}`;

Add Compatibility without Rails / ActiveSupport

Your environment

  • vscode-ruby-test-adapter version: 0.5.4
  • Ruby version: 2.5.3
  • VS Code version: 1.34.0
  • Operating System: Ubuntu

The Ruby in this extension assumes Rails / ActiveSupport is also in play. Here are the changes I had to address to get the extension to work without Rails

  • Constant Autoloading (needs require 'minitest', not sure how RSpec is handled)
  • ActiveSupport Extensions:
    • Hash#as_json
      • Seems to be unnecessary, as JSON.pretty_generate handles this.
    • String#Constantize
    • Hash#deep_dup
    • String#starts_with (similar enough to #start_with)

I was documenting how each of these came up, but it became long and noisy. I think it would be easier to just submit a PR and seek your thoughts.

Thanks!

Add a custom Rspec test formatter

The JSON formatter doesn't include as much info as I'd like to have. I've already been playing around with a custom formatter.

Need to figure out how to get Rspec to use the custom formatter provided by the extension.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.