Feature Request: RSpec Adapter about neotest HOT 17 CLOSED

@bmulholland @alxekb could you both check neotest-rspec and raise any issues in the repo? Functionality should be there but suspect there will be some edge cases:

As I mentioned above, creating IDs via Treesitter to match to RSpec is a nuisance and my fix is a tonne of string replacement. Long term, I'd love to be able to conjure up some way of making neotest + treesitter form ids which match RSpec's id key (example: "id": "./spec/neotest_spec.rb[1:2:1]",).

from neotest.

@bmulholland I started working on an rspec adapter today btw. Treesitter queries are done and I just need to build out the NeotestAdapter.results function. Will try and sort this weekend.

from neotest.

@hahuang65 the "JSON jumble" should be no more in the latest release and you should delicious color output from RSpec now.

from neotest.

Looks great! 😁 I think this can be closed alright, issues with the adapter can be filed there now

from neotest.

Sounds good! Happy to help with any questions/issues 😁

from neotest.

So you can customise the IDs assigned to the positions to be whatever you want, it's totally arbitrary as long as they're unique. You can do so by passing a position_id function in the treesitter parsing opts. The default is here https://github.com/nvim-neotest/neotest/blob/master/lua/neotest/lib/treesitter/init.lua#L148. It looks like you can just concatenate with a space instead of "::" and it should be easy to match the results then

from neotest.

It shouldn't, the size of repos which that relates to is massive. I've been testing using the CPython repo which is easily handled with 724 test files, containing nearly 22000 tests. It is possible that on some machines that many files are opened at the same time before they start closing. It's weird that I've not seen anyone else encountering it but definitely still possible.

Actually just spotted that neotest-rspec is using lib.treesitter.parse_positions_from_string instead of lib.treesitter.parse_positions. parse_positions contains a sleep to prevent hogging resources, it could also be preventing the issue seen here, @olimorris could you switch to using that (and remove the line where you read the file contents) and then we can see if that helps

from neotest.

Ooooh good spot.

@zidhuss pushed a fix which may solve your issue.

from neotest.

Basic functionality is now complete. Running lua require("neotest").run.run() and lua require("neotest").run.run(vim.fn.expand("%")) works on simple use cases.

One of the most challenging parts of this was understanding the results collection process. To align the id from neotest.Tree with RSpec's output, I had to do some messy string manipulation based on the full_description key in the output:

{
    "version": "3.11.0",
    "examples": [
        {
            "id": "./spec/neotest_spec.rb[1:1:1]",
            "description": "returns 4",
            "full_description": "Some maths calculations passing test returns 4",
            "status": "passed",
            "file_path": "./spec/neotest_spec.rb",
            "line_number": 3,
            "run_time": 0.000176,
            "pending_message": null
        }
    ],
    "summary": {
        "duration": 0.000548,
        "example_count": 1,
        "failure_count": 0,
        "pending_count": 0,
        "errors_outside_of_examples_count": 0
    },
    "summary_line": "1 example, 0 failures"
}

My example RSpec test is:

# neotest_spec.rb
describe 'Some maths calculations' do
  context 'passing test' do
    it 'returns 4' do
      expect(2 + 2).to eq(4)
    end
  end
end

@rcarriga would appreciate your input on how to make this better...it's possible I'm misunderstanding how to collect results and there maybe a much cleaner way of doing it. The challenge I will run into with string manipulation is how I deal with tests which have :: in their namespaces.

from neotest.

@olimorris thank you so much, going to check this.

from neotest.

The simple example that you have in your repo works fine for me.

However it was failing silently in a bigger project. Looking through the neotest logs there were lots of instances of the following error:

ERROR | 2022-06-13T13:43:03Z+0400 | ...te/pack/packer/start/neotest/lua/neotest/client/init.lua:415 | Couldn't find positions in path REDACTED_spec.rb .../pack/packer/start/neotest/lua/neotest/lib/file/init.lua:15: EMFILE: too many open files: REDACTED_spec.rb

Current config:

require("neotest").setup({
	adapters = {
		require("neotest-python")({
			dap = { justMyCode = false },
		}),
		require("neotest-rspec"),
		require("neotest-vim-test")({
			ignore_file_types = { "python", "vim", "lua", "ruby" },
		}),
	},
})

from neotest.

How many tests do you have in your spec folder?

from neotest.

Around 4700 tests in ~400 spec files.

from neotest.

Daym! @rcarriga could this also be related to #13?

from neotest.

I'm now up and running with @olimorris' adapter 🎉

Should I close this out @rcarriga ?

from neotest.

Have you been able to see the messages in the output section? I get a JSON jumble, so I'm assuming I set something up incorrectly?

from neotest.

@hahuang65 Yeah that's a consequence of how the adapter works so far. I filed olimorris/neotest-rspec#9 for you. neotest-rspec is intended to be by the community for the community, so your help in moving it forward (even a little) would be really helpful. Perhaps you could investigate how neotest-pytest does it to see which approach neotest-rspec should consider?

(And @rcarriga -- if you have guidance on how to make the test output human friendly while having convenient ways to parse the test results for programmatic things like gutter signs, that would probably be helpful for other adapters too.)

from neotest.