Feature request: support for Google Test about neotest HOT 1 CLOSED

Hey thanks for the report/suggestions! Great to hear you're writing an adapter 😁 GTest has been one that's always evaded my support with vim-ultest as well rcarriga/vim-ultest#104 (comment).

test discovery breaks down when opening a file outside the current project.
If I'm working on /home/me/myproject/foo.cpp and go-to-definition in /usr/include/something.h, the whole filesystem tree gets parsed

That shouldn't happen AFAIK, the only case that would happen for would be if the adapter said that the file is a test file. Is that what's happening? If you can grab the logs this should help diagnose if that's not the case.

1. Support canceling a run from build_spec.

This makes sense though it doesn't make sense to use nil to cancel because right now, if an adapter returns nil for a dir, file, namespace, it will try and break up the positions to their children to run. This is how neotest-vim-test works. So then your user would be prompted once for each test in a suite if they ran a suite and cancelled. Instead I've added some error handling around build_spec so that if an error is thrown, a user will be notified using vim.notify with the error message. So you can write No executable provided or something along those lines.

2. Consider supporting errors in a different file.

Yep that's something I've thought about (though adapters would probably still want to provide error messages for the test file as there's no guarantee that the file is open/visible), I'd just add path to neotest.Error and then the diagnostic consumer just needs to be updated to handle that. Happy to take PRs for that 😄

However, printing that error in a different file could result in hundreds of test reporting the same error in that one file, so maybe it's best left as is.

The diagnostics consumer could handle that with a limit per line/buffer? Don't think that'd be a blocker anyway

3. Keeping previous results would be helpful (in a text file somewhere).

Interesting idea, I'd probably keep that as functionality solely in the output consumer which is quite simple so I don't mind making a bit more complex. Thinking something like a version/run_count argument that could be like a python list index e.g. -1 gives last, -2 second last, 0 first etc. That's completely off the top of my head though so totally open to ideas on interface.

4. Providing an interface for an adapter to store a persistent state to disk would be nice.

I'm not sure there's much point in neotest providing a centralised location rather than adapters just using stdpath("data") as you say. It seems like this would have to be arbitrary data from an interface perspective so I don't see the advantage of it.

Finally, I need some guidance with parametrized tests: is there a definitive way to work with them? E.g., @pytest.mark.parametrize in pytest or TEST_P in Google Test. This is really multiple tests masquerading as one and I'm not sure how to report them - should they be different nodes in the tree? Should it just be one test and whenever it's run an adapter should report that all errors happened in that one test?

So this has been raised for neotest-go as well. neotest-pytest will just aggregate each result into the single result (this is one of the reasons a single result can have multiple errors) and then display all outputs together for the "short" output. The neotest-go author had discussed attempting to parse the parameters with treesitter since they are usually static and and so can be handled by treesitter (vscode-go does this to some extent nvim-neotest/neotest-go#1 (comment)). However this is getting into territory where treesitter may be a bit limited and you might want to instead use the test runner discovery if possible (or LSP). I may do that for pytest as neotest-pytest already integrates with it and so can find that information but it could be a lot slower (pytest discovery isn't even close to treesitter speed)

from neotest.