Improve performance by allowing "Files: exclude" rules when collecting test binaries about vscode-catch2-test-adapter HOT 11 CLOSED

Loved your detailed feature request.

Seems to me they improved a lot on the documentation and the features.

I was checking the files.watcherExclude option.

Configure paths or [glob patterns](vscode-file://vscode app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/code/electron-sandbox/workbench/workbench.html) to exclude from file watching. Paths can either be relative to the watched folder or absolute. Glob patterns are matched relative from the watched folder. When you experience the file watcher process consuming a lot of CPU, make sure to exclude large folders that are of less interest (such as build output folders).

How would findFiles(..., watcherExclude, ...) work for you?
The vscode.workspace.createFileSystemWatcher has been already respecting it so regardless we find those executables change events won't be coming.

from vscode-catch2-test-adapter.

Thanks! :-)

I tried vscode.workspace.getConfiguration("files").get("watcherExclude"), and indeed it works like a charm!

We'd be super happy if this got fixed upstream.

from vscode-catch2-test-adapter.

from vscode-catch2-test-adapter.

CI closed it but this change is a bit tricky, I reopen it until verification.

• what exactly the value of vscode.workspace.getConfiguration("files").get("watcherExclude") ?
• what is a typical path of your test exec?
• where are the files which meant to be ignored?

from vscode-catch2-test-adapter.

Overnight the plugin got automagically upgraded to 4.8.0, and this immediately started to work, possible for everyone, super fast!!! Thank you!

what exactly the value of vscode.workspace.getConfiguration("files").get("watcherExclude") ?

besides the default nodejs-related entries, I added

bazel-*/**
external/**

Works with this, either added here or simply under files.exclude.

what is a typical path of your test exec?

All test executables are under bazel-bin/, on a (sometimes deep) path which is the same as the git-committed Bazel BUILD file defines the binary. Example:

• If there is a cc_test called MyTest defined in long/path/for/my/test/BUILD.bazel
• Then the MyTest GoogleTest ELF binary will reside in bazel-bin/long/path/for/my/test/MyTest
• And it is symlinked as support/vscode/testmate/long/path/for/my/test/MyTest

Under support/vscode/testmate only these symlinks exist, none of the unrelated Bazel stuff.

where are the files which meant to be ignored?

Under bazel-bin/ there are many more files, these must be excluded. (And bazel brings in some other symlinked directories like bazel-out, bazel-gitreponame, bazel-testlogs. Just to see the difference it makes for us, an example from our repo:

# All the Bazel overhead:
± find bazel-bin/ | wc -l
367510

# Our trimmed down symlinks
± find support/vscode/testmate | wc -l
7996

# A realistic developer environment, with actual non-broken symlinks (tests that are built):
± find support/vscode/testmate -type l -not -xtype l | wc -l
10

from vscode-catch2-test-adapter.

Additional note:

• bazel-out, bazel-bin, and the likes are actually symlinks to $USER/.cache/_bazel/...; they are created by Bazel.

Somewhat unrelated, but now that I understand more correctly that this is an actual FS Watcher, it's meant to watch these binaries and once they are updated (by a Bazel build recompiling them), TestMate supposedly recognizes them...?

If this is the case, this actually never worked for us.

from vscode-catch2-test-adapter.

There is a FSWatcher and a findFiles. findFiles makes sure that already exiting files/executables are listed (if the testing framework is supported). The FSWatcher makes sure that new executables are added, changed ones are reloaded.

If this is the case, this actually never worked for us.

I heard from other users that symlinks are not followed by the watchers.

Now that you excluded

I'm not familiar with bazel build system and other users have issues too. It might worth to look into it a bit sometime.
As far as my understanding goes is that bazel produces the output to $USER/.cache/_bazel/... and the workspace folder only contains symlinks to the actual folders (as yourself explained too).
Symlinks broke the FSWatcher but listing works (two different endpoint and functionality).
This could be bypassed with some checking and GazeWatcher could operate nicely outside of workspace folder but I'm concerned that what is happening when rebuild is done. Ex. does bazel keep the old binaries and rewrites the symlink, etc.. I can imagine a lot of magic.

(If that's a the case you can try adding the symlink to dependsOn. That would trigger reloads.)

I will have to dig deeper of bazel. This article I found just now.
I will need some time to play around.

One more question: what is your pattern?

from vscode-catch2-test-adapter.

I did some more experimentation, especially with moving my symlinks (the ones that I create for testmate to be recognized) "close" to the bazel caches, so that there is no file system boundary crossing (our dev envs have ~/.cache mounted separately)

Although then it's the GazeWrapper that is being used, however, automated test detection/reload works. The reason for that I think, is that once the symlinks are resolved (by Gaze, vscode, the kernel, or whatever layer), the inode will still be on the same filesystem, and thus kernel's inotify would cover it.

Now I am on mobile, but will post an update about my findings next week.

from vscode-catch2-test-adapter.

I had to realise changing the default behaviour was a mistake. v4.10.0 reverts v4.8.0: files.watcherExclude is not applied by default anymore.
Added new setting: testMate.test.advancedExecutables -> exclude which can be "files.watcherExclude" or any setting path.

from vscode-catch2-test-adapter.

How do you specify the cwd for bazel?

from vscode-catch2-test-adapter.

How do you specify the cwd for bazel?

We use advancedExecutables, and have a shell script for intercepting TestMate's logic (testmate-wrapper.sh) around detecting tests. It's quite sophisticated as in setting up some paths, env variables, conditionally analyzing the test binary (readelf) and eventually, for test invocation, dispatching to Bazel itself (so that tests are rebuilt on-demand). Since at the end it's Bazel that is running the test, our cwd is simply ${workspaceDirectory} (which is the same as the Bazel root)

Bazel's approach of having many (as in, thousands of) test binaries makes it a bit hard to work with this plugin; we had to implement our per-file "test detection" mechanism, which is also done through the same wrapper, but what it really does, is compiling the particular file's nearest Bazel target. The massive amount of test binaries also caused performance problems, like this very report.

We don't like this too much. In fact, we are looking into improving the UX by trying to parse the test .cpp files for GoogleTests so that our users don't have to compile them. If you are interested in this, we should think about some kind of collaboration.

from vscode-catch2-test-adapter.