cehteh / bintest Goto Github PK

When setting up integration tests in my project, I was seeing infrequent, non-deterministic test failures which I eventually narrowed down to being a result of the test binary running each test in parallel, which therefore runs BinTest in parallel, which it appears has a chance of interfering with the running of the compiled excutable in another test.

Based on the discussion here, I think it's worth, as you say, either explicitly documenting the recommendation that BinTest should only be ran once (using lazy_static or Lazy or similar) or handling it internal to BinTest as part of the API

Brainstorm for supporting multiple configurations in future:

• make one singleton per configuration OnceLock<RwLock<HashMap<BinTestBuilder, BinTest>>>
• each configuration builds in its own --target-dir (target/bintest/<hash_over_config>/)