Proposal: Allow Bluepill to run non-app hosted tests about bluepill HOT 3 OPEN

I think the first step would be to start a WIP PR to add a few non-app hosted tests and a script to run them via xcodebuild (or however you typically run them). We can then start experimenting with calling xctest directly without needing to call xcodebuild.

from bluepill.

I think this is a good idea and am fully supportive of doing it. I'm not sure how exactly we'd plug it in though... the existing architecture is sort of tied to running simulators. Are there any details of how the non-app hosted tests are run? Is it through the xctest binary? One idea would be to have a different mode for bp that executes the xctest binary directly and outputs the test result fragment that the bluepill binary knows how to merge.

from bluepill.

Are there any details of how the non-app hosted tests are run? Is it through the xctest binary?
The best explanation I've gotten on how these tests are run is through the fbidb documentation.

Logic tests are typically injected into a small host process like the xctest commandline. macOS has a xctest commandline at /usr/bin/xctest/, the iOS Simulator xctest executable is located at $DEVELOPER_DIR/Platforms/iPhoneSimulator.platform/Developer/Library/Xcode/Agents/xctest.

The xctest commandline will load the xctest bundle provided as an argument and run tests according to the arguments, or from an XCTestConfigurationFilePath provided as an environment variable passed to the process.

However they are saying they bypass xcodebuild. We use xcodebuild our implementation and we still get a simulator to boot however it never FULLY boots (never hits the home screen) before the tests finish and it sends back a test passed signal. This is something I think that needs exploring how xcodebuild exactly runs non-app hosted logic tests.

from bluepill.