npm module not found when running turbo app about mirrord HOT 23 CLOSED

perhaps add DYLD_INSERT_LIBRARIES as well? and add to mirrord.dev?

from mirrord.

@t4lz suggested it might be because node is signed, but node is same node users had before the upgrade, so it's irrelevant.
In order to test it, I stripped the signature from turbo's Node and this still occurs.
I found that this command also reproduces it (needs to have "tampered binaries" for it to occur, atleast one run as @t4lz described)
mirrord exec /var/folders/2h/fn_s1t8n0cqfc9x71yq845m40000gn/T/mirrord-bin/usr/bin/env node /var/folders/2h/fn_s1t8n0cqfc9x71yq845m40000gn/T/mirrord-bin/Users/aviramhassan/Code/mirrord-sip-npm-issue/node_modules/.bin/turbo run dev
If that's not weird enough, I just get an empty line break if I try
mirrord exec env node turbo run dev

from mirrord.

Also this
mirrord exec env node /Users/aviramhassan/Code/mirrord-sip-npm-issue/node_modules/.bin/turbo run dev

from mirrord.

I found out that if you add --no-daemon to the command it works - I saw it connects to a socket and my theory now is that the script sends to the daemon it's path to load stuff for it, and we don't cover that flow.

Edit:
Nope, tester error.

from mirrord.

ok, @t4lz might be right it being related to node being signed, since all the tester issues was actually when I tried to codesign to see if that fixes it, but each time I tried a different node binary (nvm vs volta) and when I patched both it always worked.

from mirrord.

I feel like calling exec ourselves will create issues, especially when from extension.
I think a better solution would be to ship a binary that will act as the interpreter, and we'll just change the sip patch logic to it -> mirrordsh will be a binary that receives arguments to binary to call but will make sure mirrord is loaded (via dlopen?) then do it. SIP patch will now just prepend mirrordsh - probably we can bundle that binary like we bundle the dylib?

from mirrord.

I thought it might be getcwd but nope :(

from mirrord.

I thought it might be getcwd but nope :(

I think we should be looking at ways an application can get the current executable's path, not the CWD's path.

from mirrord.

This is where node threw an error for the user that faced this issue: https://github.com/nodejs/node/blob/v18.17.0/lib/internal/modules/cjs/loader.js#L1080

from mirrord.

I know - I checked all symbols that are related to file access - Node needs to read the file and I can't see any access to it via any APIs. - I was looking for an API that has ../lib/cli.js or part of it.
Given it only happens with Turbo, I assume it might be Turbo doing something funky?
https://github.com/vercel/turbo/blob/main/crates/turbopack-cli/src/dev/mod.rs#L68

from mirrord.

Here is the node error the user got, the one I got and posted in the issue's text is unfortunately a bit different:

│ node:internal/modules/cjs/loader:1080
│   throw err;
│   ^
│
│ Error: Cannot find module '/var/folders/j7/sjfkn72n6w17zd4z7b4b17d80000gq/T/mirrord
│ -bin/Users/some-user-name/.volta/tools/image/npm/9.6.4/bin/npm-cli.js'
│     at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
│     at Module._load (node:internal/modules/cjs/loader:922:27)
│     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:
│ 81:12)
│     at node:internal/main/run_main_module:23:47 {
│   code: 'MODULE_NOT_FOUND',
│   requireStack: []
│ }
│
│ Node.js v18.17.0
│ command finished with error: command (/Users/some-user-name/some-app
│ ) /Users/some-user-name/.volta/tools/image/npm/9.6.4/bin/npm run dev exited (1)
└────>
┌ web-launcher#dev > cache bypass, force executing **redacted**
│ node:internal/modules/cjs/loader:1080
│   throw err;
│   ^
│
│ Error: Cannot find module '/var/folders/j7/sjfkn72n6w17zd4z7b4b17d80000gq/T/mirrord
│ -bin/Users/some-user-name/.volta/tools/image/npm/9.6.4/bin/npm-cli.js'
│     at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
│     at Module._load (node:internal/modules/cjs/loader:922:27)
│     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:
│ 81:12)
│     at node:internal/main/run_main_module:23:47 {
│   code: 'MODULE_NOT_FOUND',
│   requireStack: []
│ }
│
│ Node.js v18.17.0

from mirrord.

I think I found the problem:
argv:

["/Users/aviramhassan/.volta/tools/image/node/18.17.0/bin/node","/var/folders/2h/fn_s1t8n0cqfc9x71yq845m40000gn/T/mirrord-bin/Users/aviramhassan/.volta/tools/image/node/18.17.0/bin/npm","run","dev"]

which means node loads the npm script from the tmpbin then tries to find stuff in relation to it

from mirrord.

ok. I found the issue. We don't get loaded - how I found out - I edited the sip patch npm file it's erroring about to print proces.env and saw that it's quite empty - turbo launches a new node, not inheriting any of the env to it, including DYLD!!!
AHHHHHhh

from mirrord.

Nice. We could try changing the execve hook to add DYLD_INSERT_LIBRARIES to the env if not there, right?

from mirrord.

Yup, on it.

from mirrord.

We really don't like the env stripping - the argv stripping doesn't work - because $TMPDIR isn't set, then it tries to strip another path.
I'm changing the logic of SIP to just identify mirrord-bin-RANDOM - adding a random prefix (that is hardcoded) so we'll just identify based on mirrord-bin-RANDOM.
geeeeee

from mirrord.

Probably same issue but on Linux https://discord.com/channels/933706914808889356/950020257156833340/1276641706078372021

from mirrord.

Reproduced with this example

from mirrord.

I've wasted so much time, should've done it as the first step:

razz4780@razz4780-machine:~/MetalBear/mirrord-sip-npm-issue$ file /home/razz4780/MetalBear/mirrord-sip-npm-issue/node_modules/turbo-linux-64/bin/turbo
/home/razz4780/MetalBear/mirrord-sip-npm-issue/node_modules/turbo-linux-64/bin/turbo: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=53079539de775c245d18ee32c9754ccb04c8b1f8, stripped

;___;

from mirrord.

oh no :O

from mirrord.

maybe there's out of the box feature for turbo to passing env? it's a bit weird it strips everything :|

from mirrord.

Ok, I managed to make it work. This has to merged into turbo.json:

{
    "globalPassThroughEnv": ["MIRRORD_*", "LD_PRELOAD"]
}

from mirrord.

perhaps add DYLD_INSERT_LIBRARIES as well? and add to mirrord.dev?

metalbear-co/mirrord.dev#335

from mirrord.