fetch: Enable fetch via file URL (under flag) about undici HOT 11 OPEN

I’m ok in supporting this under a allowFileUrl option in fetch.

from undici.

Related: wintercg/fetch#5

from undici.

I'm also still not sure how fetching file urls can be used in both browsers and runtimes, but that doesn't seem relevant.

tbf it doesn't need to be supported in browsers to improve interop, the use case I have in mind is fetch(new URL('./some/relative/path', import.meta.url)), where import.meta.url would be a HTTPS URL in browsers and a file: URL on runtimes.

from undici.

Moving to Undici repository

from undici.

What would be the shape of it, and its considerations?

From what I've seen from the issue shared by @aduh95 the main security points seem valid; Chromium seems to have some initial implementation.

All assuming its behind a flag, ofc

from undici.

If we implement this we will deviate from literally every other platform. We should either let wintercg handle it, or let the spec handle it. -1

from undici.

@KhafraDev I agree with you that it’d be optimal to have wintercg to formalize this. What do you mean by “will deviate from literally every other platform”? Other runtimes (Bun and Deno for sure) implement a this already

from undici.

From a quick glance:

1. Bun allows fetching file:// urls with any method, Deno throws on non-GET calls.
2. Chrome only allows this for extensions, if users opt-in to it.
3. Firefox and Safari do not implement this.

If we implement our own implementation (ie. with a non-standard option), that's another implementation that differs from every browser and every server environment. No implementation is spec compliant because there is no spec, which will also cause issues when a spec is written, and these platforms now have to break their users to match it.

There are also a number of edge cases that haven't been decided on wintercg/fetch#5 (comment)

I'm quite confused as to your mention of wasm-pack

Right now, libraries like wasm-pack (a Rust to WASM lib) have to perform different builds for browsers and Node.js, since Node.js is not able to fetch the compiled WASM binary from the file system.

Browsers aren't able to do this either. But from looking at the code, they instead download the binary from a url, which will work in node and browsers, ignoring CORs errors. Maybe you could expand on this?

from undici.

No implementation is spec compliant because there is no spec, which will also cause issues when a spec is written, and these platforms now have to break their users to match it.

This is very true, but that's also the purpose of experimental APIs. I personally think that missing APIs cause more damage to the user than experimental APIs.

Browsers aren't able to do this either. But from looking at the code, they instead download the binary from a url, which will work in node and browsers, ignoring CORs errors. Maybe you could expand on this?

Correct. Right now, if you use wasm-pack, you have to run at least three compilation processes:

1. For browsers and runtimes. Since the mainstream runtimes support fetching from a file, they can share the same bindings built for the browsers.
2. ESM bindings for Node.js, which doesn't support fetch for local files
3. CJS bindings for Node.js, which doesn't support fetch for local files

The only real blocker I found to enable a single compilation process, it's by allowing Node.js to fetch the compiled WASM binaries from the filesystem at runtime.

I wouldn't focus too much on the wasm-pack use case only, though. I'm using Deno a lot and being able to avoid calling server-side methods (such as fs.readFileSync for Node.js) allows you to write code once and use it both on the frontend and the backend, which is kinda handy.

I understand the spec is basically non-existent, yet, as I said earlier, an experimental API is far better than no API at all to me.

But of course, that's a personal opinion and I'm happy to discuss

from undici.

But we aren't missing any apis, we implement fetching file urls exactly as the spec tells us to (throw an error). I would claim that Deno and Bun, by implementing this, cause much more damage in the long run (in the style of mootools or others that added methods to Array/Object prototypes, preventing the tc39 from adding certain methods).

I'm also still not sure how fetching file urls can be used in both browsers and runtimes, but that doesn't seem relevant.

from undici.

Thanks for the use case, makes sense now! Interop with other server environments is precisely the point of WinterCG. In the past we've followed Deno, which has only ever caused issues for us (namely security vulnerabilities, edge cases, more edge cases, other issues). But the proposed solution is not interoperable with other runtimes as it requires explicitly opting in to it, for every request - the other two runtimes just work. As in, you would have to change your code for it to work in node, whereas you don't for other runtimes, nor will you when/if it's ever added into the spec.

I mentioned earlier, but Bun isn't entirely interoperable with Deno, and there are also a number of questions about the api that WinterCG should solve before we even consider implementing it.

from undici.