Handling of absolute file:// paths about universal_pathlib HOT 6 CLOSED

I also think that this should be addresses on the fsspec side.

The upath side seems to work consistently. So, you could replace

path = UPath("file:///opt/foo/bar")
fs.ls(path)

with

path = UPath("file:///opt/foo/bar")
path.iterdir()

from universal_pathlib.

My understanding is that this is correct (as of Feb 2017), as RFC8089 has made the entire authority component of a file URI, including the double slashes, optional.

A file URL takes the form:

file://<host>/<path>

When the are 3 slashes after the file: schema, this means that the host has been omitted from the URI. If the host is blank, the double slash preceding it is not required, so universal_pathlib doesn't include it when formatting the path.

If you include an authority component (i.e. a server name) in your file URI, you will see that all three slashes get printed - the two before the server name, and the one at the start of the path:

>>> print(UPath("file://localhost/usr"))
file://localhost/usr

from universal_pathlib.

@joouha Ah ok this makes sense thank you!

The reason I hit the issue was that I have some code using fsspec to handle various file operations. That code gets file URLs from elsewhere, and I'd like to be able to pass them UPath's instead because that seems convenient. But if I have

path = UPath("file:///opt/foo/bar")
fs.ls(path)

Fails because the cutting of the slashes means fs treats it as a relative path (with "file:" being part of the path) and looks for ./file:/opt/foo/bar

Whereas if I use the string directly with the triple slashes:

fs.ls("file:///opt/foo/bar")

Then this gets treated as an absolute path and is handled correctly by fs.

The first snippet with UPath works whenever the path is s3/gcp/anything else, and it feels like that's how it's meant to be used. So maybe the issue is in a discrepancy in how fsspec handles these single slash paths?

from universal_pathlib.

Yep how the strip protocol works in fsspec's Local implementation treats a single slash and triple slash differently:

https://github.com/fsspec/filesystem_spec/blob/master/fsspec/implementations/local.py#L213-L217

Happy to close this issue out if you think I should open an issue in filesystem_spec?

from universal_pathlib.

Personally I think this ought to be fixed in fsspec - but I'm not a maintainer here 😉

@normanrz ?

from universal_pathlib.

Gotcha makes sense, I'll put a PR in on the fsspec side, thanks!

from universal_pathlib.