Default manifest behavior about swarm HOT 11 OPEN

https://gist.github.com/homotopycolimit/7b70436326535ef19028f836680a1f3c

from swarm.

I agree that manifest should not have a hash associated to the empty path.

I think that the empty path entry ' ' could still function as the default, but should resolve to another path (eg. / or /index.html) and that this path then should resolve to a content hash.

If a manifest is called with bzz and there is no default entry, then we should return the HTTP 300 multiple choice error and the error page should link/redirect to the bzz-list:// view.

I also think that the browser should redirect to that path (/ or /index.html) instead of simply displaying the contents.
Pros: It solves all our trailing / problems and problems with relative paths
Cons: It makes URL a little uglier (theswarm.eth/index.html instead of theswarm.eth) --(though note, theswarm.eth doesn't work currently, you need theswarm.eth/)

from swarm.

@homotopycolimit the issue with redirecting the empty path to some other non-empty path then means I can't have content at a path which is a substring of another path. For example, assume I upload an image at the following three paths:

• bzz://example.eth/image
• bzz://example.eth/images/1
• bzz://example.eth/images/2

Then a manifest is stored at /image which has an empty path entry (which will be served for /image) and entries for s/1 and s/2.

With your proposed solution, requests to /image will actually get redirected to /image/, which may not be what I want?

from swarm.

By the way, I'm of the personal opinion that manifests should use the Unix filesystem model, branch on / rather than common prefixes, paths within a manifest don't contain /.

This is suboptimal for very large sites which have many paths in the same directory, but I think that is the exception rather than the rule, and they can use the same optimisations as is done on a filesystem, split your paths into sub-directories.

See ethereum/go-ethereum#14349.

from swarm.

I can't have content at a path which is a substring of another path.

That is a very good point. Is there a way around this? Can't we let a manifest contain a non-manifest content hash at image and also a manifest hash at images?

[
  {"path": "image", "type": "image/png", "hash": "<hash>"},
  {"path": "images", "type": "application/bzz-manifest+json", "hash": "<hash2>"}
]

I agree it can only work if the substring matches something which is not a manifest... otherwise we'd have ambiguity in resolving manifests.... :(

from swarm.

By the way, I'm of the personal opinion that manifests should use the Unix filesystem model, branch on / rather than common prefixes, paths within a manifest don't contain /.

I seem to recall that, since the very beginning already, @nagydani was against treating / as a special breaking character. Dani, can you perhaps weigh in here and fill us in?

from swarm.

Can't we let a manifest contain a non-manifest content hash at image and also a manifest hash at images?

Sure, by using a Unix filesystem model rather than branching on the common prefix 😉.

The way I see it, we have many issues and bugs with the current manifest implementation which would be fixed with a simpler model. We seem to be only maintaining this more complex manifest implementation to keep the JSON manifests small even for large collections of files which haven't already been split on a particular character (e.g. /), but have we benchmarked when this would become a problem? Thousands, millions, billions of entries in a single manifest?

I'd like to see us have a rock solid implementation for the current 99% case (i.e. I generate a static website locally as a directory tree of files and upload it using swarm --recursive up). We could then have a separate swarm --recursive --prefix-trie up which uses prefix-trie manifests rather than splitting on / if that is a requirement for performance reasons.

from swarm.

see also: #178

from swarm.

i would like to keep the question of manifest representation and interpretation separate.
Unix filesystem like interpretation is for the display.
I am resistant to going back to a suboptimal solution (splitting on /)
Also for provable site inclusions, manifest will be represented with POT structures in future so
it is best thought of as a native 'binary' format.

So how about redirecting to / if the url is path is empty, in order for relative urls to work, and in the matching ignore the leading slash. I think there was a long period when this was working.
We should also chop the trailing / of non-empty paths ending in / and redirect (whether we have catchall feature of arbitrary suffixes or not).

Please let me know what problems if any this would have.

from swarm.

So how about redirecting to / if the url is path is empty,

What exactly does this mean? Would we redirect from index.html to index.html/ ?
We can have a manifest which has a file-hash for entry index.html or we can have a manifest which has a manifest hash for index.html and this manifest in turn has the file-hash as the only/default entry.
It is always this second version which gives us problems, no?

and in the matching ignore the leading slash.

would this not lead to ambiguity? What if I have both entries in my manifest?

from swarm.

@zelig can you answer these 10 points: #178 (comment)

from swarm.