Evaluate the cost of relocatable files in parallel filesystems about julia HOT 2 OPEN

Thank you very much for taking this up. One possible solution I see is to not cache source files at the time of loading, but at the time of modification for all non-dev'd packages. This would require a strong assumption that package files are not modified except through Pkg.jl operations. But then staleness could be verified by loading a single file with the precomputed crc32 hashes and comparing them against the pkgimage files' crc32s.

from julia.

I think your proposal is close to a custom CAS system.

Regarding the restriction:
At least on my system all pkg code inside ~/.julia/packages/... is already read-only. However, I vaguely remember having seen an issue about this not being the case on all platforms, but I can't find it at the moment. Anyways, I think this makes sense, because the source files are released code that can be frozen.

Regarding keeping hashes in a separate file:
This requires perfect synchronization with the sources, and is susceptible to cases where someone (for some reason) swaps out source files.
Another solution would work with a virtual filesystem that stores the hash alongside the inode data. This would solve the synchronization issue, but it also requires more work for support on all platforms.
And yet another solution could use the LLVM CAS library.

These thoughts raise the questions:

• How foolproof does the solution have to be?
• Is it enough to put out a disclaimer saying that altering the ~/.julia/packages folder comes at the user's own risk for malfunction?
• How likely is it that people tinker with this data?

from julia.