Question: how does JuiceFS interact with S3? about juicefs HOT 3 CLOSED

Currently, JuiceFS does NOT merge small files together to reduce the PUT calls, because that will slow down future read and make it harder the manage the life cycle of data. We had tried this before, it does not work well, so abandoned.

JuiceFS could cache the small files in your local disk, if you read them frequently. So it can reduce the cost for GET requests.

from juicefs.

Thank you for the quick confirmation.

I understand your comments about making lifecycle management more complicated.

However, S3 supports range-based retrievals. So it should actually be possible to make retrievals across multiple S3 sub-objects faster as you could parallelise the range reads across multiple objects.

from juicefs.

That's true if we disable the compression in JuiceFS. But the caching is still a problem, for example, in order to cache a tiny file, you have to cache all the other files in same 4MB block.

from juicefs.