rzane / file_store Goto Github PK

This wouldn’t be that hard to add, but there are two problems that I see:

1. The maximum number of items returned from an S3 list_objects query is 1,000 items, requiring pagination to return all items. This would potentially result in some differences in the way that the different adapters work (the Disk adapter would not have that limitation) unless some sort of pagination mechanism was always assumed. The ideal here would be to make things work like FileStore.Stat.

2. The S3 list_objects query acts more like find x -type f than ls, in that “subdirectories” are returned as part of the search because the key space in S3 is flat, and the prefix is literally that: a prefix (but the / still acts a little special).

I’d be happy to contribute this.

We finally got around to upgrading and there were a number of incompatibilities (the new Prefix middleware instead of prefix:, etc.).

It’s all good, but the current released version 0.2.2 only supports middleware for FileStore.Config versions where the configuration is specified in config.exs, but master appears to do the right thing (reading either init or config as appropriate). We use module-level configuration, not config.exs configuration.

A new release would alleviate this.

The current application version is set as @version "0.0.0" in mix.exs. Ideally, this should be set to the current (or next) version in the source branch so that a dependency can be specified as:

{:file_store, "~> 0.2", github: "rzane/file_store", branch: "master"}

As it is, we cannot specify the version as anything other than >= 0.0.0 if using a git branch.

It'd be really helpful to see logs like:

[info] Uploaded file to key: 0fq3fadnywlvibplwoi2hg31lq57
[info] Download file from key: 0fq3fadnywlvibplwoi2hg31lq57
[info] Deleted file from key: 0fq3fadnywlvibplwoi2hg31lq57
[error] Failed to delete file from key: 0fq3fadnywlvibplwoi2hg31lq57