blazebit / blaze-storage Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
The idea is to introduce something like a bucket object retention plan which can be assigned to bucket objects. If needed we could introduce a default assignment to objects based on e.g. bucket object prefix.
The plan defines a retention period e.g. 30 days and an action. Interesting actions are
The use case is to provide a temporary storage for e.g. attachments that are used when sending an email. An attachment can be put into the bucket with a retention of e.g. 30 days. An e-mail is sent with the bucket objects as attachment. The bucket object is kept around for a while in case it is necessary to debug or re-send the e-mail.
Right now we only set the state of a file to deleted but don't actually delete the file.
A custom task that runs periodically should delete files and on complection also remove the database entries and update pending statistics.
That task could also be triggered in an after-transaction-synchronization-callback (CDI event)
Would be great to use updatable entity views to reduce the amount of queries.
Use https://github.com/dCache/nfs4j for implementing access to buckets through NFS mounts.
We have to think about how to assign "persistent" inode numbers to bucket objects and also to their "virtual directories" as that is what NFS needs to identify objects.
One possible implementation would be to simply not make inodes persistent, but just cache them in memory. This would be ok for uses that only do reads and creates, but even then this could be painful. All clients have to reset their local inode caches after a restart.
A much better implementation would be to allow a global configuration, that when enabled, will maintain a inode number on the bucket_object table based on a sequence through Blazebit/blaze-persistence#365
For directories, we would need a separate table where inodes are managed.
We need an API that allows streaming data. Currently, we expect an input stream to be available, which could be lazy, but usually is not. Being able to write to the final outputstream within the correct context would be nice if possible with JAX-RS
Implement a copy operation that allows to do efficient copying of files on the same storage.
Add some more tests to cover the admin/owner security model.
See https://www.ovh.com/blog/dealing-with-small-files-with-openstack-swift-part-2/ for how we could structure files on the file system to avoid running into filesystem limitations.
Implement uploading of multiple files at once.
Asynchronous copying of bucket objects between storages should be supported e.g. for archivation. Currently, this is only supported in a synchronous fashion.
Right now we test the REST API manually by using it in a front end web application. We should create end-to-end tests for it to catch possible regressions.
Currently we don't publish the sources JAR for WAR projects which results in not being able to reuse the Java classes used for the admin UI. The maven WAR plugin should be adapted to publish this artifact as well.
Like S3, we should implement support for bucket events with support for the following sinks
We should also implement the alerting based on these events.
The event publishing options are
Implement the possibility to download multiple files at once in a multipart response.
Finish the simple object delete operation and also allow to delete multiple objects at once.
Currently we do not really use the checksums. This should change.
Implement ACL security
It should be possible to publicly share files via a link. We could use a UUID for that purpose.
Right now we only collect statistics on storage and bucket level. We should also do that on a per-tag basis.
Implement support for extended feature for HTTP GET of objects like
Also see: https://www.nurkiewicz.com/2015/06/writing-download-server-part-ii-headers.html
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.