volfco / shmr Goto Github PK
View Code? Open in Web Editor NEWVirtual Filesystem for redundently storing data on SMR hard drives
Virtual Filesystem for redundently storing data on SMR hard drives
There will need to be some sort of API/RPC Interface to allow users to:
Additionally, this will require some way to manage background operations so that operations can be stopped & restarted if the main program exits
Right now metadata is bincoded and stored in two sled databases- inode_db and descriptor_db. These hold the FUSE Inodes and then the Inode Contents.
I'm not sure if this is the best approach. It works and is perfectly acceptable, I just wonder if it would make more sense to store this information in json/yaml/toml files on disk.
This would make disaster recovery easier (as the metadata is easily accessable) and remove a blackbox from the stack (not to say Sled is bad).
Because metadata is currently flushed asynchronously, swapping the storage medium doesn't impact I/O performance directly.
There will be some more overhead for serialization, but because that is done in the background; it shouldn't matter much.
I think the complexity is a wash, as this will trade Sled for a module that presents a Key/Value interface that is backed by files in a directory.
There might be some complexity around safely generating inode numbers, but that can just be an incrementing AtomicUsize that is populated on engine start.
This would only swap the underlying datastore that FsDB2
uses as backing, which doesn't change the semantics much.
The two main issues I see are:
#1 doesn't seem like a big issue. 2 could be solved by aggressive permission management and file locking.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.