There will need to be some sort of API/RPC Interface to allow users to:

• Metadata operations, such as view the full topology of a file
• Migration Operations, such as moving a file's storageblocks from one pool to another

Additionally, this will require some way to manage background operations so that operations can be stopped & restarted if the main program exits

Right now metadata is bincoded and stored in two sled databases- inode_db and descriptor_db. These hold the FUSE Inodes and then the Inode Contents.

I'm not sure if this is the best approach. It works and is perfectly acceptable, I just wonder if it would make more sense to store this information in json/yaml/toml files on disk.

This would make disaster recovery easier (as the metadata is easily accessable) and remove a blackbox from the stack (not to say Sled is bad).

Performance Impacts

Because metadata is currently flushed asynchronously, swapping the storage medium doesn't impact I/O performance directly.

There will be some more overhead for serialization, but because that is done in the background; it shouldn't matter much.

Complexity Impacts

I think the complexity is a wash, as this will trade Sled for a module that presents a Key/Value interface that is backed by files in a directory.

There might be some complexity around safely generating inode numbers, but that can just be an incrementing AtomicUsize that is populated on engine start.

This would only swap the underlying datastore that FsDB2 uses as backing, which doesn't change the semantics much.

Possible Issues

The two main issues I see are:

1. This could enable users to modify the state of files in a way that could cause undefined behavior which might result in Dataloss.
2. We need to make sure the module that provides the Database like interface can handle if the file changes underneath it.

#1 doesn't seem like a big issue. 2 could be solved by aggressive permission management and file locking.