Giter VIP home page Giter VIP logo

dynfilefs's Introduction

dynfilefs

Fuse filesystem for dynamically-enlarged file (can be mounted as loop device too)

usage: ./dynfilefs -f storage_file -m mount_dir [ -s size_MB ] [ -p split_size_MB ] [ -d ]

Mount filesystem to [mount_dir], provide a virtual file [mount_dir]/virtual.dat of size [size_MB] All changes made to virtual.dat file are stored to [storage_file] file(s)

-d                       - Do not fork to background, debug mode

--mountdir [mount_dir]
-m [mount_dir]           - Path to a directory where the fileszstem will be mounted
                         - The directory must be empty, else it will refuse to mount

--file [storage_file]
-f [storage_file]        - Path to a file where all changes will be stored
                         - If file exists, it will be used
                         - If file does not exist, it will be created empty

--size [size_MB]
-s [size_MB]             - The virtual.dat file will be size_MB big

--split [split_size_MB]
-p [split_size_MB ]      - Maximum data size per storage_file. Multiple storage files
                           will be created if [size_MB] > [split_size_MB].
                           Beware that actual file size (including index of offsets) may be
                           bigger than split_size_MB, so use max 4088 on FAT32 to be safe.

Example usage:

./dynfilefs -f /tmp/changes.dat -s 1024 -m /mnt
mke2fs -F /mnt/virtual.dat
mount -o loop /mnt/virtual.dat /mnt

How to compile:

./configure
make

How to compile statically:

DynFileFS can be statically compiled with buildroot, but it's a bit tricky.
Read buildroot/README for more information. Tested on buildroot-2022.02.8
Pre-built static binary for newest version can be found in ./static/ directory.

WARNING!

Master branch is work in progress, may be broken. See releases for "stable" versions. Latest version is 4.03 (2023) Releases can be found here: https://github.com/Tomas-M/dynfilefs/releases

dynfilefs's People

Contributors

tomas-m avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

vinely teamos-hub

dynfilefs's Issues

Support multiple storage files in different locations

Motivation

  1. use multiple partitions on multiple physical disks (USB, NVME, SATA, ...) to store large data (videos, etc.)
  2. overcome VFAT limitation of ~4GByte per storage file
    a. to allow for saving say pictures (~7MByte each) amounting to >4 GByte size
    b. to allow for saving say 4K videos (~150GByte each) amounting to tens of terabytes of storage

Solution

Support multiple storage files in different locations. The order on command line could set their priority for "bucket filling". Large files (not fitting in whole to any of the free spaces on either of the multiple storage files) could be split by binary division (this is to keep as much as possible of the original data contiguous - good for data recovery, for speed as apps often expect this, etc.).

Another cool trick would be to store files/chunks in parallel to each of the storage files using a "work stealing" algorithm (which automatically accounts for e.g. having two storage files on the same physical disk) but I think it would be an unnecessary complication code-wise with not so much speed gains in practical scenarios (but this needs testing).

License? Etc?

Was curious of licensing, whether its mit, gpl, etc...

Also, side note, but how good is it compared to ext4 and others feature wise?

Just curious

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.