Giter VIP home page Giter VIP logo

madfs's Introduction

MadFS

workflow workflow workflow

Prerequisites

  • MadFS is developed on Ubuntu 20.04.3 LTS and Ubuntu 22.04.1 LTS. It should work on other Linux distributions as well.

  • MadFS requires a C++ compiler with C++ 20 support. The compilers known to work includes GCC 11.3.0, GCC 10.3.0, Clang 14.0.0, and Clang 10.0.0.

  • Install dependencies and configure the system
    • Install build dependencies

      sudo apt update
      sudo apt install -y cmake build-essential gcc-10 g++-10
    • Install development dependencies (optional)

      # to run sanitizers and formatter
      sudo apt install -y clang-10 libstdc++-10-dev clang-format-10
      # for perf
      sudo apt install -y linux-tools-common linux-tools-generic linux-tools-`uname -r`
      # for managing persistent memory and NUMA
      sudo apt install -y ndctl numactl
      # for benchmarking
      sudo apt install -y sqlite3
    • Configure the system

      ./scripts/init.py
  • Configure persistent memory
    • To emulate a persistent memory device using DRAM, please follow the guide here.

    • Initialize namespaces (optional)

      # remove existing namespaces on region0
      sudo ndctl destroy-namespace all --region=region0 --force 
      # create new namespace `/dev/pmem0` on region0
      sudo ndctl create-namespace --region=region0 --size=20G
      # create new namespace `/dev/pmem0.1` on region0 for NOVA (optional)
      sudo ndctl create-namespace --region=region0 --size=20G
      # list all namespaces
      ndctl list --region=0 --namespaces --human --idle
    • Use /dev/pmem0 to mount ext4-DAX at /mnt/pmem0-ext4-dax

      # create filesystem
      sudo mkfs.ext4 /dev/pmem0
      # create mount point
      sudo mkdir -p /mnt/pmem0-ext4-dax
      # mount filesystem
      sudo mount -o dax /dev/pmem0 /mnt/pmem0-ext4-dax
      # make the mount point writable
      sudo chmod a+w /mnt/pmem0-ext4-dax
      # check mount status
      mount -v | grep /mnt/pmem0-ext4-dax
    • Use /dev/pmem0.1 to mount NOVA at /mnt/pmem0-nova (optional)

      # load NOVA module
      sudo modprobe nova
      # create mount point
      sudo mkdir -p /mnt/pmem0-nova
      # mount filesystem
      sudo mount -t NOVA -o init -o data_cow  /dev/pmem0.1 /mnt/pmem0-nova
      # make the mount point writable
      sudo chmod a+w /mnt/pmem0-nova           
      # check mount status
      mount -v | grep /mnt/pmem0-nova          
    • To unmount the filesystems, run

      sudo umount /mnt/pmem0-ext4-dax
      sudo umount /mnt/pmem0-nova

Build and Run

  • Build the MadFS shared library

    make BUILD_TARGETS="madfs"
  • Run your program with MadFS

    LD_PRELOAD=./build-release/libmadfs.so ./your_program

Development

  • Build

    # usage: make [build_type] 
    #             [CMAKE_ARGS="-DKEY1=VAL1 -DKEY2=VAL2 ..."] 
    #             [BUILD_TARGETS="target1 target2 ..."] 
    #             [BUILD_ARGS="..."]
    
    # build the MadFS shared library and tests
    make

madfs's People

Contributors

shawnzhong avatar chenhao-ye avatar josehu07 avatar qusuyan avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.