SnowFS is a high-performance command-line application and node library for Windows, macOS and Linux with a focus on binary file versioning. It is made for the graphics industry and was initially developed for Snowtrack.
Disclaimer: This project is in beta and actively developed.
-
Supports Branches
-
Asynchronous File Hashing
-
Project open to file-content awareness (e.g:
*.psd,*.blend,*.c4d,..) -
Super-fast-detection of modifications in large binaries
-
Support for instant snapshots**
-
Support for instant rollback**
-
Support for files bigger >4TB
-
Block-cloning and Copy-on-Write support for APFS, ReFS and Btrfs
-
Support for removing single versions and/or binaries
-
Primarily I/O bound through libuv
-
Feature XYZ made by you!
** If the underlying filesystem supports it (e.g. APFS, ReFS and Btrfs)
First and foremost, the implementations of Git - namely Git/Git-LFS
and libgit2 are excellent implementations of version control systems.
But due to their focus on the software development lifecycle they are not suitable to version binaries or graphic files.
SnowFS addresses the technical challenges for graphic files by its core design.
Advantages:
- Support on all major platforms
- Supported by hosting platforms like GitHub, GitLab, and BitBucket.
- Fast diff-operation for text-files
Disadvantages:
- If not properly tracked, binaries become accidentally part of "base" history
- Removing older commits is cumbersome due to Gits commit hashing integrity
- Complicated rewriting history procedure
- Issues with binaries >4GB on Windows as reported here, here, and here
- Slow in binary modification detection
Advantages:
- Faster zipping, packing, and delta-compression than the reference implementation Git
- Supports custom backends
Disadvantages:
- No native support for Git-LFS without custom backends
- Custom backends break compatibility with Git
SnowFS is currently written in TypeScript. It is a great language to write powerful and performant
I/O bound prototypes. We are aware of the demand for a C++ implementation, and due to
our roots as C++ developers, we are very interested in a backport as well. It would make SnowFS easier
to integrate into other projects. If you have comments, ideas or recommendations, please let us know.
We have also implemented a comparison benchmark between SnowFS vs. git-lfs. After executing the build instructions for a development build, the benchmarks can be executed with the following command:
$ npm run benchmarkExample run on a Macbook Pro (2020) with an APFS formatted SSD to check-in, delete and restore a 4GB Photoshop File.
...
git lfs track *.psd
git add texture.psd: 20164ms
snow add texture.psd: 4596ms
git rm texture.psd: 575ms
snow rm texture.psd: 111ms
git checkout HEAD~1: 9739ms
snow checkout HEAD~1: 77ms <-- Yeah!
You can find the best and up-to-date code examples in the test/ directory. Given below these are simply "Hello World!" examples to get you started.
import * as fse from "fs-extra";
import { join } from "path";
import { Index } from "./src";
import { Repository } from "./src/repository";
export async function main() {
const repoPath = "/path/to/a/non/existing/directory";
// Create a new repository.
let repo: Repository = Repository.initExt(repoPath);
// Copy a file called 'texture.psd' to the working directory.
await fse.copyFile("/path/to/texture.psd", join(repoPath, "texture.psd"));
// Retrieve the main index.
const index: Index = repo.ensureMainIndex();
// Mark 'texture.psd' as a new file in the index.
index.addFiles(["texture.psd"]);
// Write all files of the index to the object database.
await index.writeFiles();
// Create a commit for the index.
await repo.createCommit(index, "This is my first commit");
}
main();The CLI of SnowFS offers some basic functionality and is subject to enhancements. See Report #90
$ snow init foo
$ cp /path/to/texture.psd foo/texture.psd
$ cd foo
$ snow add .
$ snow commit -m "My first texture"
$ snow log
$ snow checkout -b MyNewBranch
$ snow log
-
To build
SnowFSinstall node.js for your specific platform.1.1. On Windows, Visual Studio is required. We recommend the Visual Studio 2019 Community Edition. During the installation, please enable Desktop development with C++.
1.2. On MacOS, XCode is required.
-
To build a development build execute:
$ git clone https://github.com/Snowtrack/snowfs.git
$ cd snowfs
$ npm install
$ npm run test- To build a production build including the CLI, execute the commands above, and continue with the commands below:
$ npm run tsc
$ npm run build
$ cd distFor the development of SnowFS we recommend VSCode. The repository contains a launch.json file with pre-defined runner configurations. For more information, please visit this pull-request.
Starting with version 1.0.0 SnowFS follows the semantic versioning
scheme. The API change and backward compatibility rules are those indicated by
SemVer.
SnowFS is licensed under the GPL v3 license, please review the LICENSE file.
Excluded from the license are images, artworks, and logos. Please file a request by mail, if you have any questions.
The tests and benchmarks also serve as API specification and usage examples.
Currently, Windows, macOS, and Linux are supported. SnowFS works on plain filesystems like FAT, NTFS, HFS+ and has extended support for APFS, ReFS and Btrfs.
See the guidelines for contributing.
![deepsource-autofix[bot] avatar](https://avatars.githubusercontent.com/in/57168?v=4 "deepsource-autofix[bot]")
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent
