Giter VIP home page Giter VIP logo

merkle_hash's Introduction

merkle_hash

Finds the hashes of all files and directories in a directory tree.

Usage

To use this crate, add merkle_hash as a dependency to your project's Cargo.toml:

[dependencies]
merkle_hash = "3.7"

Features

  • Finds the master hash of a directory tree with ease.
  • Offers multiple hashing algorithms.
  • Allows including names in the hashing process.
  • Uses a merkle tree algorithm to compute the hashes of directories.
  • External iteration over the paths and hashes of files and directories.

Limitations

  • Currently only supports UTF-8 paths and will fail if a path is not UTF-8 encoded.

Optional

  • sha - Add this cargo feature to include SHA-256 and SHA-512 as hashing algorithms.
  • parallel - Enabled by default, this feature makes the crate utilize all available threads.
  • encode - Enabled by default, this feature adds the bytes_to_hex and to_hex_string functions.
  • retain - Disabled by default, this feature duplicates the children paths of directories upon traversal.

Examples

Get the master hash of a directory tree:

use merkle_hash::{Algorithm, MerkleTree};

let tree = MerkleTree::builder("/path/to/directory")
    .algorithm(Algorithm::Blake3)
    .hash_names(false)
    .build()?;
let master_hash = tree.root.item.hash;

Iterate over a directory tree, getting the hash of each file and directory:

use merkle_hash::{Encodable, MerkleTree};

let tree = MerkleTree::builder("/path/to/directory").build()?;
for item in tree {
    println!("{}: {}", item.path.relative, item.hash.to_hex_string());
}

Collapse the tree into any linear collection:

use std::collections::BTreeSet;
use merkle_hash::{MerkleItem, MerkleTree};

let tree = MerkleTree::builder("/path/to/directory").build()?;
let btree_set: BTreeSet<MerkleItem> = tree.into_iter().collect();

Release notes for 3.7

  • Removed dependency on anyhow.
  • Better error handling using a custom IndexingError type.

Versioning

  • Any major version of this crate may contain changes to the hashing algorithm.
  • Any minor version of this crate may contain breaking changes to the API.
  • Any patch version of this crate will only contain bug fixes and no breaking changes.

Used technologies

  • rayon for multithreaded directory reading and hashing.
  • camino to ensure that paths are always utf-8.
  • blake3 for the blake3 hashing of file contents.
  • sha2 for the sha256 and sha512 hashing of file contents.

License

Licensed under MIT license.

merkle_hash's People

Contributors

hristogochev avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

merkle_hash's Issues

Support for Glob Patterns?

Hello ๐Ÿ‘‹ , this is a great library!

I would like to use this library to compute file and directory hashes for a very large directory, but the directory might contain files that I'm not interested in computing hashes for.

Would it be possible for this library to support a glob so that I can filter by the file extensions (or sub-directories) I'm interested in?

Version 3.0.0 Cargo build error out of the box

When upgraded the version to 3.0.0 cargo failed to build throwing internal error as follows:
--> C:\Users\Name\\.cargo\registry\src\github.com-1ecc6299db9ec823\merkle_hash-3.0.0\src\merkle_node.rs:62:21

62 | let Ok(absolute_path) = Utf8PathBuf::from_path_buf(entry?.path())else{
63 | bail!("Path is not valid UTF8 path")
64 | };
| |______________________^
|
= note: see issue #87335 rust-lang/rust#87335 for more information

Environment:
OS: Windows
Rust Version: rustc 1.64.0 (a55dd71d5 2022-09-19) (system)
Cargo rust version: rust-version = "1.57"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.