fauxfaux / ext4-rs Goto Github PK

Read files directly from ext4 filesystem images

License: MIT License

Rust 93.18% Shell 1.87% Makefile 0.67% Ruby 2.00% Python 2.27%

ext4-rs's Introduction

Status

ext4-rs can extract the basic stat information, directory listings, and file content from real images generated by other tools, and by the Linux kernel.

This operates directly on partitions. To read actual disc images, probably need to handle partition tables. This can be accomplished with the bootsector crate.

All basic file types are represented: files, directories, symlinks, char and block devices, fifos and sockets. Hard links are not a type of thing that makes sense: the item is just in multiple directories.

Practical problems

No support for extended flags (e.g. immutable, append-only).
Probably contains overflows and fencepost errors, many of which Rust will translate to panics for you. At least, in debug mode.

Non-goals

This is not a filesystem driver. It does not support efficiently modifying real filesystems. Currently, it doesn't support modifying anything at all, but that may change.

I'm not especially interested in resource-constrained platforms: memory and IO are not used efficiently.

Changelog

0.10.0: move to positioned_io2 over weird seeking; rust 1.59
0.7.0: use rust 1.34 features to remove cast dependency
0.6.0: use failure for error handling
0.5.0: update bitflags for associated constants, and rename some public constants.
0.4.1: fix for an infinite loop parsing directory entries

MSRV

Rust 1.59 (Debian Bookworm / edition = "2021") is supported, and checked by CI. Updating this is a semver bump.

Development

Tests

Some of the tests read generated image files. These images are not directly checked into git, and will be unpacked by build.rs. These files are apparently very large, but should take very little space. This requires a decent tar to be on the path.

These test assets can be rebuilt (on Linux, with root) by running ./extract-test-data --refresh.

License note

The code is licensed under the super-permissive MIT license.

However, a number of struct and bitfield definitions, and some maths expressions, are lifted directly from the Linux, or e2fsprogs, source code. These code-bases are under the GPLv2. I believe this to be fair use: these places are the documentation, and only interface definitions have been extracted, no code. I leave the final decision to you.

Thanks

This would have been practically impossible without the work a Djwong has done on the ext4 disk layout page on the kernel wiki. I believe this person is Darrick Wong. Thanks, man.

ext4-rs's People

Contributors

Stargazers

Watchers

Forkers

twebor pombredanne xvilka samhansen jbaum98 enqt-gmbh ashyisme ggvgc losynix timglabisch passware oberien isabella232 marcograss property404 nicholasbishop

ext4-rs's Issues

UnexpectedEof, error: "failed to fill whole buffer"

Reproduction:

cd tool
touch file
cargo run -- dump-ls file

I have ext4 but I see

Error: while processing 'file'

Caused by:
    0: failed to parse superblock
    1: failed to fill whole buffer

I added

            println!("ERR: {:?}", error);

in on_fs, and seen:

ERR: Custom { kind: UnexpectedEof, error: "failed to fill whole buffer" }

Reader seems to be ok

reader File { fd: 3, path: "/home/daniel/exp/ext4-rs/tool/file", read: true, write: false }

Example fails on "invalid magic number: 0"

rwalk@walkubu:/mnt/data3/git2/RustyNode$ sudo target/debug/rusty_node /dev/sdc
thread 'main' panicked at 'ext4 volume: failed to parse superblock

Caused by:
filesystem uses an unsupported feature: "invalid magic number: 0"', src/main.rs:10:63
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

`extern crate ext4;

use std::env;
use std::fs;

fn main() {
let r = fs::File::open(env::args().nth(1).expect("one argument")).expect("openable file");
let mut options = ext4::Options::default();
options.checksums = ext4::Checksums::Enabled;
let vol = ext4::SuperBlock::new_with_options(r, &options).expect("ext4 volume");
let root = vol.root().expect("root");
vol.walk(&root, "/", &mut |_, path, _, _| {
println!("{}", path);
Ok(true)
})
.expect("walk");
}`

walk() runs forever on e2fsprogs-1.42.12/tests/f_holedir3/image

Obviously something going wrong here!

ext4tool doesn't build

I get:

error[E0432]: unresolved import `anyhow`
  --> src/main.rs:16:5
   |
16 | use anyhow::Error;
   |     ^^^^^^ maybe a missing crate `anyhow`?

error[E0432]: unresolved import `anyhow`
  --> src/main.rs:17:5
   |
17 | use anyhow::Context;
   |     ^^^^^^ maybe a missing crate `anyhow`?

error: cannot find macro `anyhow` in this scope
  --> src/main.rs:81:39
   |
81 |     on_fs(file, work).with_context(|| anyhow!("while processing '{}'", file))?;
   |                                       ^^^^^^

error[E0423]: expected function, found builtin type `usize`
  --> src/main.rs:50:23
   |
50 |         let to_read = usize(std::cmp::min(inode.stat.size, u64(bytes)));
   |                       ^^^^^ not a function
   |
help: consider importing this function instead
   |
9  | use cast::usize;
   |

error[E0423]: expected function, found builtin type `u64`
  --> src/main.rs:50:60
   |
50 |         let to_read = usize(std::cmp::min(inode.stat.size, u64(bytes)));
   |                                                            ^^^ not a function
   |
help: consider importing this function instead
   |
9  | use cast::u64;
   |

warning: unused `#[macro_use]` import
 --> src/main.rs:5:1
  |
5 | #[macro_use]
  | ^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

error[E0599]: the method `root` exists for struct `SuperBlock<R>`, but its trait bounds were not satisfied
  --> src/main.rs:23:20
   |
23 |     let root = &fs.root()?;
   |                    ^^^^ method cannot be called on `SuperBlock<R>` due to unsatisfied trait bounds
   |
   = note: the following trait bounds were not satisfied:
           `R: positioned_io::ReadAt`

error[E0599]: the method `walk` exists for struct `SuperBlock<R>`, but its trait bounds were not satisfied
  --> src/main.rs:24:8
   |
24 |     fs.walk(root, "", &mut |_, path, inode, enhanced| {
   |        ^^^^ method cannot be called on `SuperBlock<R>` due to unsatisfied trait bounds
   |
   = note: the following trait bounds were not satisfied:
           `R: positioned_io::ReadAt`

error[E0599]: the method `root` exists for struct `SuperBlock<R>`, but its trait bounds were not satisfied
  --> src/main.rs:38:19
   |
38 |     let root = fs.root()?;
   |                   ^^^^ method cannot be called on `SuperBlock<R>` due to unsatisfied trait bounds
   |
   = note: the following trait bounds were not satisfied:
           `R: positioned_io::ReadAt`

error[E0599]: the method `walk` exists for struct `SuperBlock<R>`, but its trait bounds were not satisfied
  --> src/main.rs:39:8
   |
39 |     fs.walk(&root, "", &mut |fs, path, inode, _| {
   |        ^^^^ method cannot be called on `SuperBlock<R>` due to unsatisfied trait bounds
   |
   = note: the following trait bounds were not satisfied:
           `R: positioned_io::ReadAt`

error[E0277]: the trait bound `RangeReader<&mut BufReader<File>>: positioned_io::ReadAt` is not satisfied
  --> src/main.rs:69:45
   |
69 |               work.exec(ext4::SuperBlock::new(bootsector::open_partition(
   |  _____________________________________________^
70 | |                 &mut reader,
71 | |                 &part,
72 | |             )?)?)?;
   | |______________^ the trait `positioned_io::ReadAt` is not implemented for `RangeReader<&mut BufReader<File>>`
   |
   = note: required by `SuperBlock::<R>::new`

error[E0277]: the trait bound `BufReader<File>: positioned_io::ReadAt` is not satisfied
  --> src/main.rs:74:51
   |
74 |         Err(_) => work.exec(ext4::SuperBlock::new(reader)?)?,
   |                                                   ^^^^^^ the trait `positioned_io::ReadAt` is not implemented for `BufReader<File>`
   |
   = note: required by `SuperBlock::<R>::new`

error: aborting due to 11 previous errors; 1 warning emitted

Some errors have detailed explanations: E0277, E0423, E0432, E0599.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `ext4tool`

To learn more, run the command again with --verbose.

Cannot access ext4 through Rust use declaration

The following code fails with cargo build.

use ext4;

fn main() {
println!("Hello, world!");
}

#############################################
Build output:

rwalk@walkubu:~/git2/RustyNode/RustyNode$ cargo build
Compiling rusty_node v0.1.0 (/mnt/data3/git2/RustyNode/RustyNode)
warning: unused import: ext4
--> src/main.rs:1:5
|
1 | use ext4;
| ^^^^
|
= note: #[warn(unused_imports)] on by default

warning: rusty_node (bin "rusty_node") generated 1 warning
Finished dev [unoptimized + debuginfo] target(s) in 0.19s
rwalk@walkubu:~/git2/RustyNode/RustyNode$

#############################################
Cargo.toml:
[package]
name = "rusty_node"
version = "0.1.0"
edition = "2021"

[dependencies]
ext4 = "0.9.0"

#############################################
Cargo show output:
rwalk@walkubu:~/git2/RustyNode/RustyNode$ cargo show ext4

id: ext4
name: ext4
description: Read information from EXT4 filesystem images
documentation: None
homepage: None
repository: https://github.com/FauxFaux/ext4-rs
max_version: 0.9.0
downloads: 4635
license: None
created: 2017-06-03T11:31:57.606611+00:00
updated: 2021-01-28T06:58:25.428059+00:00
rwalk@walkubu:~/git2/RustyNode/RustyNode$

directory checksum mismatch parsing ext4 volume boot directory

I've been trying to use this library to parse an ext4 volume and list the files contained however it always seems to crash on the /boot/grub/x86_64-efi directory with the error:

assumption failed: "directory checksum mismatch: on-disk: b1f7e838, computed: c72c0128"'

e2fsck and mounting the image produce no such errors/indications there is anything wrong with the volume

AssumptionFailed { reason: "outside long mode, block group desc size must be zero, not 32" }

I am getting this error trying to read an ext4 filesystem extracted from an Android device. The filesystem mounts fine on a standard Linux kernel and fsck.ext4 reports no errors.

Commenting out the corresponding check in parse.rs allows ext4-rs to parse the filesystem and correctly extract files from it.

However I am afraid that we might introduce data corruption issues by simply skipping this check.

Bump version

Hello, would it be possible to bump the version of the crate ? Thank you

docs.rs build failing

The build on docs.rs does not succeed, resulting in no published documentation. The reason for this is that source folders like src/ and tests/ are set to read-only, since build scripts should never write to them in the first place. The correct location to place generated files is instead OUT_DIR.