:bulb: "Split" archive support. about lrzip-next HOT 2 CLOSED

The rzip pre-processing phase scans only a chunk at a time. For example, when I archive a virtual machine of 75G, it takes 8 passes to compress the whole file.Each pass becomes a chunk (as lrzip-next sees it) and each chunk is compressed as a series of blocks and is a subset of the whole file. So what you are suggesting is not possible.

However, if what you are suggesting is compress a chunk at a time, i.e. compress chunks 1-3 only, or 4-6 only, while possible, would be very messy since the file would have to pieced back together again on decompression.

A different approach would be to use a program like dd to pass only so many bytes to lrzip-next. Using a combination of block size and count, and or skip, you can approximate your desire.

$ dd if=enwik9 bs=1M count=100 | lrzip-next -fo partofenwik9.lrz -L1  -
Using configuration file /home/peter/.lrzip/lrzip.conf
The following options are in effect for this COMPRESSION.
Threading is ENABLED. Number of CPUs detected: 8
Detected 16,558,112,768 bytes ram
Nice Value: 19
Show Progress
Verbose
Overwrite Files
Output Filename Specified: partofenwik9.lrz
Temporary Directory set as: ./
Compression mode is: LZMA. LZ4 Compressibility testing enabled
Compression level 1
RZIP Compression level 1
Initial LZMA Dictionary Size: 262,144
SHA256 Hashing Used
Heuristically Computed Compression Window: 52 = 5,200MB
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.0590955 s, 1.8 GB/s
Per Thread Memory Overhead is 9,322,496
Beginning rzip pre-processing phase
Compression Ratio: 3.036. bpb: 2.635. Average Compression Speed: 33.333MB/s.
Total time: 00:00:02.48

I'll leave this open

from lrzip-next.

I think the dd command would achieve what you seek. The risk is that changes occur to the file you are archiving mid-way.

from lrzip-next.