Comments (4)
BurntSushi
commented on May 25, 2024
Nope. It seems totally orthogonal to me. You can always compress them after the split command.
from xsv.
d33tah
commented on May 25, 2024
@BurntSushi Thanks for replying! The problem is that you need to store files in the uncompressed form in the meantime. Which, for huge CSV files - which benefit from splitting the most - can be quite cumbersome. In my case I decided to mount a compressed btrfs system to satisfy xsv's requirements, but having a way to compress data before saving to the disk would IMO solve the problem better.
from xsv.
LemmingAvalanche
commented on May 25, 2024
@d33tah See qsv snappy.
from xsv.
d33tah
commented on May 25, 2024
@LemmingAvalanche thanks!
from xsv.
Related Issues (20)
- Use file name patterns in cat HOT 1
- New build needed for `reverse`
- Scanning of "cargo-geiger" showed unsafe code HOT 1
- xsv might silently fail
- Cargo test failed on Windows
- fixlengths -- insert extra commas not at end
- Escaping unescaped quotes in quoted strings modifies the data HOT 2
- support unbalanced partitioning HOT 1
- sample the whole file HOT 1
- bug: xsv count HOT 3
- Frequency command causes allocated memory exhausted message on csv file with many fields
- Add Parquet support
- fixlengths is missing the field delimiter option for output (-t)
- what format is accepted for non printable delimiter?
- support flexible format csv
- Escape square bracket in column header? HOT 1
- xsv table displays values with escaped `"`s incorrectly HOT 7
- `sample` command with alternative delimiter: Could not deserialize to u64 for '<sample-size>'. HOT 2
- `xsv join --cross` shouldn't require columns to match HOT 1
- crc32 a column's values?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xsv.