Comments (2)
Thanks for the note about incorrect docs. 👍
As for Compressors, it depends what you want it for: (forgive me if I explain anything which you're already aware of, this also helps me)
- As implemented, they are only for framed/stream formats. This is because formats like snappy framed or gzip is composed of independently compressed pieces; 'raw/block' in snappy framed and deflate streams for gzip. Both of which will append or prepend additional information to headers and/or footers. Same goes for Lz4. And also why none of the underlying crates implement
Read
/Write
for block/raw formats, as we saw with the discussion in snappy and lz4 missing Read/Write traits in block format
In summary, Compressor
maintains the state until complete, then wrapping up any header/foot annotations for these formats.
- Now do you want a
Compressor
for these block/raw formats to avoid repeated instantiations of underlying encoders/decoders in some hot code path?
This might be worth looking into if this is your aim. I can try to set aside some time to benchmark it a bit to see if exposing some Encoder
/Decoder
for block/raw formats would be beneficial in such a situation. But it serves a distinctly different purpose than existing Compressor
s'.
from cramjam.
The hope was to avoid python bytes.join before passing data for compression, but it's a very minor thing. I am surprised, though. It should not require one frame per input to compress a stream (so long as the only flush() is the final one), using some internal buffer. But if the third party library doesn't provide the API, so be it!
Please don't devote any time to this.
from cramjam.
Related Issues (20)
- Accept buffers with types other than u8
- pyo3_runtime.PanicException: Failed to import NumPy module HOT 2
- blosc? HOT 5
- Publish Python 3.12 wheel? HOT 6
- Any plans to release pypy wheels for windows? HOT 2
- TypeError: 'Buffer' does not support the buffer interface HOT 4
- Python test test_variants_different_dtypes[brotli] sometimes times out HOT 4
- lzma / xz support? HOT 12
- Remove extra zstd-safe dep
- Please coordinate PyPI and crates.io releases if possible HOT 2
- Equality check on values for Buffer
- proc-macro2 1.0.56 doesn't work with versions of rust since July
- cargo_audit does not like brotli-sys HOT 3
- Please consider adding license metadata to PyPI HOT 2
- Add arm64/universal wheels for Python 3.11 under MacOS HOT 1
- zlib-ng? HOT 5
- use with cargo? HOT 2
- macos wheels aren't built for the macos versions they advertise HOT 6
- Support for memoryview and PickleBuffer HOT 17
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cramjam.