Giter VIP home page Giter VIP logo

Comments (2)

milesgranger avatar milesgranger commented on June 1, 2024

Thanks for the note about incorrect docs. 👍

As for Compressors, it depends what you want it for: (forgive me if I explain anything which you're already aware of, this also helps me)

  • As implemented, they are only for framed/stream formats. This is because formats like snappy framed or gzip is composed of independently compressed pieces; 'raw/block' in snappy framed and deflate streams for gzip. Both of which will append or prepend additional information to headers and/or footers. Same goes for Lz4. And also why none of the underlying crates implement Read/Write for block/raw formats, as we saw with the discussion in snappy and lz4 missing Read/Write traits in block format

In summary, Compressor maintains the state until complete, then wrapping up any header/foot annotations for these formats.

  • Now do you want a Compressor for these block/raw formats to avoid repeated instantiations of underlying encoders/decoders in some hot code path?

This might be worth looking into if this is your aim. I can try to set aside some time to benchmark it a bit to see if exposing some Encoder/Decoder for block/raw formats would be beneficial in such a situation. But it serves a distinctly different purpose than existing Compressors'.

from cramjam.

martindurant avatar martindurant commented on June 1, 2024

The hope was to avoid python bytes.join before passing data for compression, but it's a very minor thing. I am surprised, though. It should not require one frame per input to compress a stream (so long as the only flush() is the final one), using some internal buffer. But if the third party library doesn't provide the API, so be it!
Please don't devote any time to this.

from cramjam.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.