Add transparent compression handling in the Response struct. about fasthttp HOT 10 CLOSED

Currently the client doesn't set Accept-Encoding headers to gzip or deflate, so conforming server should never send compressed response with 'Content-Encoding' response header.

Transparent compression may be added to the client as opt-in setting in Client and HostClient. EnableCompression is a good name for this setting. You may create pull request for this feature.

Enabling transparent compression in fasthttp client by default is no go, since default compression implementation in Go is slooow ;)

from fasthttp.

My idea was to check for the 'Content-Encoding' header in the response, and decompress the body if the header was present.
I didn't indent do modify the request sending path, only receiving. We could enable this behaviour by default, and disable it by setting 'AutoDecompress' to false in the client/response, where the value from the response would take precedence.

Now, We could add a EnableCompression parameter also, to transparently compress the requests, but this is a story for another pull request.

In our use case, we use haproxy to compress the responses sent from the server ( We do set 'Content-Encoding' in the request header explicitly), and then check for the 'Content-Encoding' in the response.

from fasthttp.

Nice idea! I'll try implementing it on the next week.

from fasthttp.

@szank , I decided adding helper functions for body compression instead of transparent compression. This allows using response body compression only when needed.

See Response.BodyGunzip and Response.BodyInflate.

from fasthttp.

Added CompressHandler for transparent response compression on server side.

from fasthttp.

There is github.com/klauspost/compress a fully gzip compatible drop in replacement for go standard gzip/zip/zlib packages.
Claimed to be up to 3x faster for data compression.
Description and benchmarks by the author:
http://blog.klauspost.com/go-gzipdeflate-benchmarks/
http://blog.klauspost.com/gzip-performance-for-go-webservers/

pgzip Go parallel gzip compression/decompression. This is a fully gzip compatible drop in replacement for "compress/gzip". Faster for big files more than 1MB.

cgzip wraps zlib using cgo.

from fasthttp.

What about transparent decompression of incoming requests to server?

from fasthttp.

Just added Request.BodyGunzip helper.

Transparent compression / decompression won't be implemented in fasthttp, since this may be abused by specially crafted malicious requests / responses, which may eat a lot of CPU and memory resources. So fasthttp users must manually compress / decompress request / response bodies when needed.

from fasthttp.

I think this issue may be closed now. Feel free creating new issues for unresolved topics mentioned above.

from fasthttp.

Thanks! Isn't BodyInflate() missing?

from fasthttp.