Failed to decompress gzip about librdkafka HOT 12 CLOSED

Us there a minimum size required for compression to work?

from librdkafka.

Do you know what produced this message? Scala Kafka producer? rdkafka producer?

I dont think gzip has any limits on the size of the uncompressed data.

from librdkafka.

My app using rdkafka. When I do no compression it works fine, when I do gzip it doesnot complain on the publisher side (but does in the consumer), and when I do snappy I get a message timed out error.

from librdkafka.

Reproduced this using rdkafka example on both sides

from librdkafka.

Perfect! Can you provide the exact command lines for both producer and consumer as well as your sample message(s)?

from librdkafka.

Note I have not updated from master since Dec 28 in case it matters.

My message was on stdin and was simple like 'hi'.

./rdkafka_example -C -b kafkadevcluster1-1.masked.com:5757,
kafkadevcluster1-2.masked.com:5757,kafkadevcluster1-3.masked.com:5757 -p 8
-t LaraReplicator_kafkacluster4 -o -1

./rdkafka_example -P -b kafkadevcluster1-1.masked.com:5757,
kafkadevcluster1-2.masked.com:5757,kafkadevcluster1-3.masked.com:5757 -p 8
-t LaraReplicator_kafkacluster4

With either –z gzip, -z snappy or (nothing)

On Tuesday, January 7, 2014, Magnus Edenhill wrote:

Perfect! Can you provide the exact command lines for both producer and
consumer as well as your sample message(s)?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/54#issuecomment-31787010
.

from librdkafka.

Gzip decompression should now be working.
I will look more into Snappy tomorrow.

from librdkafka.

Ok cool. The snappy seemed to be a publisher side problem since it
received a local message timeout

On Tuesday, January 7, 2014, Magnus Edenhill wrote:

Gzip decompression should now be working.
I will look more into Snappy tomorrow.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/54#issuecomment-31792333
.

from librdkafka.

Magnus- let me know if you think the snappy fix will be a straightforward one or will require some time- so that I can decide whether to update the code base now to get the gzip fix or to wait to get both. Thanks in advance.

from librdkafka.

I will hopefully have time to look at the snappy issue tonight (utc)
Den 8 jan 2014 13:09 skrev "winbatch" [email protected]:

Magnus- let me know if you think the snappy fix will be a straightforward
one or will require some time- so that I can decide whether to update the
code base now to get the gzip fix or to wait to get both. Thanks in advance.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/54#issuecomment-31826103
.

from librdkafka.

Updated, gzip now works.

On Wednesday, January 8, 2014, Magnus Edenhill wrote:

I will hopefully have time to look at the snappy issue tonight (utc)
Den 8 jan 2014 13:09 skrev "winbatch" <[email protected]<javascript:_e({}, 'cvml', '[email protected]');>>:

Magnus- let me know if you think the snappy fix will be a
straightforward
one or will require some time- so that I can decide whether to update
the
code base now to get the gzip fix or to wait to get both. Thanks in
advance.

—
Reply to this email directly or view it on GitHub<
https://github.com/edenhill/librdkafka/issues/54#issuecomment-31826103>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/54#issuecomment-31831197
.

from librdkafka.

Creating a separate issue for snappy.

from librdkafka.