Problem with heartbeats about mojo-rabbitmq-client HOT 16 CLOSED

It would be very helpful if you showed some information about the AMQP frames with an env var. Like we do in Mojolicious with MOJO_WEBSOCKET_DEBUG.

from mojo-rabbitmq-client.

That's strange. I have bunch of workers based on this module (but created using Mojo::RabbitMQ::Client->consumer() running on RabbitMQ 3.6.5 without any problems.

Could you please share your RabbitMQ server version?

from mojo-rabbitmq-client.

I do not control the server and do not know the version. Could be 3.6.14.

from mojo-rabbitmq-client.

I'll try to do some checks tomorrow, but it looks like an empty frame was received and not properly handled by code.

from mojo-rabbitmq-client.

Please let me know if there is anything i can do to help.

from mojo-rabbitmq-client.

Just tested your code with RabbitMQ 3.5.7, Erlang 18.3, Mojolicious 6.66, Net::AMQP 0.06, Perl v5.23.6 and works like a charm. I just had to drop passive => 1 for exchange to declare properly.
So probably the issue lies somewhere in dependency tree.

from mojo-rabbitmq-client.

If i remove the passive => 1 it never reaches the success or error events and nothing happens.

from mojo-rabbitmq-client.

Btw. That version of Mojolicious is fairly out of date. And Perl 5.23.6 is an early development release, you really shouldn't be using that (although unrelated to the problem at hand).

from mojo-rabbitmq-client.

I just got confirmation that the server i'm testing against also runs RabbitMQ 3.5.7.

from mojo-rabbitmq-client.

I know, my perl build environment is pretty rusty right now.
In the meantime of brewing Perl 5.27.7 to test everything on Mojolicious@master I've made Client to use $ENV{MOJO_RABBITMQ_DEBUG} to enable listening of wire messages, and some initial method exchange in 801d367.

from mojo-rabbitmq-client.

You can see your working example wire log here: https://gist.github.com/SPodjasek/481d67cb42f105bef05665fe50bebbde

from mojo-rabbitmq-client.

That looks very useful.

from mojo-rabbitmq-client.

So, i've started doing some tests again, and this problem still appears to exist. What i'm seeing with MOJO_RABBITMQ_DEBUG activated is very strange. It appears that things break when there is multiple AMQP frames in the same read operation. I've removed most actual message data (replaced with ...) since i couldn't make sure there is nothing confidential in there, but kept all the protocol metadata.

<- "\1\0\1\0\0\0U\0<\0<\37amq.ctag-eC9dqmBs8xIlgD4nOpcdyA\0\0\0\0\0\0\0\330\0\6pubsub suse.obs.package.build_unchanged\316\2\0\1\0\0\0)\0<\0\0\0\0\0\0\0\0\1}\230\0\30application/octet-stream\2\0\316\3\0\1\0\0\1}{\"project\":...}\316"
<- "\1\0\1\0\0\0U\0<\0<\37amq.ctag-eC9dqmBs8xIlgD4nOpcdyA\0\0\0\0\0\0\0\331\0\6pubsub suse.obs.package.build_unchanged\316\2\0\1\0\0\0)\0<\0\0\0\0\0\0\0\0\1~\230\0\30application/octet-stream\2\0\316\3\0\1\0\0\1~{\"project\":...}\316"
<- "\1\0\1\0\0\0U\0<\0<\37amq.ctag-eC9dqmBs8xIlgD4nOpcdyA\0\0\0\0\0\0\0\332\0\6pubsub suse.obs.package.build_unchanged\316\2\0\1\0\0\0)\0<\0\0\0\0\0\0\0\0\1n\230\0\30application/octet-stream\2\0\316\3\0\1\0\0\1n{\"project\":...}\316"
<- "\1\0\1\0\0\0S\0<\0<\37amq.ctag-eC9dqmBs8xIlgD4nOpcdyA\0\0\0\0\0\0\0\333\0\6pubsub\36suse.obs.package.build_success\316\2\0\1\0\0\0)\0<\0\0\0\0\0\0\0\0\1\201\230\0\30application/octet-stream\2\0\316\3\0\1\0\0\1\201{\"project\":...}\316\1\0\1\0\0\0S\0<\0<\37amq.ctag-eC9dqmBs8xIlgD4nOpcdyA\0\0\0\0\0\0\0\334\0\6pubsub\36suse.obs.package.build_success\316\2\0\1\0\0\0)\0<\0\0\0\0\0\0\0\0\1\200\230\0\30application/octet-stream\2\0\316\3\0\1\0\0\1\200{\"project\":...}\316\1\0\1\0\0\0U\0<\0<\37amq.ctag-eC9dqmBs8xIlgD4nOpcdyA\0\0\0\0\0\0\0\335\0\6pubsub suse.obs.package.build_unchanged\316\2\0\1\0\0\0)\0<\0\0\0\0\0\0\0\0\1w\230\0\30application/octet-stream\2\0\316\3\0\1\0\0\1w{\"project\":..."repository\":\"standar"
Mojo::Reactor::Poll: Timer failed: Can't call method "isa" on an undefined value at /usr/lib/perl5/vendor_perl/5.26.2/Mojo/RabbitMQ/Client.pmline 318.

Yes, the frame is cut off like that at the end. I've been testing it with this consumer on a moderately active exchange (maybe 0.5-2 messages per second on average).

use Mojo::Base -strict;

use Mojo::Promise;
use Mojo::RabbitMQ::Client;

Mojo::IOLoop->singleton->reactor->unsubscribe('error');

my $client = Mojo::RabbitMQ::Client->new(url => 'amqp://...:5672');
my $queue_name;
$client->connect_p->then(sub { shift->acquire_channel_p() })->then(sub {
  shift->declare_exchange_p(
    exchange => 'pubsub',
    type     => 'topic',
    passive  => 1,
    durable  => 1
  );
})->then(sub { shift->declare_queue_p(exclusive => 1) })->then(sub {
  my ($channel, $result) = @_;
  $queue_name = $result->method_frame->{queue};
  $channel->bind_queue_p(
    exchange    => 'pubsub',
    queue       => $queue_name,
    routing_key => '#'
  );
})->then(sub {
  my $channel = shift;

  my $promise = Mojo::Promise->new;
  my $consumer = $channel->consume(queue => $queue_name, no_ack => 1);
  $consumer->on(error => sub { $promise->reject('Consumer error') });
  $consumer->on(
    message => sub {
      my $frame   = pop;
      my $key     = $frame->{deliver}->method_frame->routing_key;
      my $message = $frame->{body}->to_raw_payload;
      say "\n*** $key: " . length $message;
    }
  );
  $consumer->deliver;

  return $promise;
})->catch(sub { warn shift })->wait;

Hope this helps with tracking down the problem.

from mojo-rabbitmq-client.

Looking at the responsible code, there appears to be something very very wrong. Net::AMQP->parse_raw_frames is already removing the frames it parses from the buffer, yet there is custom buffer manipulation happening here. If i understand Net::AMQP correctly you shouldn't have to mess with the protocol, ever, and just let it deal with the low level bits.

from mojo-rabbitmq-client.

The actual bug here seems to be that _parse_frames recurses with next_tick and then removes half a frame from the buffer, which causes the error. The correct fix will be to get rid of all that useless buffer handling. I'll see if i can make a patch tomorrow.

from mojo-rabbitmq-client.

I think this problem has been resolved with the 0.2.0 release.

from mojo-rabbitmq-client.