Slow performance when reading from pub sub and adding to Redis Streams about redis-dart HOT 5 CLOSED

Hey, I now get up to 35k/s essages forwarded, thanks for the help

from redis-dart.

When you call LPUSH you wait for response to be received from server before advancing with second command.

Take a look in Readme when we present fast example. What we do here is that using send().then() approach, we are sending and receiving in asynchronous order - we send multiple "INCR" commands, before we process one.

I am not sure how you can take this approach from fast example with async, because after each async send() you wait for server to respond and that seems to be a bottleneck.

This library does not have requirements that we need to wait for response after each send. What is required is that each send() has exactly one (1) response from redis server.

from redis-dart.

from redis-dart.

from redis-dart.

Simple way you can validate this is something where you modify just the line that sends LPUSH

  void connectToPubSub(String channel) async {
    pubSub!.subscribe([channel]);
    final stream = pubSub!.getStream();
    var streamWithoutErrors = stream.handleError((e) => print("error $e"));
    await for (var msg in streamWithoutErrors) {
      var kind = msg[0];
      var jsonPayload = msg[2];
      if (kind == "message") {
        // final payload = jsonDecode(jsonPayload);
        // await redisCommand?.send_object(
        //     ['XADD', 'soketi-stream', '*', 'payload', jsonPayload]);
        // no await here
        redisCommand
            ?.send_object(['LPUSH', 'soketi-queue', jsonPayload.toString()]).then((_) { /* we dont care what is received */ })
      } else {
        print("received non-message ${msg}");
      }
    }
  }

from redis-dart.