Question: Delay between MQTT messages about homie-esp8266 HOT 16 CLOSED

I am not looking at the current implementation of async-mqtt-client, but having that queue is a great improvement. Before that queue was implemented, messages not fitting into the TCP send buffer have just been dropped and have not been sent at all. Devices with larger amounts of properties did not work at all.
If the queue now waits for the acknowledge of sent messages, then 1 second for this acknowledgement sounds very large to me. I am running mosquitto in Docker, too, but in x86-64 hardware. Acknowledgements from this mosquitto are much faster than 1 second. So, I guess your problem is not really the homie-esp8266 or async-mqtt-client's queue but the slow response of your MQTT broker.

from homie-esp8266.

OK, since I changed nothing to the default IOTStack installation I have a closer look what might be the problem with Mosquitto. I'm glad it is not a general problem.

from homie-esp8266.

What do you mean with "not a general problem"? Is it just one device, or a couple? Perhaps bad WiFi reception?

from homie-esp8266.

Sorry, that was not quite clear. I mean "a general homie/asyc-mqtt-client problem that affects everyone"

I tested now with different devices, different homie-projects, all show the same behavior.
My WiFi is fine, and the "older" devices in my home automation (not updated to latest homie) do not show that behavior.

As said, it is not a real showstopper for me. But I was curious if someone else has seen that behavior.

By now I don't know how to optimize the mosquitto broker, I turned off persistance with no effect.
So for now I will live with it.

from homie-esp8266.

Persistance is not the problem. Maybe setting tcp_nodelay to true will help:

set_tcp_nodelay [ true | false ]
If set to true, the TCP_NODELAY option will be set on client sockets to disable Nagle's algorithm. This has the effect of reducing latency of some messages at potentially increasing the number of TCP packets being sent. Defaults to false.

This option applies globally.

Reloaded on reload signal.

Especially for the Ack messages: they are small and the TCP stack might queue them, or in case no further data is coming, sent after (typically) 500msec.

from homie-esp8266.

I do not see a change with nodelay set to true.

from homie-esp8266.

I have several devices using the homie framework which send more than 20 messages on startup. None of them shows the behaviour of yours.
This really looks like the 500ms ack delay is hitting you. Or you have a 500ms delay somewhere in your code.

from homie-esp8266.

OK, I created a test-project, doing nothing except the homie setup. It stays the same....
Maybe google has an answer to the 500ms ACK delay issue.

from homie-esp8266.

I did see the same behaviour. It doesn't surprise me.

Goal is still to speed up sending: "more than 1 message in-the-air".

from homie-esp8266.

Ok, I'll double-check. Maybe I was wrong. I noticed that since the update to asyncmqtt it takes longer to send a bunch of messages but I never had the impression that there was a 500ms delay between messages.

from homie-esp8266.

BTW, the Mosquitto version is 2.0.14, in case that matters.

from homie-esp8266.

It turns out that this could be a problem in daily life.
In case I press a button while the stats are published in worst case it could take a few seconds for the message to be published.
That is to slow e.g. to turn a light on.
Maybe there are some ideas to solve the problem out there...

from homie-esp8266.

OK, finally I managed to see what is going on at TCP-level.

But I don't understand what exacly is going on.....

1. the mqtt message is send from esp to mosquitto.
2. mosquitto sends the mqtt ACK (no delay)
3. homie sends a TCP-ACK with a notable delay (~300ms)
4. direct after that, homie sends the next mqtt message.

So the delay is not happing on the mosquitto broker, but in homie.
Any ideas about that are highly appreciated.

from homie-esp8266.

Is it possible to have a look on your code?

from homie-esp8266.

Yes, but the test project contains nothing but the homie setup and loop, and one advertised node.
Homie, AsyncMqtt and ESPAsyncTCP are from latest github commits.

#include <Arduino.h>
#include <Homie.h>
#define FW_VERSION "0.0.1"
#define FW_NAME "homietest"
#define DEBUGNODE_TYPE "debug"
#define DEBUGMESSAGE "debug_message"

HomieNode debugNode(DEBUGNODE_TYPE, DEBUGNODE_TYPE, DEBUGNODE_TYPE);

void setup()
{
Serial.begin(115200);
Homie_setBrand("IoT"); // before Homie.setup()
Homie_setFirmware(FW_NAME, FW_VERSION); // The underscore is not a typo! See Magic bytes
Homie.disableLedFeedback(); // before Homie.setup()
debugNode.advertise(DEBUGMESSAGE);
Homie.setup();
}

void loop()
{
Homie.loop();
}

Maybe the "Delayed ACK"-issue is located in the used ESPAsyncTCP/lwip stack.
As far as I can see there is the option to disable "Nagle's algorithm" (which is done in AsyncMqttClient via _client.setNoDelay(true);)
But there seems to be no way to disable the delayed ACK...

from homie-esp8266.

OK, after hours of reading about LWIP etc. I was not able to track that problem down.
The workaround for me to get a stable environment is to set the async-mqtt-client back to commit 7f1ba481a22d56 (before the queue was introduced).
I know there has been a lot of bugfixes since than, but I'm not using TLS or anything, and with that version I don't have any problems...
I can close this issue, it will be fixed when the async-mqtt-client will not causing that delay anymore.

from homie-esp8266.