bertmelis / espmqttclient Goto Github PK
View Code? Open in Web Editor NEWMQTT 3.1.1 client library for the Espressif devices ESP8266 and ESP32 on the Arduino framework.
Home Page: https://www.emelis.net/espMqttClient/
License: MIT License
MQTT 3.1.1 client library for the Espressif devices ESP8266 and ESP32 on the Arduino framework.
Home Page: https://www.emelis.net/espMqttClient/
License: MIT License
espMqttClient on ESP32 (non-async version) creates its own task which calls loop()
to manage networking.
Allow the user to call loop()
themselves.
This is not possible now because loop()
is marked private on ESP32.
Hello Bert,
Long time no see.
I was happy to see your MQTT library (V1.1.3).
But when I try to compile a simple project for the ESP32 I get the following error:
Compiling .pio\build\wemos_d1_mini32\libf63\espMqttClient\espMqttClient.cpp.o .pio/libdeps/wemos_d1_mini32/espMqttClient/src/Transport/ClientSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSync::connect(IPAddress, uint16_t)': .pio/libdeps/wemos_d1_mini32/espMqttClient/src/Transport/ClientSync.cpp:29:71: error: no matching function for call to 'WiFiClient::setSocketOption(int, int, int*, unsigned int)' client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
Do you have a hint for me?
Greetings, Siggi
Hi,
I've seen that in Logging.h, the symbol DEBUG_ESP_MQTT_CLIENT can enable or disable logging for the ESP8266, but not for the ESP32. Is there any reason that? Otherwise I'd add it and open a PR.
Describe the bug
Today I got the following core dump:
09:49:54.696 > Guru Meditation Error: Core 1 panic'ed (StoreProhibited). Exception was unhandled.
09:49:54.696 >
09:49:54.696 > Core 1 register dump:
09:49:54.696 > PC : 0x400f4578 PS : 0x00060c30 A0 : 0x800f45d0 A1 : 0x3ffb2650
09:49:54.696 > A2 : 0x3ffd0ffc A3 : 0x00000001 A4 : 0x3ffd15fc A5 : 0x3ffd0d9c
09:49:54.696 > A6 : 0x00000000 A7 : 0x00000004 A8 : 0x800f44f2 A9 : 0x3ffb2620
09:49:54.696 > A10 : 0x00000000 A11 : 0x3ffb2687 A12 : 0x00000001 A13 : 0x3ffda2d0
09:49:54.696 > A14 : 0x00000000 A15 : 0x3ffd0d9c SAR : 0x00000015 EXCCAUSE: 0x0000001d
09:49:54.957 > EXCVADDR: 0x0000002c LBEG : 0x400898c8 LEND : 0x400898d2 LCOUNT : 0x00000000
09:49:54.957 >
09:49:54.957 >
09:49:54.957 > Backtrace:0x400f4575:0x3ffb26500x400f45cd:0x3ffb26b0 0x400d813a:0x3ffb26d0 0x400d774d:0x3ffb2710 0x400e8734:0x3ffb2800 0x400fd735:0x3ffb2820
09:49:54.957 >
09:49:54.957 > #0 0x400f4575:0x3ffb2650 in espMqttClientInternals::Outbox<espMqttClientInternals::Packet>::Iterator espMqttClientInternals::Outbox<espMqttClientInternals::Packet>::emplace<espMqttClientTypes::Error&, unsigned short&, char const*&, unsigned char const*&, unsigned int&, unsigned char&, bool&>(espMqttClientTypes::Error&, unsigned short&, char const*&, unsigned char const*&, unsigned int&, unsigned char&, bool&) at .pio/libdeps/generic/espMqttClient/src/Outbox.h:89
09:49:54.957 > (inlined by) bool MqttClient::_addPacket<unsigned short&, char const*&, unsigned char const*&, unsigned int&, unsigned char&, bool&>(unsigned short&, char const*&, unsigned char const*&, unsigned int&, unsigned char&, bool&) at .pio/libdeps/generic/espMqttClient/src/MqttClient.h:149
09:49:54.957 > (inlined by) MqttClient::publish(char const*, unsigned char, bool, unsigned char const*, unsigned int) at .pio/libdeps/generic/espMqttClient/src/MqttClient.cpp:152
09:49:54.957 > #1 0x400f45cd:0x3ffb26b0 in MqttClient::publish(char const*, unsigned char, bool, char const*) at .pio/libdeps/generic/espMqttClient/src/MqttClient.cpp:163
09:49:54.957 > #2 0x400d813a:0x3ffb26d0 in MqttSettingsClass::publish(String, String) at src/MqttSettings.cpp:240
09:49:54.957 > #3 0x400d774d:0x3ffb2710 in MqttPublishingClass::loop() at src/MqttPublishing.cpp:43 (discriminator 9)
09:49:54.957 > #4 0x400e8734:0x3ffb2800 in loop() at src/main.cpp:120
09:49:54.957 > #5 0x400fd735:0x3ffb2820 in loopTask(void*) at C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:50
Its a ESP32 without TLS and Platformio.
The following library versions are in use:
PACKAGES:
- framework-arduinoespressif32 @ 3.20005.220925 (2.0.5)
- tool-esptoolpy @ 1.40201.0 (4.2.1)
- tool-openocd-esp32 @ 2.1100.20220706 (11.0)
- toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 43 compatible libraries
Scanning dependencies...
Dependency Graph
|-- ESP Async WebServer @ 1.2.3+sha.1dde9cf
| |-- AsyncTCP @ 1.1.1
| |-- ArduinoJson @ 6.19.4
| |-- FS @ 2.0.0
| |-- WiFi @ 2.0.0
|-- ArduinoJson @ 6.19.4
|-- espMqttClient @ 1.3.2+sha.f4c2f93
| |-- AsyncTCP @ 1.1.1
| |-- WiFiClientSecure @ 2.0.0
| | |-- WiFi @ 2.0.0
| |-- WiFi @ 2.0.0
|-- RF24 @ 1.4.5
| |-- SPI @ 2.0.0
|-- LittleFS @ 2.0.0
| |-- FS @ 2.0.0
|-- Hoymiles @ 0.0.1
| |-- RF24 @ 1.4.5
| | |-- SPI @ 2.0.0
| |-- Every
| |-- SPI @ 2.0.0
| |-- CircularBuffer
| |-- TimeoutHelper
|-- Ticker @ 2.0.0
|-- Ethernet @ 2.0.0
| |-- WiFi @ 2.0.0
|-- WiFi @ 2.0.0
|-- Update @ 2.0.0
|-- ResetReason
|-- DNSServer @ 2.0.0
| |-- WiFi @ 2.0.0
Currently I cannot reproduce the issue and it doesn't seem to occour often. In fact, it's the first time a saw it. But maybe you have an idee what could be wrong at Outbox.h:89 ?
I wanted to use qos1 messages to make sure messages from my esp32 reach aws cloud because the wifi in my factory is very unreliable. I was testing to see if messages that were supposed to be sent when the wifi is off are sent when the esp32 reconnects to wifi however this is not the case. am i missing something?
Describe the bug
This library seems to have some kind of problem when working together with i2s driver. I've done many tests and the results are always the same.
I'm sending audio files read by the inmp441 MEMS. You can get a few published messages withouth any problem, in fact, it only takes around 300 microseconds to publish a 14KB message(which is amazing) but eventually it will freeze and next message will take around 800-1200 ms and that is not acceptable at all. If I disable I2S then this won't happen. Well actually the sending time its very unestable and some messages might take a few miliseconds, but always within a reasonable amount of time.
Which platform, esp8266 or esp32?
I'm using ESP32
Do you use TLS or not?
I do use TLS.
Do you use an IDE (Arduino, Platformio...)?
PlatformIO
Which version of the Arduino framework?
2.0.9
Please include any debug output and/or decoded stack trace if applicable.
Expected behaviour
Estable sending time. I can expect messages to vary a bit, but not this huge amount of time.
To Reproduce
Steps to reproduce the behaviour:
Set up the mqtt library, objects,functions and get a valid SSL/TLS certificate for your broker. Then include library "/driver/i2s.h" and configure the microphone i2s communications( SR of 16KHz, 16 bit data width) and then send the audio files with a QoS 2. Measure with the function micros() the time the library spent delivering the message to de broker.
Example code
#include "Arduino.h"
#include <driver/i2s.h>
#include "espMqttClient.h"
#include "certificados.hpp"
espMqttClientSecure mqttclient(espMqttClientTypes::UseInternalTask::NO);
const char* broker="192.168.1.xx";
const char* SSID="xxxxx";
const char* PASS="xxxxx";
const uint16_t port=8883;
static TaskHandle_t taskHandle;
bool connectFlag=false;
uint32_t SAMPLE_RATE = 16000;
#define I2SPORT I2S_NUM_0
void i2s_init(){
i2s_config_t i2s_config={
.mode = i2s_mode_t (I2S_MODE_MASTER |I2S_MODE_RX ),
.sample_rate = SAMPLE_RATE,
.bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
.channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT,
.communication_format = I2S_COMM_FORMAT_STAND_I2S,
.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
.dma_buf_count = 4 ,
.dma_buf_len = 256,
.use_apll = false,
.tx_desc_auto_clear = false,
.mclk_multiple = I2S_MCLK_MULTIPLE_256,
.bits_per_chan = I2S_BITS_PER_CHAN_32BIT
};
i2s_pin_config_t i2s_pinout={
.bck_io_num = 32,
.ws_io_num = 25,
.data_out_num = -1,
.data_in_num = 33,
};
i2s_driver_install(I2SPORT, &i2s_config,0,NULL);
i2s_set_pin(I2SPORT,&i2s_pinout);
}
void MqttLoopTask(){
for(;;){
mqttclient.loop();
}
}
void mqttOnConnect( bool sessionPresent){
Serial.print(" Conexión con broker realizada con éxito.\n");
mqttclient.subscribe("cnn",2);
connectFlag=true;
}
void mqttOnDisconnect( espMqttClientTypes :: DisconnectReason reason){
Serial.print("Desconectado del broker.\n");
mqttclient.subscribe("cnn",2);
connectFlag=false;
}
void mqttConnection(){
if(mqttclient.connect()) Serial.println("Dispositivo ESP32 se ha conectado al broker.\n");
else {
Serial.println("Dispositivo ESP32 no logró conectarse al broker, reintentando de forma burda...");
delay(3000);
ESP.restart();
}
}
void mqttinit(){
mqttclient.setCACert(WifiCA_cert);
mqttclient.setServer(broker,port);
mqttclient.onConnect(mqttOnConnect);
mqttclient.onDisconnect(mqttOnDisconnect);
//mqttclient.setTimeout()
mqttConnection();
xTaskCreatePinnedToCore((TaskFunction_t)MqttLoopTask,"MqttloopTask", 8192,nullptr, 3, &taskHandle,0);
Serial.print("Esperando conexión con el brokercito. ");
}
void setup(){
Serial.begin(115200);
i2s_init();
WiFi.begin(SSID,PASS);
Serial.print("Conectándose a red .");
while(WiFi.status() != WL_CONNECTED){
Serial.print(" ."); delay(500);
}
Serial.print(" Conexión con red realizada exitosamente.\n");
mqttinit();
}
uint16_t samples = 7000;
uint32_t buff_size = samples* sizeof(int16_t);
int16_t *buff= (int16_t*)malloc(samples*sizeof(int16_t));
void loop(){
uint32_t ti=0,tf=0, tsi=0, tsf=0, bytes_read=0;
while(!connectFlag){
Serial.print(" .");
delay(400);
}
tsi=micros();
esp_err_t i2s_err = i2s_read(I2SPORT, buff, buff_size, &bytes_read,pdMS_TO_TICKS(100));
if(i2s_err != ESP_OK) {
Serial.print("\nSe ha producido un fallo desconocido durante la comunicación i2s.\n");
delay(5000);
return;
}
tsf=micros();
ti=micros();
mqttclient.publish("prueba",2,false,(uint8_t*)buff,buff_size);
tf=micros();
Serial.printf("t_i2s=%u t_mqtt: %u\n",(tsf-tsi),(tf-ti));
//mqttclient.clearQueue(false); Not solving anything
}
Additional context
Add any other context about the problem here.
The simple-esp32 example fails to compile on Visual Studio/PlatformIO.
Processing ESP32Test (platform: espressif32; board: nodemcu-32s; framework: arduino) ---------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via
-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/nodemcu-32s.html
PLATFORM: Espressif 32 (4.4.0+sha.314a54f) > NodeMCU-32S
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd,
olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
Describe the bug
When sending data every 5 seconds in loop, I receive the error "_checkPing(): Disconnecting, server exceeded keepalive"
If not sending constant data it does not disconnect.
Which platform, esp8266 or esp32?
esp32 sparkfun thingplus
Do you use TLS or not?
No tls
Do you use an IDE (Arduino, Platformio...)?
PlatformIO
Which version of the Arduino framework?
5.1.0
Please include any debug output and/or decoded stack trace if applicable.
14:56:27.318 > [ 26916][I][MqttClient.cpp:289] _checkOutgoing(): tx 120/120
14:56:32.213 > [ 31811][W][MqttClient.cpp:374] _checkPing(): Disconnecting, server exceeded keepalive
14:56:32.218 > [ 31816][I][MqttClient.cpp:601] _clearQueue(): clearing queue (clear session: false)
Expected behaviour
It shouldn't disconnect
To Reproduce
Steps to reproduce the behaviour:
Example code
// Put code here to reproduce the bug, if possible
Additional context
Add any other context about the problem here.
Describe the bug
I was using the lib in version 1.3.3 and ugpraded to 1.4.0.
I am getting the following error message:
.pio/libdeps/generic/espMqttClient/src/MqttClient.cpp: In member function 'void MqttClient::_checkTimeout()':
.pio/libdeps/generic/espMqttClient/src/MqttClient.cpp:465:22: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
if (millis() - *((uint32_t*)&(it.get()->token)) > _timeout) { // NOLINT(readability/casting)
I am buildin my source with
-Wall -Wextra -Werror
Hi was using the example code of esp32.
#include <WiFi.h>
#include <Ticker.h>
#include <espMqttClient.h>
#define WIFI_SSID "yourSSID"
#define WIFI_PASSWORD "yourpass"
#define MQTT_HOST "mqtt.yourhost.com"
#define MQTT_PORT 8883
#define MQTT_USER "username"
#define MQTT_PASS "password"
const char rootCA[] = \
"-----BEGIN CERTIFICATE-----\n" \
" add your certificate here \n" \
"-----END CERTIFICATE-----\n";
espMqttClientSecure mqttClient;
bool reconnectMqtt = false;
uint32_t lastReconnect = 0;
void connectToWiFi() {
Serial.println("Connecting to Wi-Fi...");
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
}
void connectToMqtt() {
Serial.println("Connecting to MQTT...");
if (!mqttClient.connect()) {
reconnectMqtt = true;
lastReconnect = millis();
Serial.println("Connecting failed.");
} else {
reconnectMqtt = false;
}
}
void WiFiEvent(WiFiEvent_t event) {
Serial.printf("[WiFi-event] event: %d\n", event);
switch(event) {
case SYSTEM_EVENT_STA_GOT_IP:
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
connectToMqtt();
break;
case SYSTEM_EVENT_STA_DISCONNECTED:
Serial.println("WiFi lost connection");
break;
default:
break;
}
}
void onMqttConnect(bool sessionPresent) {
Serial.println("Connected to MQTT.");
Serial.print("Session present: ");
Serial.println(sessionPresent);
uint16_t packetIdSub0 = mqttClient.subscribe("foo/bar/0", 0);
Serial.print("Subscribing at QoS 0, packetId: ");
Serial.println(packetIdSub0);
uint16_t packetIdPub0 = mqttClient.publish("foo/bar/0", 0, false, "test");
Serial.println("Publishing at QoS 0, packetId: ");
Serial.println(packetIdPub0);
}
void onMqttDisconnect(espMqttClientTypes::DisconnectReason reason) {
Serial.printf("Disconnected from MQTT: %u.\n", static_cast<uint8_t>(reason));
if (WiFi.isConnected()) {
reconnectMqtt = true;
lastReconnect = millis();
}
}
void onMqttSubscribe(uint16_t packetId, const espMqttClientTypes::SubscribeReturncode* codes, size_t len) {
Serial.println("Subscribe acknowledged.");
Serial.print(" packetId: ");
Serial.println(packetId);
for (size_t i = 0; i < len; ++i) {
Serial.print(" qos: ");
Serial.println(static_cast<uint8_t>(codes[i]));
}
}
void onMqttUnsubscribe(uint16_t packetId) {
Serial.println("Unsubscribe acknowledged.");
Serial.print(" packetId: ");
Serial.println(packetId);
}
void onMqttMessage(const espMqttClientTypes::MessageProperties& properties, const char* topic, const uint8_t* payload, size_t len, size_t index, size_t total) {
Serial.println("Publish received.");
Serial.print(" topic: ");
Serial.println(topic);
Serial.print(" qos: ");
Serial.println(properties.qos);
Serial.print(" dup: ");
Serial.println(properties.dup);
Serial.print(" retain: ");
Serial.println(properties.retain);
Serial.print(" len: ");
Serial.println(len);
Serial.print(" index: ");
Serial.println(index);
Serial.print(" total: ");
Serial.println(total);
}
void onMqttPublish(uint16_t packetId) {
Serial.println("Publish acknowledged.");
Serial.print(" packetId: ");
Serial.println(packetId);
}
void setup() {
Serial.begin(115200);
Serial.println();
Serial.println();
WiFi.setAutoConnect(false);
WiFi.setAutoReconnect(true);
WiFi.onEvent(WiFiEvent);
//mqttClient.setInsecure();
mqttClient.setCACert(rootCA);
mqttClient.setCredentials(MQTT_USER, MQTT_PASS);
mqttClient.onConnect(onMqttConnect);
mqttClient.onDisconnect(onMqttDisconnect);
mqttClient.onSubscribe(onMqttSubscribe);
mqttClient.onUnsubscribe(onMqttUnsubscribe);
mqttClient.onMessage(onMqttMessage);
mqttClient.onPublish(onMqttPublish);
mqttClient.setServer(MQTT_HOST, MQTT_PORT);
mqttClient.setCleanSession(true);
connectToWiFi();
}
void loop() {
static uint32_t currentMillis = millis();
if (reconnectMqtt && currentMillis - lastReconnect > 5000) {
connectToMqtt();
}
static uint32_t lastMillis = 0;
if (currentMillis - lastMillis > 5000) {
lastMillis = currentMillis;
Serial.printf("heap: %u\n", ESP.getFreeHeap());
}
static uint32_t millisDisconnect = 0;
if (currentMillis - millisDisconnect > 60000) {
millisDisconnect = currentMillis;
mqttClient.disconnect();
}
}
Works perfectly when i set my AWS iot policy as
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "iot:Connect",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "iot:Publish",
"Resource": "*"
}
]
}
But when i set my iot policy to include only my client thing name, for example:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "iot:Connect",
"Resource": "arn:aws:iot:us-west-2:123456789:thing/testingesporegon"
},
{
"Effect": "Allow",
"Action": "iot:Publish",
"Resource": "*"
}
]
}
My device just connects and disconnects repeatedly. what could be the problem?
I read up on the documentation and read in the documentation that
iot:Connect
Represents the permission to connect to the AWS IoT Core message broker. The iot:Connect permission is checked every time a CONNECT request is sent to the broker.
could this be causing an issue?
I migrate from https://github.com/marvinroger/async-mqtt-client but espMqttClient could not build with ESP IDF.
You can see the change I make to support ESP IDF here: https://github.com/dzungpv/espMqttClient
To build Arduino project in ESP IDF following the steps:
https://espressif-docs.readthedocs-hosted.com/projects/arduino-esp32/en/latest/esp-idf_component.html
It show error with String.h, I rename it to StringUtil.h but still not success.
I'm trying you example with my preferences, but it seems not connecting because there is something wrong. Does this sound familiar?
esp32
Arduino IDE 1.8.16
Arduino ESP 2.0.4
Connecting to MQTT...
[ 22006][I][Packet.cpp:322] _allocate(): Alloc (l:53)
[ 22007][V][ssl_client.cpp:62] start_ssl_client(): Free internal heap before TLS 90320
[ 22009][V][ssl_client.cpp:68] start_ssl_client(): Starting socket
[ 22060][V][ssl_client.cpp:149] start_ssl_client(): Seeding the random number generator
[ 22073][V][ssl_client.cpp:158] start_ssl_client(): Setting up the SSL/TLS structure...
[ 22073][V][ssl_client.cpp:181] start_ssl_client(): Loading CA cert
[ 22107][E][ssl_client.cpp:37] _handle_error(): [start_ssl_client():190]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected
[ 22110][E][WiFiClientSecure.cpp:135] connect(): start_ssl_client: -8576
[ 22124][V][ssl_client.cpp:324] stop_ssl_socket(): Cleaning SSL connection.
[ 22124][E][WiFiClient.cpp:313] setSocketOption(): fail on -1, errno: 9, "Bad file number"
[ 22140][V][ssl_client.cpp:324] stop_ssl_socket(): Cleaning SSL connection.
[ 22141][I][MqttClient.cpp:627] _clearQueue(): clearing queue (clear session: false)
Disconnected from MQTT: 0.
Describe the bug
Out of order PUBACKs are being ignored.
For QOS 2 publishes that were sent in loop I got packet ids in order 3, 4, 5, 6, 7
,
but from MQTT server I got out of order PUBACKs: 3, 6, 5, 7, 4
Resulting onPublish
was only called for packet id 3 and 4.
Observed on ESP8266 using Non TLS version. Arduino IDE with latest esp8266 board library.
Expected behaviour
onPublish
should be called for every PUBACK regardles of order.
Describe the bug
When executing the tls-esp32 example and changing the ssl auth by the PSK auth I get an error that aparently is not critical since the code seems to work fine
Which platform, esp8266 or esp32? ESP32
Do you use TLS or not? PSK
Do you use an IDE (Arduino, Platformio...)? VSCode+PlatformIO
Which version of the Arduino framework? - framework-arduinoespressif32 @ 3.20004.220825 (2.0.4)
Please include any debug output and/or decoded stack trace if applicable.
Connecting to MQTT...
[ 1839][E][WiFiClient.cpp:335] setOption(): fail on fd -1, errno: 9, "Bad file number"
Connected to MQTT.
Expected behaviour
It works. However the error makes you wonder if something is wrong
To Reproduce
I will put the lines that I modified from the initial example to use PSK. Basically I changed the setup() function
Example code
void setup() {
Serial.begin(115200);
Serial.println();
Serial.println();
WiFi.onEvent(WiFiEvent);
//mqttClient.setInsecure();
mqttClient.setPreSharedKey("testuser", MYSECRETPSK);
//mqttClient.setCredentials(MQTT_USER, MQTT_PASS);
mqttClient.onConnect(onMqttConnect);
mqttClient.onDisconnect(onMqttDisconnect);
mqttClient.onSubscribe(onMqttSubscribe);
mqttClient.onUnsubscribe(onMqttUnsubscribe);
mqttClient.onMessage(onMqttMessage);
mqttClient.onPublish(onMqttPublish);
mqttClient.setServer(MQTT_HOST, MQTT_PORT);
mqttClient.setCleanSession(true);
Hi,
I had the same observation due to an instable WiFi connection to an ESP32-CAM module.
http Server was always reachable after WiFi reconnects, but the MQTT stops after some time.
Then I changed to this 1.4.4. version, but still have this problem.
In the http page I added two links for ESP.restart() and for mqttClient.disconnect().
Both are working. But after some time, when MQTT is dead, the mqttClient.disconnect() is not helping.
I want not to open this issue as it is may be another reason, but MQTT is not stable.
Environment:
Dependency Graph
|-- espMqttClient @ 1.4.4
|-- AsyncElegantOTA @ 2.2.7
|-- AsyncTCP-esphome @ 2.0.0
|-- ESPAsyncWebServer-esphome @ 3.0.0
|-- SPIFFS @ 2.0.0
|-- AsyncTCP @ 1.1.1
|-- Ticker @ 2.0.0
|-- WiFi @ 2.0.0
Code snippets:
void onMqttDisconnect(espMqttClientTypes::DisconnectReason reason) {
#ifdef DEBUG
Serial.printf("Disconnected from MQTT: %u.\n", static_cast<uint8_t>(reason));
#endif
if (WiFi.isConnected()) {
reconnectMqtt = true;
lastReconnect = millis();
}
}
// Reconnect MQTT via Webinterface using disconnect() and auto-reconnect from async MQTT
httpserver.on("/MQTTrestart", HTTP_GET, [](AsyncWebServerRequest *request)
{
String htmlBody = htmlHead_pre_b + htmlHead + "<h1>MQTT restarting ...</h1>";
request->send(200, "text/html", htmlBody);
delay(500);
if (!mqttClient.disconnect()) {
#ifdef DEBUG
Serial.println("Disconnecting failed.");
#endif
} else {
#ifdef DEBUG
Serial.println("Disconnection done.");
#endif
}
});
Debugs:
After ESP32 restart:
Connecting to MQTT...
Connection done.
Connected to MQTT broker: 192.168.xxx.xxx, port: 1883
************************************************
Session present: 0
Last Will -online- setting- QoS 1/retain false, packetId: 1
Subscribing at QoS 1, packetId: 2
Subscribing at QoS 1, packetId: 3
Subscribing at QoS 1, packetId: 4
Subscribing at QoS 1, packetId: 5
Subscribing at QoS 1, packetId: 6
Subscribing at QoS 1, packetId: 7
Subscribing at QoS 1, packetId: 8
Subscribing at QoS 1, packetId: 9
MQTT Publish succesful: 10
Publish acknowledged.
packetId: 1
Subscribe acknowledged.
packetId: 2
qos: 1
Subscribe acknowledged.
packetId: 3
qos: 1
Subscribe acknowledged.
packetId: 4
qos: 1
Subscribe acknowledged.
packetId: 5
qos: 1
Subscribe acknowledged.
packetId: 6
qos: 1
Subscribe acknowledged.
packetId: 7
qos: 1
Subscribe acknowledged.
packetId: 8
qos: 1
Subscribe acknowledged.
packetId: 9
qos: 1
Publish acknowledged.
packetId: 10
After mqttClient.disconnect():
Disconnection done.
Disconnected from MQTT: 0.
Connecting to MQTT...
Connection done.
Connected to MQTT broker: 192.168.xxx.xxx, port: 1883
************************************************
Session present: 0
Last Will -online- setting- QoS 1/retain false, packetId: 11
Subscribing at QoS 1, packetId: 12
Subscribing at QoS 1, packetId: 13
Subscribing at QoS 1, packetId: 14
Subscribing at QoS 1, packetId: 15
Subscribing at QoS 1, packetId: 16
Subscribing at QoS 1, packetId: 17
Subscribing at QoS 1, packetId: 18
Subscribing at QoS 1, packetId: 19
MQTT Publish succesful: 20
Publish acknowledged.
packetId: 11
Subscribe acknowledged.
packetId: 12
qos: 1
Subscribe acknowledged.
packetId: 13
qos: 1
Subscribe acknowledged.
packetId: 14
qos: 1
Subscribe acknowledged.
packetId: 15
qos: 1
Subscribe acknowledged.
packetId: 16
qos: 1
Subscribe acknowledged.
packetId: 17
qos: 1
Subscribe acknowledged.
packetId: 18
qos: 1
Subscribe acknowledged.
packetId: 19
qos: 1
Publish acknowledged.
packetId: 20
Originally posted by @GnomiBerlin in #102 (comment)
Describe the bug
When setting a last will on the mqtt client using setWill
, the client can not connect to AWS IoT core. Maybe this is related to #52. When trying to connect, the client returns a disconnection due to reason 7. My guess is that when using setWill
, the EMC_WAIT_FOR_CONNACK
is ignored and it tries to immediately publish a message which causes AWS to close the connection. If I do not set a last will, connecting and publishing to AWS works fine.
Platform: ESP32
TLS: Yes
IDE: Arduino IDE for testing
Version: 1.4.5
Expected behaviour
Can connect to AWS IoT Core MQTT with last will set.
To Reproduce
Steps to reproduce the behaviour:
Example code
#include <WiFi.h>
#include <Ticker.h>
#include <espMqttClient.h>
#define WIFI_SSID "WIFI"
#define WIFI_PASSWORD "PASS"
const char* awsEndpoint = "URL";
#define MQTT_PORT 8883
const char certificate_pem_crt[] = "foo";
const char private_pem_key[] = "foo";
const char rootCA[] = "foo";
espMqttClientSecure mqttClient;
Ticker reconnectTimer;
void connectToWiFi() {
Serial.println("Connecting to Wi-Fi...");
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
}
void connectToMqtt() {
Serial.println("Connecting to MQTT...");
mqttClient.connect();
}
void WiFiEvent(WiFiEvent_t event) {
Serial.printf("[WiFi-event] event: %d\n", event);
switch(event) {
case SYSTEM_EVENT_STA_GOT_IP:
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
connectToMqtt();
break;
case SYSTEM_EVENT_STA_DISCONNECTED:
Serial.println("WiFi lost connection");
reconnectTimer.once(5, connectToWiFi);
break;
default:
break;
}
}
void onMqttConnect(bool sessionPresent) {
Serial.println("Connected to MQTT.");
Serial.print("Session present: ");
Serial.println(sessionPresent);
uint16_t packetIdSub0 = mqttClient.subscribe("foo/bar/1", 0);
Serial.print("Subscribing at QoS 0, packetId: ");
Serial.println(packetIdSub0);
uint16_t packetIdPub0 = mqttClient.publish("foo/bar/0", 0, false, "test");
Serial.println("Publishing at QoS 0, packetId: ");
Serial.println(packetIdPub0);
}
void onMqttDisconnect(espMqttClientTypes::DisconnectReason reason) {
Serial.printf("Disconnected from MQTT: %u.\n", static_cast<uint8_t>(reason));
if (WiFi.isConnected()) {
reconnectTimer.once(5, connectToMqtt);
}
}
void onMqttSubscribe(uint16_t packetId, const espMqttClientTypes::SubscribeReturncode* codes, size_t len) {
Serial.println("Subscribe acknowledged.");
Serial.print(" packetId: ");
Serial.println(packetId);
for (size_t i = 0; i < len; ++i) {
Serial.print(" qos: ");
Serial.println(static_cast<uint8_t>(codes[i]));
}
}
void onMqttUnsubscribe(uint16_t packetId) {
Serial.println("Unsubscribe acknowledged.");
Serial.print(" packetId: ");
Serial.println(packetId);
}
void onMqttMessage(const espMqttClientTypes::MessageProperties& properties, const char* topic, const uint8_t* payload, size_t len, size_t index, size_t total) {
Serial.println("Publish received.");
Serial.print(" topic: ");
Serial.println(topic);
Serial.print(" qos: ");
Serial.println(properties.qos);
Serial.print(" dup: ");
Serial.println(properties.dup);
Serial.print(" retain: ");
Serial.println(properties.retain);
Serial.print(" len: ");
Serial.println(len);
Serial.print(" index: ");
Serial.println(index);
Serial.print(" total: ");
Serial.println(total);
}
void onMqttPublish(uint16_t packetId) {
Serial.println("Publish acknowledged.");
Serial.print(" packetId: ");
Serial.println(packetId);
}
void setup() {
Serial.begin(115200);
Serial.println();
Serial.println();
WiFi.onEvent(WiFiEvent);
//mqttClient.setInsecure();
mqttClient.setCACert(rootCA);
mqttClient.setServer(awsEndpoint, MQTT_PORT);
mqttClient.setCertificate(certificate_pem_crt);
mqttClient.setPrivateKey(private_pem_key);
mqttClient.setClientId("opendtu");
mqttClient.setWill("foo/bar/will", 2, false, "last");
mqttClient.setCleanSession(true);
mqttClient.onConnect(onMqttConnect);
mqttClient.onDisconnect(onMqttDisconnect);
mqttClient.onSubscribe(onMqttSubscribe);
mqttClient.onUnsubscribe(onMqttUnsubscribe);
mqttClient.onMessage(onMqttMessage);
mqttClient.onPublish(onMqttPublish);
connectToWiFi();
}
void loop() {
delay(1000);
static uint32_t lastMillis = 0;
if (millis() - lastMillis > 5000) {
lastMillis = millis();
Serial.printf("heap: %u\n", ESP.getFreeHeap());
}
mqttClient.publish("foo/bar/0", 0, false, "test");
}
Output:
Disconnected from MQTT: 7.
heap: 211220
Connecting to MQTT...
Disconnected from MQTT: 7.
heap: 211308
Connecting to MQTT...
Disconnected from MQTT: 7.
Does this library support TLS 1.2?
Platform IO:
[env:esp32doit-devkit-v1]
platform = espressif32
board = esp32doit-devkit-v1
framework = arduino
lib_deps = bertmelis/espMqttClient@^1.1.1
Hardware : ESP32 Dev Kit V1
Compling Example: simple-esp32
Please find the version and other details in the verbose messages below
Verbose mode can be enabled via -v, --verbose
option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32doit-devkit-v1.html
PLATFORM: Espressif 32 (3.3.2) > DOIT ESP32 DEVKIT V1
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
Hi Bert,
yesterday I switched my first device (esp8266) from asyncMqttClient to espMqttClient. Looks good! No crashes, no runaway memory, but also no heavy traffic. Just some status reporting and change of settings going back and forth every few seconds.
In the images below you see the heap of the same firmware running running AsyncMqtt (first and espMqtt (second)
Please ignore the peak in the asyncmqtt graph. I made a mistake with my test board and accidentally posted to the same topic as my real hardware. Also the PC went into suspend mode, but the gist remains the same.
Summary:
Don't know if these observations play a role, but the most important info is that the free heap doesn't decrease over time :)
So to me it looks really good after 24 hours. 👍
i was working with the example code of esp32 perfectly a few days ago and now i am getting this error? i was just compiling the example of tlsesp32.
i am using esp32 version 2.0.3 on Arduino.
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFi\src/WiFiClient.h:89:9: note: candidate expects 3 arguments, 4 provided
exit status 1
Compilation error: exit status 1
d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSecureSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSecureSync::connect(IPAddress, uint16_t)':
d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSecureSync.cpp:29:71: error: no matching function for call to 'WiFiClientSecure::setSocketOption(int, int, int*, unsigned int)'
client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
^
In file included from d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSecureSync.h:13,
from d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSecureSync.cpp:11:
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFiClientSecure\src/WiFiClientSecure.h:83:9: note: candidate: 'int WiFiClientSecure::setSocketOption(int, char*, size_t)'
int setSocketOption(int option, char* value, size_t len);
^~~~~~~~~~~~~~~
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFiClientSecure\src/WiFiClientSecure.h:83:9: note: candidate expects 3 arguments, 4 provided
d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSecureSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSecureSync::connect(const char*, uint16_t)':
d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSecureSync.cpp:43:71: error: no matching function for call to 'WiFiClientSecure::setSocketOption(int, int, int*, unsigned int)'
client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
^
In file included from d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSecureSync.h:13,
from d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSecureSync.cpp:11:
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFiClientSecure\src/WiFiClientSecure.h:83:9: note: candidate: 'int WiFiClientSecure::setSocketOption(int, char*, size_t)'
int setSocketOption(int option, char* value, size_t len);
^~~~~~~~~~~~~~~
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFiClientSecure\src/WiFiClientSecure.h:83:9: note: candidate expects 3 arguments, 4 provided
d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSync::connect(IPAddress, uint16_t)':
d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSync.cpp:29:71: error: no matching function for call to 'WiFiClient::setSocketOption(int, int, int*, unsigned int)'
client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
^
In file included from d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSync.h:13,
from d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSync.cpp:11:
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFi\src/WiFiClient.h:89:9: note: candidate: 'int WiFiClient::setSocketOption(int, char*, size_t)'
int setSocketOption(int option, char* value, size_t len);
^~~~~~~~~~~~~~~
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFi\src/WiFiClient.h:89:9: note: candidate expects 3 arguments, 4 provided
d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSync::connect(const char*, uint16_t)':
d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSync.cpp:43:71: error: no matching function for call to 'WiFiClient::setSocketOption(int, int, int*, unsigned int)'
client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
^
In file included from d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSync.h:13,
from d:\Programs\Arduino\Arduino IDE\libraries\espMqttClient\src\Transport\ClientSync.cpp:11:
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFi\src/WiFiClient.h:89:9: note: candidate: 'int WiFiClient::setSocketOption(int, char*, size_t)'
int setSocketOption(int option, char* value, size_t len);
^~~~~~~~~~~~~~~
C:\Users\myname\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3\libraries\WiFi\src/WiFiClient.h:89:9: note: candidate expects 3 arguments, 4 provided
exit status 1
Compilation error: exit status 1
what does this mean?
I created a platformio project and used exactly this code in the main.cpp file: https://github.com/bertmelis/espMqttClient/blob/main/examples/simple-esp32/simple-esp32.ino
Then I created a platformio.ini with the following content:
[env:az-delivery-devkit-v4]
platform = espressif32
board = esp32dev
framework = arduino
lib_deps =
https://github.com/bertmelis/espMqttClient.git#v1.3.2
https://github.com/bblanchon/ArduinoJson
And I get the following error:
Compiling .pio\build\az-delivery-devkit-v4\FrameworkArduino\base64.cpp.o
.pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSync::connect(IPAddress, uint16_t)':
.pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSync.cpp:29:71: error: no matching function for call to 'WiFiClient::setSocketOption(int, int, int*, unsigned int)'
client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
^
In file included from .pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSync.h:13,
from .pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSync.cpp:11:
C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/libraries/WiFi/src/WiFiClient.h:89:9: note: candidate: 'int WiFiClient::setSocketOption(int, char*, size_t)'
int setSocketOption(int option, char* value, size_t len);
^~~~~~~~~~~~~~~
C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/libraries/WiFi/src/WiFiClient.h:89:9: note: candidate expects 3 arguments, 4 provided
.pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSync::connect(const char*, uint16_t)':
.pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSync.cpp:43:71: error: no matching function for call to 'WiFiClient::setSocketOption(int, int, int*, unsigned int)'
client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
^
In file included from .pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSync.h:13,
from .pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSync.cpp:11:
C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/libraries/WiFi/src/WiFiClient.h:89:9: note: candidate: 'int WiFiClient::setSocketOption(int, char*, size_t)'
int setSocketOption(int option, char* value, size_t len);
^~~~~~~~~~~~~~~
C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/libraries/WiFi/src/WiFiClient.h:89:9: note: candidate expects 3 arguments, 4 provided
*** [.pio\build\az-delivery-devkit-v4\lib2c6\espMqttClient\Transport\ClientSync.cpp.o] Error 1
.pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSecureSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSecureSync::connect(IPAddress, uint16_t)':
.pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSecureSync.cpp:29:71: error: no matching function for call to 'WiFiClientSecure::setSocketOption(int, int, int*, unsigned int)'
client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
^
In file included from .pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSecureSync.h:13,
from .pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSecureSync.cpp:11:
C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.h:83:9: note: candidate: 'int WiFiClientSecure::setSocketOption(int, char*, size_t)'
int setSocketOption(int option, char* value, size_t len);
^~~~~~~~~~~~~~~
C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.h:83:9: note: candidate expects 3 arguments, 4 provided
.pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSecureSync.cpp: In member function 'virtual bool espMqttClientInternals::ClientSecureSync::connect(const char*, uint16_t)':
.pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSecureSync.cpp:43:71: error: no matching function for call to 'WiFiClientSecure::setSocketOption(int, int, int*, unsigned int)'
client.setSocketOption(IPPROTO_TCP, TCP_NODELAY, &val, sizeof(int));
^
In file included from .pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSecureSync.h:13,
from .pio/libdeps/az-delivery-devkit-v4/espMqttClient/src/Transport/ClientSecureSync.cpp:11:
C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.h:83:9: note: candidate: 'int WiFiClientSecure::setSocketOption(int, char*, size_t)'
int setSocketOption(int option, char* value, size_t len);
^~~~~~~~~~~~~~~
C:/Users/Thomas/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.h:83:9: note: candidate expects 3 arguments, 4 provided
*** [.pio\build\az-delivery-devkit-v4\lib2c6\espMqttClient\Transport\ClientSecureSync.cpp.o] Error 1
If I am using version 1.3.1 it works without problems:
[env:az-delivery-devkit-v4]
platform = espressif32
board = esp32dev
framework = arduino
lib_deps =
https://github.com/bertmelis/espMqttClient.git#v1.3.1
https://github.com/bblanchon/ArduinoJson
When using this libiary I am seeing a sizable reduction in heapsize.
Before MQTT Connection:
free heap size = 99900 min_free_heap_size = 83408
Connected to MQTT.
free heap size = 59188 min_free_heap_size = 45432
Any suggestions on how to reduce this.
I am also trying to use the HTTPS OTA but this is crashes due to insufficant free heap.
Help...
Thanks
Maybe an idea for you is to define the reason of disconnection like this info.wifi_sta_disconnected.reason do, so description of disconnect is shown. The info is WiFiEventInfo_t, take a look... you will see.
Simplify the examples by removing the reconnect timer.
Hi!
1st to say - this is a nice lib to have, I like the code style.
To issue, comparing to origin Marvinroger's async-mqtt-client this lib allows to bind only single one callback function to each of the event types available, i.e. if one callback is assigned for onConnect()
, the second assigned callback will simply override previous one.
Marvinroger's lib has a vector container that allows assigning multiple callbacks that are executed one by one for the specified event.
std::vector<AsyncMqttClientInternals::OnConnectUserCallback> _onConnectUserCallbacks;
For sure such a container could be created in a user code and has a wrapper that will iterate an execute it, but that incompatibility is a bit disappointing.
I could also have come up with a derived class for this functionality, but MqttClientSetup
is a template and it does not allow any simple inheritance tricks here.
It that an intentional behavior or just not implemented (yet)?
I want to adjust the load on the CPU. I am focusing on...
Publishing at QoS 0, packetId: 1
I think this causes a delay in my soft. I want to begin with shorten the time that it's occupied with it? It's not a big message but maybe I can do something to improve it?
esp32
Arduino IDE 1.8.16
Arduino ESP 2.0.4
using Arduino IDE 1.8.19 with "AI Thinker ESP32-CAM" board.
In file included from D:\SYS_ext\Joerg\Documents\Arduino\libraries\espMqttClient-1.4.2\src/MqttClient.h:22:0,
from D:\SYS_ext\Joerg\Documents\Arduino\libraries\espMqttClient-1.4.2\src/MqttClientSetup.h:14,
from D:\SYS_ext\Joerg\Documents\Arduino\libraries\espMqttClient-1.4.2\src/espMqttClientAsync.h:18,
from D:\SYS_ext\Joerg\Documents\Arduino\simpleAsync-esp32.ino\simpleAsync-esp32.ino.ino:3:
D:\SYS_ext\Joerg\Documents\Arduino\libraries\espMqttClient-1.4.2\src/Packets/Packet.h: In constructor 'espMqttClientInternals::Packet::Packet(espMqttClientTypes::Error&, uint16_t, const char*, uint8_t, const char*, uint8_t, Args&& ...)':
D:\SYS_ext\Joerg\Documents\Arduino\libraries\espMqttClient-1.4.2\src/Packets/Packet.h:100:43: error: expected ',' before ')' token
static_assert(sizeof...(Args) % 2 == 0);
^
D:\SYS_ext\Joerg\Documents\Arduino\libraries\espMqttClient-1.4.2\src/Packets/Packet.h:100:43: error: expected string-literal before ')' token
Thanks
Joerg
I'm not sure if this is really a bug or i understand something wrong.
In proddys emsesp-project some users reports freezing which is (maybe) related to the mqtt at low memory when mqtt is disconnected, With old asyncMqttClient we delete old queued messages when getFreeHeap goes below 60k. In espMqttClient there is the EMC_MIN_FREE_MEMORY setting. Discussion about that is here
I've tested with a modifcated https://github.com/bertmelis/espMqttClient/blob/main/examples/notask-esp32/notask-esp32.ino, and added in loop:
if (currentMillis - lastMillis > 10)
{
lastMillis = currentMillis;
static uint32_t messageNo = 0;
char payload[120];
sprintf(payload, "MessageNo: %u qwertzuiopüasdfghjklöyxcvbnmqwertzuioasdfghj", ++messageNo);
uint16_t id = mqttClient->publish("Testmessage", 1, 0, payload);
Serial.printf("messageNo: %u, publish: %s, heap: %u, maxAlloc: %u\n", messageNo, id > 0 ? "ok" : "fail", ESP.getFreeHeap(), ESP.getMaxAllocHeap());
}
(BTW: currentMillies should not be static to update it in every loop)
config.h
#define EMC_MIN_FREE_MEMORY 61440
#define EMC_ALLOW_NOT_CONNECTED_PUBLISH 1
helpers.h
#define EMC_GET_FREE_MEMORY() ESP.getFreeHeap()
(BTW: the mem-check for PSRAM is critical, the esp32 default in sdkconfig.h is CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=4096,
so mqtt publish is mainly allocated in internal ram. Better check both RAM for enough space)
With the above settings on a esp32/4M, nodemcu (no PSRAM) i get an output:
messageNo: 1384, publish: ok, heap: 54860, maxAlloc: 53236
messageNo: 1385, publish: ok, heap: 54796, maxAlloc: 53236
messageNo: 1386, publish: fail, heap: 54796, maxAlloc: 53236
messageNo: 1387, publish: fail, heap: 54796, maxAlloc: 53236
which looks as expected, except the allocation fail is at 53k,not at 60k.
with debug messages i see that allocation fails at 60k, but publish gives back a valid message-id and free heap is still going down.
messageNo: 1248, publish: ok, heap: 61576, maxAlloc: 53236
[ 15277][I][Packet.cpp:319] _allocate(): Alloc (l:79)
messageNo: 1249, publish: ok, heap: 61416, maxAlloc: 53236
[ 15289][W][Packet.cpp:305] _allocate(): Packet buffer not allocated: low memory
messageNo: 1250, publish: ok, heap: 61352, maxAlloc: 53236
[ 15302][W][Packet.cpp:305] _allocate(): Packet buffer not allocated: low memory
messageNo: 1251, publish: ok, heap: 61288, maxAlloc: 53236
[ 15315][W][Packet.cpp:305] _allocate(): Packet buffer not allocated: low memory
Some messages later strange things happens:
[ 15562][W][Packet.cpp:305] _allocate(): Packet buffer not allocated: low memory
messageNo: 1271, publish: ok, heap: 59944, maxAlloc: 53236
[ 15600][E][WiFiClient.cpp:268] connect(): socket error on fd 48, errno: 104, "Connection reset by peer"
messageNo: 1272, publish: fail, heap: 59904, maxAlloc: 53236
messageNo: 1273, publish: fail, heap: 59904, maxAlloc: 53236
[ 15615][I][MqttClient.cpp:717] _clearQueue(): clearing queue (clear session: 0)
Disconnected from MQTT: 7.
[ 15629][W][Packet.cpp:305] _allocate(): Packet buffer not allocated: low memory
messageNo: 1274, publish: ok, heap: 61312, maxAlloc: 53236
[ 15642][W][Packet.cpp:305] _allocate(): Packet buffer not allocated: low memory
messageNo: 1275, publish: ok, heap: 61248, maxAlloc: 53236
[ 15655][W][Packet.cpp:305] _allocate(): Packet buffer not allocated: low memory
messageNo: 1276, publish: ok, heap: 61184, maxAlloc: 53236```
and at last, when heap is down at 53k the message-Id is returned as zero and no log messages from the client
messageNo: 1375, publish: ok, heap: 54888, maxAlloc: 53236
[ 16955][W][Packet.cpp:305] _allocate(): Packet buffer not allocated: low memory
messageNo: 1376, publish: ok, heap: 54824, maxAlloc: 53236
messageNo: 1377, publish: ok, heap: 54760, maxAlloc: 53236
messageNo: 1378, publish: ok, heap: 54696, maxAlloc: 53236
messageNo: 1379, publish: fail, heap: 54696, maxAlloc: 53236
messageNo: 1380, publish: fail, heap: 54696, maxAlloc: 53236
Testing the same with a own task is similar, but some with some other events:
log_with_own_task.txt
Workaround is not to use EMC_ALLOW_NOT_CONNECTED_PUBLISH, or stop feeding publishes on an own memory limit higher than the setting in the library.
Your solution not to allocate mem for the new messages is not ideal, better is to delete the oldest message in the queue and queue the new message. Is there a way to delete the oldest message in queue by a function call?
I've added a getQueueSize to check the queued messages, but in case of disconnected mqtt memory is allocated but the outbox does not grow. Is there a better way to read the number of messages in the queue?
uint16_t MqttClient::getQueue() const {
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
uint16_t count = 0;
while (it) {
++count;
++it;
}
EMC_SEMAPHORE_GIVE();
return count;
}
Hi Bert,
I noticed that there is a warning:
Resolving wemos_d1_mini32 dependencies... Library Manager: Installing bertmelis/espMqttClient @ ^1.4.0 Unpacking Library Manager: [email protected] has been installed! Library Manager: Resolving dependencies... Library Manager: Installing AsyncTCP @ >=1.1.1 Library Manager: Warning! More than one package has been found by AsyncTCP @ >=1.1.1 requirements: Library Manager: - me-no-dev/[email protected] Library Manager: - kubafilinger/[email protected] Library Manager: - zeed/[email protected] Library Manager: Please specify detailed REQUIREMENTS using package owner and version (shown above) to avoid name conflicts Unpacking Library Manager: [email protected] has been installed!
This may not be a real bug, but I wanted to at least point it out.
I know it has a timer to reconnect but I am wondering if the MQTT is not connected the first time after the WiFi got an IP, will it try to reconnected? Because I not sure if this statement will be called... I could be wrong. Don't know how to test it.
void onMqttDisconnect(espMqttClientTypes::DisconnectReason reason) {
Serial.printf("Disconnected from MQTT: %u.\n", static_cast<uint8_t>(reason));
if (WiFi.isConnected()) {
reconnectTimer.once(5, connectToMqtt);
}
}
case SYSTEM_EVENT_STA_GOT_IP:
Serial.print("Obtained IP address: ");
Serial.println(WiFi.localIP());
connectToMqtt();
break;
Hi Bert,
thank You for this new library. I'm trying to switch from asyncmqtt to this one mainly because of SSL and I found strange behavior - when I use mqttClient.setKeepAlive(120);
the library wont connect to the server. I've used Your simple-esp32.ino
example, added credentials and KeepAlive
Unfortunately the result is
If I remove mqttClient.setKeepAlive(120);
everything is working as expected. Can You please take a look or suggest, what I did wrong?
Thanks.
Eh: I'm using v1.1.0 version of the library.
Describe the bug
With the newer library versions 1.4.0
and 1.4.1
reconnect does not work in all cases.
Which platform, esp8266 or esp32?
Please include any debug output and/or decoded stack trace if applicable.
Expected behaviour
MqTT reconnects after broker comes back online. This works with library version 1.3.3
To Reproduce
I use ioBroker with MqTT adapter as broker.
Example code
affected code
espMqttClient/src/MqttClient.cpp
Lines 223 to 234 in 15041c4
if EMC_WAIT_FOR_CONNACK
isn't defined the reconnect works fine.
Additional context
EMC_WAIT_FOR_CONNACK
needs to be undefined as workaround
This does not work:
#define EMC_WAIT_FOR_CONNACK 0
before #include <espMqttClient.h>
this works as expected
compile flag in platformio.ini
-DEMC_WAIT_FOR_CONNACK=0
Hi,
great library, thanks!
I have a question. Is it possible to publish a payload in chunks like in Print/Stream?
I'm using ArduinoJson to create a large object, serialize it and publish in to mqtt. ArduinoJson supports chunk serialization in Stream, Print and custom Writer.
Right now I'm using this code:
bool publish(const char* topic, JsonDocument& doc) {
if (this->client == nullptr) {
return false;
}
doc[FPSTR(HA_DEVICE)][FPSTR(HA_IDENTIFIERS)][0] = devicePrefix;
doc[FPSTR(HA_DEVICE)][FPSTR(HA_SW_VERSION)] = deviceVersion;
if (deviceManufacturer) {
doc[FPSTR(HA_DEVICE)][FPSTR(HA_MANUFACTURER)] = deviceManufacturer;
}
if (deviceModel) {
doc[FPSTR(HA_DEVICE)][FPSTR(HA_MODEL)] = deviceModel;
}
if (deviceName) {
doc[FPSTR(HA_DEVICE)][FPSTR(HA_NAME)] = deviceName;
}
if (deviceConfigUrl) {
doc[FPSTR(HA_DEVICE)][FPSTR(HA_CONF_URL)] = deviceConfigUrl;
}
Log.straceln("MQTT", topic);
size_t docSize = measureJson(doc);
//char* buffer = (char*) malloc(docSize * sizeof(*buffer));
char buffer[docSize + 1];
size_t length = serializeJson(doc, buffer, docSize + 1);
doc.clear();
doc.shrinkToFit();
bool pubResult = false;
if (length != 0) {
pubResult = this->client->publish(topic, 0, true, buffer);
}
//free(buffer);
if (this->yieldCallback != nullptr) {
this->yieldCallback(yieldArg);
}
return pubResult;
}
But this doesn't work well. For some reason there seems to be a memory leak and after a while the heap space runs out.
I guess this is due to the fact that there are 3 copies in memory: doc, buffer, packet (after call publish). As you can see, I tried to allocate space for the buffer on the heap rather than on the stack. And it works even worse.
After each reconnection, I publish sequentially about 50 messages of about 400 to 1500 characters in length. After each publish, I call yield to change the context and run other tasks.
But after a while the heap runs out. I see in the logs that the heap is reduced at the time of serializeJson/publish. Strange, but this may not happen immediately, but after 100-600 minutes. That is, there is not always a leak.
Therefore, I had the idea to add packets from a custom writer. Without buffer.
Is it possible? Or maybe you have ideas on how this can be done? Thank you!
When sending a payload in a message it react significantly faster when there is not.
Interested to add support for PSRAM? I am glad to test...
esp32 wrover
Arduino IDE 1.8.16
Arduino ESP 2.0.4
Describe the bug
Client no longer sends keep alive packets after reconnect from disconnect on exceeded keepalive.
I got it working after setting _pingSent to false in _onConnack function:
void MqttClient::_onConnack() {
if (_parser.getPacket().variableHeader.fixed.connackVarHeader.returnCode == 0x00) {
+ _pingSent = false;
_state = State::connected;
if (_parser.getPacket().variableHeader.fixed.connackVarHeader.sessionPresent == 0) {
_clearQueue(true);
}
...
I looked in to the library.properties file and I see there is no top header file reference includes=*.h
for importing libs.
Hi,
From what I can see in the code, if I call any publish method, the topic and payload are stored in the outbox as a "Packet". As far as I can see, both are stored as pointers, so the actual content isn't copied.
That would mean I have to make sure the data (topic / payload) are kept in a valid memory location until they are sent to the broker, which would be problematic to do.
For example:
string path = "...";
string payload = "...";
publish(path.c_str(), MQTT_QOS_LEVEL, true, payload.c_str());
This has a high chance to cause problems, because the pointer that c_str() returns isn't valid anymore after publish() is finished. By the time the library loops and sends the messages from the outbox the memory may or may not be overwritten by something else.
Is this correct, or am I misreading something in the code? It would make using the library way harder, because I have to take care the memory is valid ... which is hard because I don't know when the library loops and sends the data.
Describe the bug
Device lost connection during OTA update and after that it stuck in reconnection loop.
OTA update performs in the chunks of qos1 publish packets with length of 65536 bytes.
Logs:
Disconnecting, server exceeded keepalive
:wustmo
:close
clearing queue (clear session: false)
premature end: res:0, pos:65536/286395 <-- debug log from OTA
---> mqtt.connect()
payload: index 1582, total 65536, avail 4/4
Disconnecting, expected CONNACK - protocol error
:close
clearing queue (clear session: false)
---> mqtt.connect()
payload: index 1586, total 65536, avail 4/4
Disconnecting, expected CONNACK - protocol error
:close
clearing queue (clear session: false)
esp32
Arduino IDE 1.8.16
Arduino ESP 2.0.4
No bug, request for a feature.
Can you add some function in the class to remove the MQTT please, like I do for the WiFi network?
void DeIniWifi() {
WiFi.removeEvent(WiFiEvent);
WiFi.disconnect(true);
}
ESP32
When publishing. I am seeing the published message printed in the console. How to disable this?
All links to documentation are broken.
I currently face the problem, that if the broker goes offline the library tries to reconnect for about 1,5h (a second test took about 5 hours until it stopped) and then it stops reconnecting. It seems that the disconnect event is missing. I've implemented it as in your previous examples using a timer in the disconnect event. (Please see code samples below)
If I stop the machine with the broker I get the following output (as expected):
21:40:47.157 > Disconnected from MQTT.
21:40:47.159 > Disconnect reason:TCP_DISCONNECTED
21:40:48.509 > Fetch inverter: 116480145268
21:40:48.530 > Request SystemConfigPara
21:40:49.159 > Connecting to MQTT...
21:40:52.174 > Disconnected from MQTT.
21:40:52.176 > Disconnect reason:TCP_DISCONNECTED
21:40:54.176 > Connecting to MQTT...
21:40:57.186 > Disconnected from MQTT.
21:40:57.189 > Disconnect reason:TCP_DISCONNECTED
21:40:59.188 > Connecting to MQTT...
21:41:02.202 > Disconnected from MQTT.
21:41:02.204 > Disconnect reason:TCP_DISCONNECTED
This repeats again and again... If I start the broker quick enough (lets say after 10minutes) everything works fine again.
For testing purposes I just let the broker offline and got the following behavior:
22:15:49.367 > Connecting to MQTT...
22:15:52.377 > Disconnected from MQTT.
22:15:52.398 > Disconnect reason:TCP_DISCONNECTED
22:15:54.380 > Connecting to MQTT...
22:15:56.580 > Request SystemConfigPara
22:15:57.390 > Disconnected from MQTT.
22:15:57.393 > Disconnect reason:TCP_DISCONNECTED
22:15:59.392 > Connecting to MQTT...
22:16:02.402 > Disconnected from MQTT.
22:16:02.405 > Disconnect reason:TCP_DISCONNECTED
22:16:04.404 > Connecting to MQTT...
Here the output just stops... no more reconnect attempts.
The connection state returns false:
https://github.com/tbnobody/OpenDTU/blob/9b0d2ff25f48fd39155894351d4069fba5900efa/src/MqttSettings.cpp#L150
Which platform, esp8266 or esp32? ESP32
Do you use TLS or not? NO TLS
Do you use an IDE (Arduino, Platformio...)? PlatformIO
Which version of the Arduino framework? [email protected]
Example code
Connect: https://github.com/tbnobody/OpenDTU/blob/9b0d2ff25f48fd39155894351d4069fba5900efa/src/MqttSettings.cpp#L127
Disconnect event: https://github.com/tbnobody/OpenDTU/blob/9b0d2ff25f48fd39155894351d4069fba5900efa/src/MqttSettings.cpp#L52-L81
Other Remarks
Not sure why you removed the example with the timer in the disconnect event. I could also add a workaround to check wether the connection state is disconnected and the last connect attempt is older then e.g. 1 minute. I am just not sure what should be the right behavior.
Hi I am trying to connect my esp32 to my AWS. I have checked that my endpoint and certificates are correct. I have used the pubsubclient library before but i am using this as i want qos1 messages. Here is my code:
#include <WiFi.h>
#include <Ticker.h>
#include <espMqttClient.h>
#define WIFI_SSID "wifiname"
#define WIFI_PASSWORD "wifipassword"
#define MQTT_PORT 8883
#define MQTT_USER "username"
#define MQTT_PASS "password"
const char* awsEndpoint = "awsendpoint-ats.iot.us-west-2.amazonaws.com";
//End initialise AWS var
// xxxxxxxxxx-certificate.pem.crt
const char certificate_pem_crt []= \
"-----BEGIN CERTIFICATE-----\n"\
"sdfwerfiuweewkndwlekmdnewklmoiejdodjlemdklmwe*(Y&(*)LKmewpofmwpeomfweferreL\n"\
"BQAwTTFLMEksdfkmweoifwef*&BIjknjem9uIFdlYiBTZXJ2aWNlcyBPPUFtYXpvbi5jb20g\n"\
"wefiuwefowief208927492dneklndqowiIUhuygtybiohufuybkjoih*yuygjKKBKJBDSFEnlknkjl\n"\
"spfiubgerupifberpijbrepijbnrepinireniorenvrenvernvk;jnerr;kjvnev;kjnre;kjvnrek;vjner;kvnerk;jvn\n"\
"-----END CERTIFICATE-----\n";
// xxxxxxxxxx-private.pem.key
const char private_pem_key[] = \
"-----BEGIN RSA PRIVATE KEY-----\n"\
"ewnpifh3043jk4fn348fn34f3498fhn34fpierfjVESRGTrtgiu45bgg938gbf3o4fwwewe\n"\
"wefewiunfjerngjierbgoierbfij34nfiup34h89fh34iufnewjnwenkjwenfkjeanfkjafen\n"\
"sdfoigneroigfnerionglrkenglkerngklernglknerklgnerlknklernklernklernlkernlken\n"\
"-----END RSA PRIVATE KEY-----\n";
/* root CA can be downloaded in:
https://www.symantec.com/content/en/us/enterprise/verisign/roots/VeriSign-Class%203-Public-Primary-Certification-Authority-G5.pem
*/
const char rootCA[]= \
"-----BEGIN CERTIFICATE-----\n" \
"ewnpifh3043jk4fn348fn34f3498fhn34fpierfjVESRGTrtgiu45bgg938gbf3o4fwwewe\n"\
"wefewiunfjerngjierbgoierbfij34nfiup34h89fh34iufnewjnwenkjwenfkjeanfkjafen\n"\
"sdfoigneroigfnerionglrkenglkerngklernglknerklgnerlknklernklernklernlkernlken\n"\
"-----END CERTIFICATE-----\n";
const char* clientID = "testingesppolicy";
const char* topicchar = "testtopic";
const char* payloadchar = "testpayload";
espMqttClientSecure mqttClient;
Ticker reconnectTimer;
void connectToWiFi() {
Serial.println("Connecting to Wi-Fi...");
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
}
void connectToMqtt() {
Serial.println("Connecting to MQTT...");
// mqttClient.setClientId(clientID);
mqttClient.connect();
}
void WiFiEvent(WiFiEvent_t event) {
Serial.printf("[WiFi-event] event: %d\n", event);
switch(event) {
case SYSTEM_EVENT_STA_GOT_IP:
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
connectToMqtt();
break;
case SYSTEM_EVENT_STA_DISCONNECTED:
Serial.println("WiFi lost connection");
reconnectTimer.once(5, connectToWiFi);
break;
default:
break;
}
}
void onMqttConnect(bool sessionPresent) {
Serial.println("Connected to MQTT.");
Serial.print("Session present: ");
Serial.println(sessionPresent);
uint16_t packetIdPub0 = mqttClient.publish(topicchar, 0, false, payloadchar);
Serial.println("Publishing at QoS 0, packetId: ");
Serial.println(packetIdPub0);
}
void onMqttDisconnect(espMqttClientTypes::DisconnectReason reason) {
Serial.printf("Disconnected from MQTT: %u.\n", static_cast<uint8_t>(reason));
if (WiFi.isConnected()) {
reconnectTimer.once(5, connectToMqtt);
}
}
void onMqttMessage(const espMqttClientTypes::MessageProperties& properties, const char* topic, const uint8_t* payload, size_t len, size_t index, size_t total) {
Serial.println("Publish received.");
Serial.print(" topic: ");
Serial.println(topic);
Serial.print(" qos: ");
Serial.println(properties.qos);
Serial.print(" dup: ");
Serial.println(properties.dup);
Serial.print(" retain: ");
Serial.println(properties.retain);
Serial.print(" len: ");
Serial.println(len);
Serial.print(" index: ");
Serial.println(index);
Serial.print(" total: ");
Serial.println(total);
}
void onMqttPublish(uint16_t packetId) {
Serial.println("Publish acknowledged.");
Serial.print(" packetId: ");
Serial.println(packetId);
}
void setup() {
Serial.begin(115200);
Serial.println();
Serial.println();
WiFi.onEvent(WiFiEvent);
mqttClient.setInsecure();
mqttClient.setCACert(rootCA);
mqttClient.setCertificate(certificate_pem_crt);
mqttClient.setPrivateKey(private_pem_key);
mqttClient.setCredentials(MQTT_USER, MQTT_PASS);
mqttClient.onConnect(onMqttConnect);
mqttClient.onDisconnect(onMqttDisconnect);
mqttClient.onMessage(onMqttMessage);
mqttClient.onPublish(onMqttPublish);
mqttClient.setServer(awsEndpoint, MQTT_PORT);
mqttClient.setCleanSession(true);
connectToWiFi();
}
void loop() {
delay(1000);
static uint32_t lastMillis = 0;
if (millis() - lastMillis > 5000) {
lastMillis = millis();
Serial.printf("heap: %u\n", ESP.getFreeHeap());
}
mqttClient.publish(topicchar, 0, false, payloadchar);
}
I only copied and pasted the example in this library and pasted my own certs but i get this in the Serial monitor:
ELF file SHA256: 7e4ab9d6b66e846e
Rebooting...
ets Jun 8 2016 00:22:57
rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1344
WiFi connected
IP address:
192.168.1.17
Connecting to MQTT...
Disconnected from MQTT: 7.
heap: 233476
Connecting to MQTT...
Disconnected from MQTT: 7.
heap: 233488
Connecting to MQTT...
Disconnected from MQTT: 7.
heap: 233488
Connecting to MQTT...
Disconnected from MQTT: 7.
heap: 233488
Connecting to MQTT...
and keeps on repeating.... i read somewhere that subscribing to the same topic can cause problems so i removed the subscribing part of the code. i also use the endpoint of the oergon region where i have no registered devices.....what is going wrong?
Implement retransmission on acknowledge timeout.
Hi,
In my project I already have a network task, and as such I'd like to change to the async client so I can save those 5000 bytes of stack.
I'm a bit confused though, usually the "mqttclient" task calls the loop() function, which I want to call now from my network task, but it's protected. I've also not really seen another way of calling it, how is this supposed to be achieved?
Also, as far as I can see there's only an espMqttClientAsync, not an espMqttClientSecureAsync, is this as it's to be and there's some trick to it, or it it just missing?
Hello @luebbe and @bertmelis
I am happy because thanks to espMqttClient library I have finally managed to add TLS to MQTT to my ESP32 project and it is working great.
I have started from the example for TLS MQTT fo ESP32 and while I tested it isolated it was working great but when I included TLS in my code I had 3 big issues. I am reporting the issues and how I was able to go around with some strategies.
My ESP32 includes:
To introduce the TLS code I had to make some changes to my MQTT code that I had from async-mqtt-client library
The changes made are below:
#include "espMqttClient.h" //instead than AsyncMqttClient.h
/*add your real certificate here*/
const char rootCA[] PROGMEM = "-----BEGIN CERTIFICATE-----\n" \
"MIIDwzCCAqugAwIBAgIUbTivEoSEFfAFAFWfzwFSFSFESFESFSEFSEQYJKoZIhvcNAQEL\n"\
"BQAwcTELMAkGA1UEBhMCSVQxDzANBgNVBADSF4EFAEFAEFMA0GA1UEBwDVmVy\n"\
"bouESFSFESFSEChNw==\n"\
"-----END CERTIFICATE-----\n";
espMqttClientSecure mqttClient; //insted of AsyncMqttClient mqttClient
// before it was void onMqttDisconnect(AsyncMqttClientDisconnectReason reason) now it changed as below
void onMqttDisconnect(espMqttClientTypes::DisconnectReason reason) ;
//before the function was void onMqttSubscribe(uint16_t packetId, uint8_t qos) now it is
void onMqttSubscribe(uint16_t packetId, const espMqttClientTypes::SubscribeReturncode* codes, size_t len) ;
// before the function was void onMqttMessage(char* topic, char* payload, AsyncMqttClientMessageProperties properties, size_t len, size_t index, size_t total) now it is as below
void onMqttMessage(const espMqttClientTypes::MessageProperties& properties, const char* topic, const uint8_t* payload, size_t len, size_t index, size_t total);
void launchMQTT() {
mqttReconnectTimer = xTimerCreate("mqttTimer", pdMS_TO_TICKS(2000), pdFALSE, (void*)0, reinterpret_cast<TimerCallbackFunction_t>(connectToMqtt)); //I make sure MQTT will reconnect if it loses connection
mqttClient.setCACert(rootCA); //added line
mqttClient.setCredentials(MQTT_USERNAME, MQTT_PASSWORD);
mqttClient.onConnect(onMqttConnect);
mqttClient.onDisconnect(onMqttDisconnect);
mqttClient.onSubscribe(onMqttSubscribe);
mqttClient.onUnsubscribe(onMqttUnsubscribe);
mqttClient.onPublish(onMqttPublish);
mqttClient.onMessage(onMqttMessage);
mqttClient.setServer(MQTT_HOST, MQTT_PORT);
mqttClient.setCleanSession(true); //added line
connectToMqtt();
}
Once I uploaded the code with these variations I had many other things that stopped working:
ISSUE 1: web server gave me back a blank page, I had already experience with it. For details look at this my old issue.
The string for the webpage from now started to go into overflow. I did't have enough heap because TLS MQTT was taking I think 52KB of RAM while connecting
I have sorted like this. sending the webpage in pieces
server.sendHeader("Cache-Control", "no-cache, no-store, must-revalidate");
server.sendHeader("Pragma", "no-cache");
server.sendHeader("Expires", "-1");
server.setContentLength(CONTENT_LENGTH_UNKNOWN);
server.send(200, "text/html", ""); // here begin chunked transfer
server.sendContent("<html> <head> <title> ..................."); //put a piece of your webpage here
server.sendContent("<body> <row> ...................</html>"); //put another piece of your webpage here
server.client().stop();
Serial.printf("Free heap: %u\n", ESP.getFreeHeap());
ISSUE 2: OTA HTTPS call stopped to work and I could not update my ESP32 wirelessly, error below
I have solved disconnecting MQTT just before OTA and putting a delay before launching the update function
disconnectMQTT()
delay(3000);
handleUpdate();
setFlagUpdate(false);
I have the timer to reconnect MQTT so I just disconnected than it will automatically reconnect
ISSUE 3:
The more problematic issues. Post request returned error -1 as in the picture
I have solved disconnecting MQTT before each post request and putting a small delay
setPostInProgress(true);
disconnectMQTT();
delay(100);
This is the full code for the post request.
#include <HTTPClient.h>
#include <WiFiClientSecure.h>
bool sendCounterDataPhp()
{
setPostInProgress(true);
disconnectMQTT();
delay(100);
bool successFlag = false;
WiFiClientSecure *client3 = new WiFiClientSecure;
if (client3) {
client3->setCACert(rootCACert);
{ // Add a scoping block for HTTPClient https to make sure it is destroyed before WiFiClientSecure *client is
HTTPClient https;
Serial.print("[HTTPS] begin...\n");
if (https.begin(*client3, "PUT_YOUR_WEBLINK_TO_YOUR_phpScript.php")) {
/*start POST request*/
https.addHeader("Content-Type", "application/x-www-form-urlencoded");
String sendCounters = "my data";
Serial.println(sendCounters);
int httpResponseCode = https.POST(sendCounters);
if (httpResponseCode > 0) {
String response = https.getString(); //Get the response to the request
Serial.println(httpResponseCode); //Print return code
Serial.println(response); //Print request answer
response.trim(); //remove spaces and carrige return from string
setMyResponse(response);
successFlag = true;
}
else {
Serial.print("Error on sending POST: ");
Serial.println(httpResponseCode);
}
/*end POST request*/
https.end();
}
else {
Serial.printf("[HTTPS] Unable to connect\n");
}
}//for loop
} // End extra scoping block
delete client3;
}
else {
Serial.println("Unable to create client");
}
if (successFlag) {
setPostInProgress(false);
return true;
}
else {
setPostInProgress(false);
return false;
}
}
So my conclusion is that setting the MQTT TLS certificate was going into conflict with my other certificates that I use for HTTPS OTA to retrive my .bin file and for HTTPS POSTs. All the libraries from what I understand use WiFiClientSecure.h
The certificates were not the same and not on the same server. My MQTT mosquitto server has a certificate, my other server - where I have the .bin files - have another certificate.
I believe having two certificates was making the situation messy so I just disconnected MQTT every time I made another call.
I hope this helps as I have worked a lot to sort my code out. Now all works great, MQTT is encrypted and everything I had is working as before.
When my receiving packages starting to grow I got some disconnection to the WiFi with reason assoc_leave. It means that the station mode has stopped. My question is: What wifi mode do I need for my esp32 to only connect to the mqtt and receive and publish packages? I selected station mode, but is it the right on? Could it be when sending and receiving happens on the same time with other clients, that the wifi mode need to APSTA?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.