FHEM support request about esp32_bletracker HOT 16 CLOSED

Hi Xavier0Smith, I have never used FHEM so I need to investigate the feasibility using the Collector daemon. Anyway taking a quick look to the FHEM documentation I saw it support the MQTT protocol https://wiki.fhem.de/wiki/MQTT, the same protocol supported by the tracker.
Have you already tried to give a look to that?
Regards.

from esp32_bletracker.

Hi, first of all thanks for the quick reply.

I am familiar with MQTT support for FHEM: I have been using OpenMQTTGateway for a long time before leaving it because it has no battery level support.
But the biggest problem is the MQTT management within FHEM.
It is fine as long as you are using single devices (light bulbs, smart sockets, relays), but it is terribly complicated in managing multi-room devices such as the ESP32 used as trackers.
This wiki explains (partially) how to use OpenMQTTGateway in FHEM, the rest you have to find out in the forum, especially here (MQTT sample) and here (filter rooms code).
I have tried all. Lots of entities are generated and the FHEM main configuration file code becomes extremely complex and difficult to maintain.
Instead, using Collectord you only need one line of code in FHEM for each BLE tag to be checked and four lines for each ESP32 in use in a external text configuration file.
To get an idea, you can read this section of the FHEM PRESENCE module wiki or, for a summary, this simpler guide (just look at the figure in the paragraph "Konfiguration Collectord" and read in the paragraph "Konfiguration FHEM" starting from "Mit folgendem Befehl ...").

In conclusion, I'm not asking for a complete and working code. I'm satisfied with just an example of a way to send strings via TCP/IP on a specific port, then maybe for the precise syntax I can try to hear the author of Collectord (if he answers me ...).

Best regards,
Xavier0Smith

from esp32_bletracker.

Hi, I have just pushed a new branch "https://github.com/shogunxam/ESP32_BLETracker/tree/fhem_support" with the changes to support FHEM. Could you please give a try?It's just a beta version, as said I have never used FHEM so I don't know If I have correctly configured it in my VM to make some tests.
If you use platform.io to compile the firmware you can just select the variant env:esp32dev-ble-fhem-release ( or env:esp32dev-ble-fhem-debug to have some print in the console) If instead you are using Arduino IDE you need to install the Regexp library (https://www.arduino.cc/reference/en/libraries/regexp/). The Port used by the ESP32 is 5333 and it support at least 16 clients
Let me know if you find some issue.
Best Regards

from esp32_bletracker.

You are The Best!!

Here is a preliminary screen of FHEM detecting of your BLETracker V2.1R, the battery level and a fake room named Bedroom.

Everything froze after the first detection, but it doesn't matter.

For today it's already a great result.

Tomorrow night I review the FHEM logs and update you.

Best regards,
Xavier0Smith

from esp32_bletracker.

Hi Xavier0Smith, I need some help to configure properly FHEM in order to debug issues. Until now I have installed fhem, collectord and lepresenced (I don't know if I really need it) than I wrote in /etc/collectord.conf these lines

[living room]
address=192.168.1.127
port=5333
presence_timeout=10
absence_timeout=120

where 192.168.1.127 is the ip of the ESP32

than in using the FHEM frontend I added this line to fhem.cfg
define MeinGtAG PRESENCE lan-bluetooth DE:VI:CE:MA:CA:DR 127.0.0.1:5222 10 120
where DE:VI:CE:MA:CA:DR is obviously the Mac Address of the device to track
I notice that the value 10 and 120 in the fhem.cfg are ignored and are always used the values stored in collectord.conf
Is this enough or have I to do something different?

PS.
I forgot to tell you in order to build with Arduino IDE you have to change two #define in the config.h, you have to set
USE_MQTT false
USE_FHEM_LEPRESENCE_SERVER true

from esp32_bletracker.

Hi shogunxam,

your setup is basically correct.

If you want to make your life easier, you can take inspiration from my configuration in fhem.cfg where with "devStateIcon" you define an icon that displays whether the tag is considered "present" or "absent", and with "group" and "room" you create a virtual room and a group of icons.

define gTag.Green PRESENCE lan-bluetooth DE:VI:CE:MA:CA:DR 127.0.0.1:5222
attr gTag.Green devStateIcon present:FS20.on absent:FS20.off .*:set_off
attr gTag.Green group PRESENCE_DEVICES
attr gTag.Green room 00 Presence

define gTag.Orange PRESENCE lan-bluetooth DE:VI:CE:MA:CA:DR 127.0.0.1:5222
attr gTag.Orange devStateIcon present:FS20.on absent:FS20.off .*:set_off
attr gTag.Orange group PRESENCE_DEVICES
attr gTag.Orange room 00 Presence

define gTag.Red PRESENCE lan-bluetooth DE:VI:CE:MA:CA:DR 127.0.0.1:5222
attr gTag.Red devStateIcon present:FS20.on absent:FS20.off .*:set_off
attr gTag.Red group PRESENCE_DEVICES
attr gTag.Red room 00 Presence

I would add that the collectord configuration is ok and that I "believe" that lepresenced may not be useful.

I have a slightly more complicated configuration: three rPi and three ESP32. The ESP32s work with three different firmware including your firmware. Yesterday everything had stalled on the first try. Now I try to disable some things and then I tell you.

P.S. I use PlatformIO and had noticed your config.h. I like it when I find mysterious configuration files. I always put everything "on" to see what happens ... :)

I'll be back to write here asap.

from esp32_bletracker.

Ok, this is the result in my fhem.log:

2021.01.27 23:17:02 3: PRESENCE (gTag.Green) - collectord lost connection to room house
2021.01.27 23:17:02 3: PRESENCE (gTag.Orange) - collectord lost connection to room house
2021.01.27 23:17:02 3: PRESENCE (gTag.Red) - collectord lost connection to room house

and in PlatformIO Serial Monitor window I see:

or:

Sometimes I see the tag reading data, then it starts over with these messages, then it resumes reading the tags, and so on.

Instead in FHEM, I found only one reading like yesterday and then stop.

If I turn off the ESP32 with your firmware and turn on another ESP32 with a different firmware, I still don't read the gTags, a sign that collectord no longer works. If I restart FHEM, I have readings again.

I remain available if you want other checks.

Best regards,
Xavier0Smith

from esp32_bletracker.

Hi I have pushed some changes (always the same branch https://github.com/shogunxam/ESP32_BLETracker/tree/fhem_support). To avoid the stack overflow I have raised the stack size used by the clients' thread, but I think now the number of clients can no longer be 16.
There is also another problem, if an absence interval greater than one minute is set, the connection is closed due to a timeout that triggers, I think, due to lack of traffic and the client has to reconnect, I tried to change the WiFiClient timeout but it doesn't seem to work. For this reason, in the logs you will see disconnection and reconnection messages every minute.

from esp32_bletracker.

I forgot to mention that to debug a crash I need you build the debug variant and copy and paste the back-trace you can find in the console. I will need also the firmware.elf file to correctly translate the addresses of the back-trace into readable lines code.
This is a full example of a back-trace (not always the register section is present, the important thing is the back-trace)

Guru Meditation Error: Core  0 panic'ed (Unhandled debug exception)
Debug exception reason: Stack canary watchpoint triggered (Tmr Svc)
Core 0 register dump:
PC      : 0x40081708  PS      : 0x00060336  A0      : 0x3ffbc990  A1      : 0x3ffbc8d0
A2      : 0x3ffb7dd0  A3      : 0x00000000  A4      : 0x3ffb7e18  A5      : 0x00000000
A6      : 0x00000001  A7      : 0x00000018  A8      : 0x800894ac  A9      : 0x3ffbc970
A10     : 0x00000000  A11     : 0x3ffc12c8  A12     : 0x3ffc12c8  A13     : 0x00000001
A14     : 0x00060320  A15     : 0x00000000  SAR     : 0x00000002  EXCCAUSE: 0x00000001
EXCVADDR: 0x00000000  LBEG    : 0x40001609  LEND    : 0x4000160d  LCOUNT  : 0x00000000

Backtrace: 0x40081708:0x3ffbc8d0 0x3ffbc98d:0x3ffbc9d0 0x400d85b6:0x3ffbc9f0 0x40082926:0x3ffbca10 0
x400822fe:0x3ffbca30 0x400dd591:0x3ffbcaa0 0x400dc019:0x3ffbcac0 0x400dc467:0x3ffbcae0 0x400dc5f5:0x
3ffbcb50 0x400db105:0x3ffbcbd0 0x400db1b4:0x3ffbcc50 0x400da27e:0x3ffbccb0 0x400da7b1:0x3ffbccf0 0x4
01396b2:0x3ffbcd10 0x401398a6:0x3ffbcd60 0x401398f9:0x3ffbcd90 0x401017ec:0x3ffbcdb0 0x401019fa:0x3f
fbcde0 0x400d9721:0x3ffbce10 0x400

Best Regards

from esp32_bletracker.

Hi, I was again able to compile your latest esp32dev-ble-fhem-release.
Now I read correctly the data of the various gTags in the PlatformIO terminal window, without any crash.
But... the FHEM server is not receiving any data.
There appears to be no communication between ESP32 and Collectord.
Where am I wrong?

Best regards,
Xavier0Smith

P.S. My limited knowledge of PlatformIO made me waste a lot of time updating to your latest version, but failed. So I deleted the folder and downloaded it all over again. Can you explain me an easy way to upgrade without going crazy?

from esp32_bletracker.

Hi when the ESP32 connects itself to the Wi-Fi Network the IP is assigned by the DHCP (it's not configured to use a static IP) are you sure your router is assigning, each time, the same IP you wrote in the collectord.conf ? In case try to force the router to assign a static IP to the ESP32
About Platform.io you have to push the recycler bin icon in order to clean the last build; some time you need to build twice because Platform.io don't download all the dependencies the first time. In the worst case you have to delete manually the .pio folder and rebuild

from esp32_bletracker.

SOLVED!!

It was a rights issue on the /etc/collectord.conf file. Changing the access rights from the default rw-r--r-- to rwxrwxr-- with

sudo chmod 777 collectord.conf

everything started working again.

As soon as possible I also update the other ESP32s and see how it works.

Best regards,
Xavier0Smith

from esp32_bletracker.

Glad to know it's working, I'll integrate it in the next release. Let me know If you find other issues.
Best regards,
Shogunxam

from esp32_bletracker.

Hi Xavier0Smith, I made some other changes to improve the code, if you can give a try and don't find any issue I'll merge the Pull request into the develop branch, so that I'll release the feature.
Best regards.

from esp32_bletracker.

Hi Shogunxam,

I'm done updating my installation just now. I have removed all the lepresenced sessions in all the rPis and updated the firmware of all the ESP32s.

Initially I had customized the value of GATEWAY_NAME by changing it to (for example) "Garage BLETracker", to have it as a custom title in the webinterface. I realized that the same value also appears in the FHEM "daemon" reading but it is not always updated in real time with the other readings. Practically a Ble tag is detected in the house but as "daemon" it is still indicated "Garage BLETracker" and only after a while it changes to "House BleTracker". So I finally put back the standard "BLETracker" value for all ESP32s.

I also had to update the rights (as well as /etc/collectord.conf) of /etc/default/collectord. I don't know the reason by otherwise I had no readings.

Finally, in the FHEM log I initially had a lot of connections and disconnections. ESP32s have a much lower ble range than rPIs, and your software is damn fast. :-) So, I physically repositioned my three ESP32s to three nearby locations. I believe I will buy several more in the future for optimal coverage.

Nice idea of ​​hiding MQTT parameters in the webinterface if you use FHEM mode, and nice OTA update too. I had an OTA of an ESP32 locked at 27%, but I have not lost anything. I just re-launched it, when finished it restarted by itself and started perfectly up-to-date.

The only thing I'd like to add is the ability to set a static IP in the firmware. I don't want a captive portal, it takes up memory and if the power fails you get a lot of devices trying to connect. An optional line in user_config.h would suffice if possible. But that's already fine too.

Best regards,
Xavier0Smith

from esp32_bletracker.

Hi Xavier0Smith,
I will try to add the static IP in another PR. I have create a new issue for this. #17
Best Regards.

from esp32_bletracker.