Duplicated Telegrams in KNXnet/IP Routing about knxd HOT 25 CLOSED

Please also see the excellent report of Patrik:
https://www.dropbox.com/s/2710fybtf7jpnys/knxd_x16_WG_KNX_Routing.pdf?dl=0

• still, I assume more a KNX-topologie problem, but I might be fully wrong
  -> so we should keep this open to investigate further.

from knxd.

From the report I think this is no topologie problem.

I was not able to reproduce this with an FT12 or an IP tunnel Interface. So I think this may be a bug in tpuart code.

from knxd.

It's a problem with the tpuart code. Tpuart chips echo back the packets while sending them. Tpuarts backend handles them as incoming packets.

you can try fix-tpuart-duplicates branch and see if it works now. https://github.com/Makki1/knxd/tree/fix-tpuart-duplicates

from knxd.

Note http://knx-user-forum.de/453406-post149.html

from knxd.

Fixed in master.

from knxd.

I'm also having a similar issue:

Setup is very simple, a single TP line with an ABB IP Interface (only a single IP tunneling connection is supported)

knxd (current master) is installed in a VM along with an instance of gira homeserver. knxd is started with:

/usr/bin/knxd -d -p /var/run/knxd.pid -e 1.1.254 -u /tmp/eib -u /var/run/knx -i -D -T -R -S -b ipt:$ABB_IP_IFACE_ADDR

The gira server can only communicate with the rest of the KNX ecosystem using multicast. ETS connects with multicast fine, but any telegrams transmitted by gira homeserver (residing on the same machine as knxd) are reported in ETS monitor as "repeated" and thus are not retransmitted on the KNX bus.

Any idea what the problem could be?

from knxd.

Here's a packet capture for testing:

A. ETS running on 172.17.178.249 sends to 224.0.23.12. knxd runs on 172.17.178.178/224.0.23.12 and talks to KNX via an ipt connection to 172.17.178.254. The group address write is successful.

0.000000 172.17.178.249 -> 224.0.23.12  UDP Source port: efcp  Destination port: efcp
0.000405 172.17.178.178 -> 172.17.178.254 UDP Source port: lispworks-orb  Destination port: efcp
0.003106 172.17.178.254 -> 172.17.178.178 UDP Source port: efcp  Destination port: lispworks-orb
0.033228 172.17.178.254 -> 172.17.178.178 UDP Source port: efcp  Destination port: lispworks-orb
0.033385 172.17.178.178 -> 172.17.178.254 UDP Source port: lispworks-orb  Destination port: efcp
0.085555 172.17.178.254 -> 172.17.178.178 UDP Source port: efcp  Destination port: lispworks-orb
0.085770 172.17.178.178 -> 172.17.178.254 UDP Source port: lispworks-orb  Destination port: efcp
0.086004 172.17.178.178 -> 224.0.23.12  UDP Source port: efcp  Destination port: efcp

B. Gira homeserver running on 172.17.178.178 sends to 224.0.23.12. knxd runs on 172.17.178.178/224.0.23.12 and talk to KNX via an ipt connection to 172.17.178.254. The group address write is not propagated to KNX. Additionally, the "group monitor" functionality of ETS4 reports the telegram as "repeated".

2.034970 172.17.178.178 -> 224.0.23.12  UDP Source port: efcp  Destination port: efcp

The only difference I can see is that in case A, the original multicast packet is transmitted on a different interface (on a separate host). In case B, the multicast packet is transmitted on the same interface which is subscribed to the multicast address the packet is destined to. Additionally, two users (knxd and gira homeserver) are listening to the multicast address involved.

Here's the configuration of the eth0 interface on the host where both gira homeserver and knxd are running:

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    inet 172.17.178.178/24 brd 172.17.178.255 scope global eth0

2:      eth0
        inet  224.0.0.1
        inet  224.0.23.12 users 2

from knxd.

On 07.01.2016 21:34, gedia wrote:

knxd (current master) is installed in a VM along with an instance of
gira homeserver. knxd is started with:

You have to explain that, i'm not aware of any supported variant of HS
in a VM ;)

In general, this is no problem (a problem is having things like a
Fritzbox..)
Maybe an issue with your VM-environment ?

Makki

from knxd.

You have to explain that, i'm not aware of any supported variant of HS in a VM ;)

Well, it's out there, just google it :-). I'm using it for testing as there is no other way to learn how to configure this thing, documentation is scarce...

In general, this is no problem (a problem is having things like a Fritzbox..)

You mean it shouldn't be a problem that both HS and knxd listen on the same address/port?

netstat -paleWv | grep 3671
udp        0      0 *:3671                  *:*                                 root       28032       22750/knxd      
udp        0      0 *:3671                  *:*                                 root       26068       21787/hs_main   

from knxd.

What router are you using?
There are several known problems with KNX multicast if a AVM router (aka FritzBox) is present.

from knxd.

I'm not using an AVM router. The router at the installation is a Mikrotik Routerboard, but why would the router be involved? We're taking about two processes on the same VM/host...

from knxd.

On 08.01.2016 11:23, gedia wrote:

I'm not using an AVM router. The router at the installation is a
Mikrotik Routerboard, but why would the router be involved? We're
taking about two processes on the same VM/host...

Ah.

You may need the "conn_sep" branch of knxd for that. Multicast packets
from the same host is somewhat tricky, the current master version
doesn't like them.

-- Matthias Urlichs

from knxd.

I don't understand how you can launch two different processes using the same socket. The second should complain about "address already in use". I'm not very experienced with multicast that much but I cannot imagine how two different processes can listen on the same multicast endpoint (multicast-ip:port) on the same host.

from knxd.

Supposedly it can happen with UDP sockets, but the apps must use SO_REUSEPORT. I doubt this is the case here though.

I didn't do anything for this to happen, I just launched the two daemons...

from knxd.

You may need the "conn_sep" branch of knxd for that. Multicast packets from the same host is somewhat tricky, the current master version doesn't like them.

Unfortunately conn_sep won't compile on that machine (debian squeeze i386). It does compile fine however on another box with gcc 5.3 after fixing a typo (in emi2.cpp, ifcae instead of iface if I remember correctly).

from knxd.

Do I need multicast routing for this to work with conn_sep?

from knxd.

On 08.01.2016 14:07, gedia wrote:

Do I need multicast routing for this to work with conn_sep?

The difference between conn_sep and master in this regard (there are
others) is that conn_sep turns the "deliver to localhost" flag on.
Master turns this off explicitly because historically eibd had no way to
ignore its own packets. conn_sep uses a separate source port for them.

Thus no special multicast address setup is necessary for the conn_sep
branch.

-- Matthias Urlichs

from knxd.

On 08.01.2016 13:56, Othmar Truniger wrote:

I don't understand how you can launch two different processes using
the same socket.

Multicast is different: two programs need to be able to bind to the same
multicast address+socket if they both want to receive a data stream.

-- Matthias Urlichs

from knxd.

@smurfix Thank you, I didn't know that. Maybe it's needed to bind the source to different addresses for the two programs? How could one program otherwise differ packets sent by the other one? Or is this also solved so a program always knows what it has sent before and therefore knows that it must accept a packet as new (sent by the other program) even it looks like it is sent from itself?

from knxd.

I can confirm it works charmingly with the conn_sep branch. Thanks!

from knxd.

On 08.01.2016 20:53, gedia wrote:

I can confirm it works charmingly with the conn_sep branch. Thanks!

Great! I tested it with two knxd processes on the same machine and I'm
happy to have that method validated. ;-)

It's probably time to make that the new master and relegate the current
master to some sort of "stable" branch.

-- Matthias Urlichs

from knxd.

@SgtSeppel commented on 8 Jan 2016

What router are you using?
There are several known problems with KNX multicast if a AVM router (aka FritzBox) is present.

@SgtSeppel, can you elaborate a bit on this? I also do have issues with FritzBoxes.

I have issues using knxd and KNX/IP with a FritzBox 7490 (and several FRITZ!WLAN Repeater 1750E). I wanted to integrate Tasmota-based ESP8266 devices (Arduino) into my KNX network. Tasmota implements KNX/IP using multicast and my plan was to use knxd to route packets from KNX/TP to KNX/IP.

However, it seems that the FritzBox does not correctly forward multicast packets (LAN and WiFi). I did some research and found out that Apple's Bonjour/mDNS multicast packets are forwarded correctly to all LAN ports and WiFi stations but KNX/IP multicast packets are not forwarded. I observed the following behaviour:

• Forwarded: Apple's multicast IP packets with destination IP224.0.0.251 and destination MAC address: 01:00:5e:00:00:fb.
• Not forwarded: multicast IP packets with destination IP 224.0.23.12and destination MAC address 01:00:5e:00:17:0c.

However, according to Wikipedia - Multicast address - Ethernet MAC addresses in range 01:00:5E:00:00:00- 01:00:5E:7F:FF:FF should be forwarded to all LAN and WiFi stations.

I contacted AVM's support about this issue and got the following reply (in German):

Nach interner Rücksprache und intensiverer Recherche muss ich Ihnen leider mitteilen, dass die FRITZ!Box derzeit nicht KNX/IP-Router fähig sind. Das von Ihnen geschilderte Verhalten ist somit leider erwartungskonform, ich habe eine Unterstützung der Funktion jedoch als Verbesserungsvorschlag an unsere Entwicklung weitergeleitet.

Basically, the say that the FritzBox do not adhere to the standard and thus KNX/IP does not work currently.

A related issue is discussed here: No response to KNX telegramms after about 15 minutes. Status telegramms to KNX are still working.

Does anybody have an idea how to get KNX/IP to work with a FritzBox?

Best,
Dennis

from knxd.

Does anybody have an idea how to get KNX/IP to work with a FritzBox?

Yes. [Get someboy to] teach Tasmota to do tunnel-based KNX instead. (Or alternately.)

This is likely to be more successful / will work for more people than trying to fix every broken router out there, of which there are quite a few unfortunately.

Moreover, the tunnel mode has retries. Multicast on the other hand is fire-and-forget. For that reason alone I wouldn't use it on WLANs.

from knxd.

@smurfix Thanks for your thoughts on this issue. I also thought about using tunnel mode but then using MQTT + OpenHab + Tasmota seems to be an easier and more scalable alternative.

I like the multicast approach because it behaves just like KNX/TP and is a decentralized approach w/o any requirements on other components (apart from a transparent IP <--> gateway/router).

I'll try to reconfigure Tasmota's KNX multicast IP and knxd's multicast address (using the multicast-address parameter) to something in the range of Apple's multicast IP (224.0.0.251) and see if this works.

This would at least be a workaround for FritzBox users as they don't seem to be willing to adhere to the standard.

Update: This seems to be working fine. Just for my own reference: tcpdump -i br0 -s0 -vv host 224.0.0.251

from knxd.

Ok, for everybody using a AVM FritzBox or routers with similar multicast issues: it's working for me now. Tasmota's KNX/IP support <--> knxd <--> KNX/TP devices. Basically, I changed Tasmotas broadcast IP address to 224.0.0.251 and matched this setup in knxd's config.

To get this working, perform the following steps:

1. In Tasmota's src edit lib/esp-knx-ip-0.5.2/esp-knx-ip.h:

#define MULTICAST_IP IPAddress(224, 0, 0, 251)  // [Default IPAddress(224, 0, 23, 12)]

2. In knxd.ini, add a section

[tasmota]
driver = ip
multicast-address = 224.0.0.251

and enable it in the main section by adding tasmota to the connections parameter. For example like this:

connections = server,B.unix,C.tpuarts,tasmota

from knxd.