umlaeute / pd-iemnet Goto Github PK

low-level networking for Pd - fork of https://git.iem.at/pd/pd-iemnet

Home Page: https://git.iem.at/pd/pd-iemnet

License: GNU General Public License v2.0

C 68.95% Shell 2.55% Makefile 23.03% Python 0.43% M4 4.79% C++ 0.24%

pd-iemnet's Introduction

iemnet - networking for Pd

this is a fork of martin peach's "net" library, that allows low-level interaction with networks on OSI-layer 5 (transport layer).

for a list of features, see FEATURES.

Why fork?

the original library is still actively maintained by martin peach. however: - forking allows me to experiment with new features/techniques more easily - forking allows to remove all the legacy cruft (and not care about compatibility now) - the development mode in the original library would involve the upstream author "signing-off" any changes (debatable; but i don't want to submit experimental code to their stable code base)

in practice one of the major drawbacks i see in upstream is, that (in the multithreaded objects), for each message a separate thread is spawned. this leads to excessive use of system ressources (detaching and joining threads takes time), easy DoS (each thread uses one in a limited number of thread handles), and abandons determinism (nobody guarantees that parallel threads are executed "in order"; thus a message in a later-spawned thread might be delivered to the socket earlier than older messages - effectively circumventing one of the promises of TCP/IP: that all packets will reappear in order; users have already reported this behaviour, which makes using those objects a bit unreliable)

on the long run compatibility with the upstream library is intended. (though probably not for all the cruft that is in there)

Design

easy to maintain: re-used code is bundled in a small "library" (currently only a single file iemnet.c), which is linked statically against the externals. this library handles all the send/receive stuff (whether it uses threads or not and if so how, is an implementation detail) the lib doesn't know anything about the actual transport protocol. it only interacts with a socket.

easy to run: think speed, think reliability all known implementations for pd are either slow or will freeze Pd when under heavy load. most do both. iemnet wants to provide objects whih allow you to saturate the network connection and still keep Pd reactive. (sidenote: saturating even a 100MBit network with Pd might lead to audio dropouts; this is not necessarily related to the network but rather to the amount of data processed by Pd...)

easy to use: probably not; but at least it has the same (basic) API as mrpeach/net so a switch should be easy. "basic" means "not everything", so messages for special workarounds in mrpeach/net (e.g. the block/unblock stuff) are not supported, as well as debugging features ("dump") and features not related to networking (e.g. the ability to read a file from harddisk)

Authors

currently iemnet is developed by IOhannes m zmölnig

it (being a fork) is heavily based on code written by Martin Peach, who again has used code by Olaf Matthes and Miller Puckette.

LICENSE

iemnet is published under the GPL. see LICENSE for more information.

pd-iemnet's People

Contributors

Stargazers

Watchers

Forkers

avilleret cjfitz markdenardo

pd-iemnet's Issues

tcpclient crashes

when using [tcpclient] I got this :

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff73afe84 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) bt
#0  0x00007ffff73afe84 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007fffddfcdc3c in queue_use_increment (_this=0x1) at iemnet_data.c:242
#2  queue_pop_noblock (_this=0x1) at iemnet_data.c:355
#3  0x00007fffddfce2bb in iemnet__receiver_tick (x=0x7fffd40110a0) at iemnet_receiver.c:144
#4  0x00007fffddfcec7d in pollfun (z=<optimized out>, fd=<optimized out>) at iemnet_notify.c:126
#5  0x00000000004792ea in sys_domicrosleep.constprop.3 ()
#6  0x000000000047485d in m_mainloop ()
#7  0x00007ffff6e0476d in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#8  0x00000000004114d1 in _start ()
(gdb)

$ uname -vs
Linux #81-Ubuntu SMP Thu Aug 22 21:01:03 UTC 2013
$ pd -version
Pd-0.44.0 ("test1") compiled 11:16:08 Jan 15 2013

I built iemnet today from this repo and after reporting another issue on sourceforge [1].

[1] : http://sourceforge.net/p/pure-data/bugs/1090/

rethink shutdown flow

currently it's not clear where the sockets are shut-down on a disconnect (either in the iemnet_sender/receiver modules or in the calling object).

this needs formal clarification.

Affects #5

[udpclient] doesn't output connection information nor received data when using broadcast address

When I'm connecting to an UDP server with it's IP address, it answer with "acknowledge"
when I'm connection with a broadcast IP (i.e. 192.168.0.255) I didn't not receive the answer, but a python script does.

My UDP server run on a Atmega 328P with an Ethernet shield, here the code :

/*
 UDPSendReceiveString:
 This sketch receives UDP message strings, prints them to the serial port
 and sends an "acknowledge" string back to the sender

 A Processing sketch is included at the end of file that can be used to send
 and received messages for testing with a computer.

 created 21 Aug 2010
 by Michael Margolis

 This code is in the public domain.
 */


#include <EthernetUdp.h>

// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = {
  0x98, 0x76, 0xB6, 0x11, 0x18, 0x30
};

unsigned int localPort = 8888;      // local port to listen on

// buffers for receiving and sending data
char packetBuffer[UDP_TX_PACKET_MAX_SIZE];  // buffer to hold incoming packet,
char ReplyBuffer[] = "acknowledged";        // a string to send back

// An EthernetUDP instance to let us send and receive packets over UDP
EthernetUDP Udp;

void setup() {
  
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  // start the Ethernet connection:
  Serial.println("Initialize Ethernet with DHCP:");
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    if (Ethernet.hardwareStatus() == EthernetNoHardware) {
      Serial.println("Ethernet shield was not found.  Sorry, can't run without hardware. :(");
    } else if (Ethernet.linkStatus() == LinkOFF) {
      Serial.println("Ethernet cable is not connected.");
    }
    // no point in carrying on, so do nothing forevermore:
    while (true) {
      delay(1);
    }
  }
  // print your local IP address:
  Serial.print("My IP address: ");
  Serial.println(Ethernet.localIP());
  
  // start UDP
  Udp.begin(localPort);
}

unsigned long count = 0;

void loop() {
  // if there's data available, read a packet
  int packetSize = Udp.parsePacket();
  if (packetSize) {
    Serial.print("Received packet of size ");
    Serial.println(packetSize);
    Serial.print("From ");
    IPAddress remote = Udp.remoteIP();
    for (int i=0; i < 4; i++) {
      Serial.print(remote[i], DEC);
      if (i < 3) {
        Serial.print(".");
      }
    }
    Serial.print(", port ");
    Serial.println(Udp.remotePort());

    // read the packet into packetBufffer
    Udp.read(packetBuffer, UDP_TX_PACKET_MAX_SIZE);
    Serial.println("Contents:");
    Serial.println(packetBuffer);

    // send a reply to the IP address and port that sent us the packet we received
    Udp.beginPacket(Udp.remoteIP(), Udp.remotePort());
    Udp.write(ReplyBuffer);
    Udp.endPacket();
  }
  delay(10);
}

.
You will find the client.py and client.pd script that demonstrate the issue.
But you'll need an arduino and an ethernet shield to reproduce it.
client-issue.tar.gz
Unfortunately I could not reproduce exactly the same behavior with a Python script as UDP server

tcpclient crashes on re-creation

hi found a behavior that leads to crash and two patches to checkit.
The patches are here : https://github.com/avilleret/pd-iemnet/tree/buggy_patches/tests/buggy

The two patches depends on osc, slipenc and slipdec objects by mrpeach.

And here is what to do to crash :

open buggy_server.pd and buggy_client.pd in pd (either both in the same instance or not)
click on the top most bang in buggy_client.pd
edit the [tcpclient] object box to re-create it.
click again on the top most bang. Pd should crash here.

Here is a full backtrace of pd with only the buggy_client.pd opened.

Program received signal SIGSEGV, Segmentation fault.
0x00000000004693c9 in outlet_list ()

Thread 7 (Thread 0x7ffff0e47700 (LWP 3216)):
#0  0x00007ffff6ece763 in select () at ../sysdeps/unix/syscall-template.S:82
#1  0x00007ffff305319c in iemnet__receiver_readthread (arg=0x7fffec011640) at iemnet_receiver.c:105
#2  0x00007ffff73ace9a in start_thread (arg=0x7ffff0e47700) at pthread_create.c:308
#3  0x00007ffff6ed53fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#4  0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7ffff1648700 (LWP 3214)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00007ffff3052be4 in queue_pop_block (_this=0x7fffec011430) at iemnet_data.c:313
#2  0x00007ffff3053638 in iemnet__sender_dosend (q=0x7fffec011430, sockfd=<optimized out>) at iemnet_sender.c:79
#3  iemnet__sender_sendthread (arg=0x7fffec0113e0) at iemnet_sender.c:134
#4  0x00007ffff73ace9a in start_thread (arg=0x7ffff1648700) at pthread_create.c:308
#5  0x00007ffff6ed53fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#6  0x0000000000000000 in ?? ()

Thread 5 (Thread 0x7ffff304f700 (LWP 3212)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00007ffff3257833 in tcpclient_connectthread (w=0x75b2a0) at tcpclient.c:157
#2  0x00007ffff73ace9a in start_thread (arg=0x7ffff304f700) at pthread_create.c:308
#3  0x00007ffff6ed53fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#4  0x0000000000000000 in ?? ()

Thread 4 (Thread 0x7ffff1e49700 (LWP 3211)):
#0  0x00007ffff6ece763 in select () at ../sysdeps/unix/syscall-template.S:82
#1  0x00007ffff305319c in iemnet__receiver_readthread (arg=0x7fffec000f30) at iemnet_receiver.c:105
#2  0x00007ffff73ace9a in start_thread (arg=0x7ffff1e49700) at pthread_create.c:308
#3  0x00007ffff6ed53fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#4  0x0000000000000000 in ?? ()

Thread 3 (Thread 0x7ffff264a700 (LWP 3208)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00007ffff3052be4 in queue_pop_block (_this=0x7fffec000d20) at iemnet_data.c:313
#2  0x00007ffff3053638 in iemnet__sender_dosend (q=0x7fffec000d20, sockfd=<optimized out>) at iemnet_sender.c:79
#3  iemnet__sender_sendthread (arg=0x7fffec000cd0) at iemnet_sender.c:134
#4  0x00007ffff73ace9a in start_thread (arg=0x7ffff264a700) at pthread_create.c:308
#5  0x00007ffff6ed53fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#6  0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7ffff7fd5740 (LWP 3200)):
#0  0x00000000004693c9 in outlet_list ()
#1  0x00007ffff3257d59 in tcpclient_receive_callback (y=<optimized out>, c=0x7fffe40008c0) at tcpclient.c:269
#2  0x00007ffff30532aa in iemnet__receiver_tick (x=0x7fffec000f30) at iemnet_receiver.c:148
#3  0x00007ffff3053c7d in pollfun (z=<optimized out>, fd=<optimized out>) at iemnet_notify.c:126
#4  0x000000000047b17a in sys_domicrosleep.constprop.3 ()
#5  0x0000000000476731 in m_mainloop ()
#6  0x00007ffff6e0276d in __libc_start_main (main=0x411800 <main>, argc=6, ubp_av=0x7fffffffe138, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe128) at libc-start.c:226
#7  0x0000000000411831 in _start ()

The crash occurs when, after the recreation the [tcpclient], it receives bytes from the server.
Also if the client is disconnected before recreation, it doesn't crash.
I tested it on Ubuntu 12.04 64bit / pd 0.45.4 and Ubuntu 13.10 64bit / pd 0.45.4.

fails to compile under MinGW-64

trying to cross-compile iemnet for W32 using MinGW-64 (on Debian) fails:

$ PDDIR=/path/to/w32/pd/
$ make UNAME=MINGW CC=i686-w64-mingw32-gcc PD_PATH=${PDDIR} CFLAGS= LDFLAGS="-Wl,--enable-auto-image-base -L${PDDIR}/bin" ALL_LIBS="-Wl,-l:pd.dll -lwsock32  -lpthread" 
[...]
i686-w64-mingw32-gcc  -shared -L"/home/zmoelnig/lib/W32/pd/src" -L"/home/zmoelnig/lib/W32/pd/bin" -L"/home/zmoelnig/lib/W32/pd/obj" -o libiemnet.dll iemnet.o iemnet_data.o iemnet_receiver.o iemnet_sender.o iemnet_notify.o -Wl,-l:pd.dll -lwsock32  -lpthread -lkernel32 -luser32 -lgdi32
iemnet_notify.o:iemnet_notify.c:(.text+0x37b): undefined reference to `pipe'
collect2: error: ld returned 1 exit status
Makefile:278: recipe for target 'libiemnet.dll' failed
make: *** [libiemnet.dll] Error 1
$

[tcpclient] hangs if remote disconnects

if the remote server disconnects, then [tcpclient] hangs.

the problem seems to be the disconnection procedure...

[udpsend] strange behavior

open udpsend-help.pd
change the connect message to [connect 127.0.0.1 8888( and click it -> [udpsend] reports 1. on outlet, it is connected while nobody is listening on port 8888.
open udpreceive-help (with [udpreceive 9997] inside)
in udpsend-help.pd, click on [send 0 1 2 3(, nothing is receive, OK
now click on [port 8888( on udpsend-help.pd to change port number.
in udpsend-help.pd, click again on [send 0 1 2 3(, nothing is received while it's reported to be connected on port 8888

I have to disconnect and connect the [udpsend] again to make it work.

So, if we change the binding port of [udpreceive], the sended messages are not received until reconnection of [udpsend].