Not able to connect after last kernel upgrade about pyserial HOT 23 CLOSED

Landed here because I have been experiencing the same error "device reports readiness to read but returned no data (device disconnected?)" over the last couple of days. In my case, the device works most of the time, but fails sporadically. I'm running a Python data logger saving CSV strings from an Arduino to disk. Host running 16.04 with 4.4.0-53 kernel. Python 2.7.12.

The logger seems to die randomly with the exact error message, but in looking for answers here, I believe I've figured it out. I checked the kernel ring buffer, and sure enough there are USB error messages with the EXACT time as the last entry in my log file.

[Wed Jan 4 21:55:29 2017] usb usb3-port1: disabled by hub (EMI?), re-enabling...

I'm logging the power consumption and energy expended to keep a refrigerator at temperature. Sometimes when the compressor starts up, the EMI gets into the USB cable and interrupts enough traffic to force the kernel to drop the device. The Arduino resets and a moment later logs a re-insert. This is enough to make pyserial lose it's lunch.

I don't see this as a bug, but more of a missing feature. Ideally, pyserial should handle removal/insertion events (and EMI that causes the system to drop devices) more gracefully (maybe it does, but I haven't dug into it). Pyserial has provisions to specify VIP/PID and serial number. This is more than enough info to ensure that when the device coms back online, the program using pyserial can continue on.

Anyway, hope this helps

from pyserial.

Approve this. Have no idea why, but my old code, which works just fine couple months ago, crashes for two or three weeks. Also, fix does not work for me, after some time I have

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/serial/serialposix.py", line 468, in read
    buf = os.read(self.fd, size-len(read))
BlockingIOError: [Errno 11] Resource temporarily unavailable

minicom have no problems, and seems there is something «tricky»

[broomrider ~]$ uname -a
Linux 065-pc 3.13.0-65-generic #105-Ubuntu SMP Mon Sep 21 18:50:58 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[broomrider ~]$ python3 --version
Python 3.4.3

from pyserial.

Please upgrade to the latest version, available here, for further tests. It may be that the bug is already fixed. At least the second problem with EAGAIN (errno 11) should be fixed.

from pyserial.

The same happened with pronterface, but couldn't test with the miniterm. Commenting the same lines made it work again. The only differences are that

• Now I can't run miniterm.py. I get

Traceback (most recent call last):
  File "/usr/local/bin/miniterm.py", line 23, in <module>
    from . import hexlify_codec
ValueError: Attempted relative import in non-package

hexlify_codec is in tools directory, but for some reason it's not taking it.

• Also when I comment the same lines as before as a workaround, it seems to be able to disconnect faster than before. Previously it seemed to get blocked, but now it disconnects very quickly.

from pyserial.

start miniterm as python -m serial.tools.miniterm ... and the relative import will work (but maybe i should remove that one anyway, it's the only one, all others are absolute) - [just done that change]

from pyserial.

It happens the same as before

Exception in thread rx:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/idlt/instalacion2/pyserial/pyserial/serial/tools/miniterm.py", line 402, in reader
    data = self.serial.read(self.serial.in_waiting or 1)
  File "/usr/local/lib/python2.7/dist-packages/serial/serialposix.py", line 494, in read
    raise SerialException('device reports readiness to read but returned no data (device disconnected or multiple access on port?)')
SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

Commenting the lines works, but also blocks when disconnecting (it's reasonable)

from pyserial.

Commenting causes high cpu load, since we do not wait in select at all, I think.

from pyserial.

Also, access to the /dev/ttyS0 via windows virtualbox from ubuntu 14.04 causes same high CPU load.
added:
It is really looks like ubuntu problem.

from pyserial.

I also see that problem, but it only seems to affect the kernel from Ubuntu 14.04 LTS with updates (that exact kernel version or even Ubuntu specific), at least the newer kernel on the Raspi still works as expected as did all older ones for many years. Also looking at the launchpad.net bug tracker, it seems to be an Ubuntu specific issue than may be fixed with the next kernel update.

PosixPollSerial still works as expected but poll is not supported properly on some other platforms.
So as a temporary workaround you can either boot to a previous kernel or import PosixPollSerial as Serial (though that's not really nice for cross platform compatibility :/ )

I've tried some different settings and could not get it to work otherwise, so i'll wait a moment and hope for a fix in the next Ubuntu kernel, before I toss away an implementation that worked on many posix systems over many years...

from pyserial.

Looks like 3.13.0-66 is fixing that issue. The Kernel arrived in 14.04 LTS today (and was available for some time in trusty-proposed)

from pyserial.

This bug is back in Ubuntu 16.04
uname -a
Linux greger 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line 495, in read
raise SerialException('device reports readiness to read but returned no data (device disconnected or multiple access on port?)')
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

from pyserial.

@jhagberg I was using miniterm.py a lot this week and didn't have any problems (but in the past, I had them too, with the same hardware). Maybe your problem has a different source?

me$ dpkg -s python-serial
Package: python-serial
Status: install ok installed
Version: 3.0.1-1

me$ uname -a
Linux 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

me$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04 LTS
Release: 16.04
Codename: xenial

from pyserial.

getting the same issue, following that thread.

from pyserial.

Same issue here on an up to date 16.04.

I made a fork version with the raise commented as suggested in first post just the time of having this issue properly patched. You can install it this way:
pip install -e git+https://github.com/flagos/pyserial.git#egg=pyserial

from pyserial.

I have also updated to 16.4.1, but I can not reproduce the issue.

I have a more recent Kernel:
Linux 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

As a workaround, you may also try the alt:// protocol with VTIMESerial, e.g. alt:///dev/ttyUSB0\?class=VTIMESerial (if your app uses a timeout or using git master if it has no timeout)

from pyserial.

Thanks. It helps since I now think I have the same issue. I have several freezers and when the compressor turns on I get a glitch and the sounds pauses from the computer since I use SPDIF over a RCA cabel to my sound system. It is probably the freezers that mess upp the USB connection. Thanks! @playaspec 🥇

from pyserial.

I see this happen for the Cura 2.4 application on latest Ubuntu 16.04 on 2/15/2017.
There are no freezers or compressors involved, just a 3D printer driven by the Printrboard (which is an AVR USB device, different from the Arduinos but related.)
My guess is that this is actually a mis-match between pyserial expectations and kernel driver behaviors.
It's reasonable for character devices to return 0 data for a read request, and I think pyserial should just deal with that.
If you're worried about an infinite read-nothing loop, then detect "no data," usleep for 100 microseconds, and retry. You may even want to use this as a signal to close and re-open the file descriptor, and if re-open fails, then actually die.

from pyserial.

About the "expectations".

pySerial uses select on POSIX and expects that when it signals "ready-to-read", that either actual data is available or otherwise its interpreted as fatal fault and thus an exception is raised. Early versions of pySerial just did read again and then you'd end up in 100% CPU load when someone unplugged the USB-Serial adapter. Not a good thing and no chance for the application to do anything about. Now you have the exception and can decide in the application what to do, e.g. close and try to reopen the port.

Actually poll would have nicer properties, such as separate flags for "ready-to-read" and "error". I tried to use that one but it turns out that some fruity OS and hardware vendor decided to not support that part of POSIX... and I rather have the same behavior on all POSIXy systems than have an other differently behaving implementation. But if someone wants to try poll on a system that actually implements that call (such as Linux), you can try it out with alt:///dev/ttyUSB0?class=PosixPollSerial

from pyserial.

Problem is, I can 't change the application, because it's delivered as a "baked" AppImage.
Also, select() has the "exception" file descriptor list, separate from the "read" file descriptor list.
I don't know what's actually going on in this case, because gtkterm works fine, as does "dd" into the serial port, as does custom C++ programs that talk to the serial port (using termios.)
So, to me, it seems like pyserial is not providing the same behavior as you'd expect from other programs.
Separately, I don't even understand why select() is needed. If you want timeouts on reading a serial port, that's what VTIME is for, and Python doesn't expose a globally multiplexed asynchronous I/O model like Node or Haskell.

from pyserial.

I think the "exception" descriptor did work for errors, it's for "out of band data".

I use select because VTIME is not very flexible and has a limited range of possible timeouts. But you could try it using alt:///dev/ttyUSB0?class=VTIMESerial as port, if the application internally uses serial_for_url().

from pyserial.

Well, most other programs work great, and don't use select(), whereas pyserial has this problem pop up in a number of different situations over different years, and uses select().
From a general code robustness point of view, I would suggest you have enough data to now know that select() was a mistake and use VTIME instead. When select() was initially used, it's pretty clear that wasn't a well documented shortcoming!
Anyway, no, I can't try it myself, because the program I'm using, Cura v2.4, is delivered as an AppImage, which I can't edit, and it doesn't even allow me to configure the port name to use.

from pyserial.

Go to the Cura archives and get the last version of cura 15. Then make the patches to pyserial. That's where i am, but i don't know what changes to make.

from pyserial.

My 16.04 started with this today, and after upgrade to 18.04 it actually got worse :(

Now is get
'device reports readiness to read but returned no data '
also with a couple of FTDI's including the ESP dev card.

And also device busy and other stuff.
Also gets disconnected after a few minutes.

from pyserial.