robotpy / pynetworktables Goto Github PK

Haven't checked it yet. Probably needs to be fixed.

Instead of:

NetworkTable.initialize()

Do

NetworkTables.initialize()
...etc

This will separate out the static initialization code from the main table code, and it's a clearer distinction between functionality.

For backwards compatibility purposes, in 2017 the static methods in NetworkTables will be copied to the NetworkTable class, but wrapped in a deprecated decorator.

Im getting this error
ImportError: No module named 'networktables'
when running any of the sample code

pip freeze shows
pynetworktables==2019.0.2
Thank you

ERROR:nt:Exception in valueChanged callback!
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/networktables2/networktablenode.py", line 159, in fireTableListeners
    listener.valueChanged(None, key, value, isNew)
  File "/usr/lib/python3.4/site-packages/networktables/__init__.py", line 86, in valueChanged
    value, isNew)
  File "/home/christian/PycharmProjects/RoboRIO-webdash/webdash/networktables_controller.py", line 50, in val_listener
    target_ref[key] = source.getValue(key)
  File "/usr/lib/python3.4/site-packages/networktables/__init__.py", line 619, in getValue
    return self.node.getValue(self.absoluteKeyCache.get(key))
  File "/usr/lib/python3.4/site-packages/networktables2/networktablenode.py", line 107, in getValue
    with self.entryStore.mutex:
AttributeError: 'ClientNetworkTableEntryStore' object has no attribute 'mutex'

It seems that neither ClientNetworkTableEntryStore object, nor any parent classes, ever initialize a mutex object. The AbstractNetworkTableEntryStore object does initialize an entry_lock object, but i'm not sure if that is supposed to be for a different function.

It appears that installation isn't working correctly due to missing files in the archive

Using the "pip install networktables" method resulted in:

• the neworktables.py file is missing from the neworktables package folder
• the ntcore module is missing

I tested on a windows system and a raspberry pi with the same results.

networktables install output.txt

It's very difficult to reproduce this bug, but we've seen it happen with multiple computers connected to the robot wireless. My suspicion is that the socket.settimeout that is commented out in SocketServerStreamProvider is the culprit. Unfortunately, that doesn't work in python 2.7, so I'm loathe to uncomment it. However, I suspect nobody runs NetworkTable servers using python 2.7...

The good news is that the robot code does not hang when this occurs -- it only disables networktables transmissions.

Seems like a bug.

platform windows 10, python3.7.2 native build, pynetworktables version 2019.0.0

in one git-bash window, i start a pynetworktables server:

networktables.NetworkTables.initialize()

in another git-bash window, I start a pynetworktables client:

networktables.NetworkTables.initialize(server="localhost")

back in the server window, i see a stream like this:

read error in handshake: end of file
read error in handshake: end of file
read error in handshake: end of file
read error in handshake: end of file
read error in handshake: end of file

fwiw: connections to robot-based server appear solid, just trying to get work done without a robot.

I've been continuously getting these errors in the DS console. Nothing appears to be failing, but it makes it really hard to debug other code since they fill up the console after a crash.

 09:33:51:575 WARNING : nt                  : Unhandled exception processing entry-notifier callback 
 Traceback (most recent call last): 
   File "/usr/local/lib/python3.6/site-packages/ntcore/callback_manager.py", line 116, in main 
     self.doCallback(listener.callback, item) 
   File "/usr/local/lib/python3.6/site-packages/ntcore/entry_notifier.py", line 79, in doCallback 
     callback(data) 
   File "/usr/local/lib/python3.6/site-packages/cscore/cameraserver.py", line 367, in _onTableChange 
     event.getEntry().setNumber(prop.get()) 
 AttributeError: 'EntryNotification' object has no attribute 'getEntry'

I'm not sure I want to know how the high-level NetworkTables API deals with keys that don't start with /. But I suspect users wouldn't be able to use keys like that.

We should just drop any keys that aren't valid UTF-8 (but still log a warning).

I'm trying to use pynetworktables in a very strange threading environment (eventlet). Everything works fine, except that it doesn't connect to a NetworkTables server if the server is started after the client program.
It connects properly and works fine if the server process is started first.
I know this isn't a normal use-case, so I'm not sure whether or not to call it a bug. Is there a command I can call to force the library to check for a connection?

Need to update this for NT 3.0 compatibility. WPILib Java/C++ was basically a complete rewrite, so this could be done as either a translation of that rewrite into Python or by updating this code with the NT 3.0 protocol.

Title sums it up.

>>> type(nt.getBooleanArray('LED'))
tuple 

Documentation says it should be this:
:rtype: list(bool)

It's commented out right now

https://github.com/robotpy/pynetworktables/blob/master/sip/module.sip#L55-L57

Some of the tests in the java library are useful. I don't really want to do the comprehensive networktables2 interface test, as users probably shouldn't be using that interface, and we'll probably change it because it's too complex.

There should be at least one test that starts a socket and sends data back and forth.

Don't want to crash the NetworkTables thread. Use logging.exception() to send the error to the python logger instead.

This may be fairly involved, there are some updates to the way ntcore does synchronization. Additionally, they changed NT to have instances instead of lots of static globals -- but we do that anyways. Still, the code will need to be reviewed and updated.

UnicodeDecodeError trying to connect to Network Tables on roboRIO (Java) from Raspberry Pi with Microsoft Lifecam plugged into roboRIO.

>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> from networktables import NetworkTables as NT
>>> NT.initialize('10.35.28.32')
INFO:nt:NetworkTables 2017.0.4 initialized in client mode
>>> DEBUG:nt:client connected
ERROR:nt:Unhandled exception during handshake
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/ntcore/network_connection.py", line 240, in _readThreadMain
    handshake_success = self.m_handshake(self, _getMessage, self._sendMessages)
  File "/usr/local/lib/python2.7/dist-packages/ntcore/dispatcher.py", line 488, in _clientHandshake
    msg = get_msg()
  File "/usr/local/lib/python2.7/dist-packages/ntcore/network_connection.py", line 228, in _getMessage
    return Message.read(self.m_stream, decoder, self.m_get_entry_type)
  File "/usr/local/lib/python2.7/dist-packages/ntcore/message.py", line 123, in read
    value = codec.read_value(value_type, rstream)
  File "/usr/local/lib/python2.7/dist-packages/ntcore/wire.py", line 126, in read_value
    return Value.makeStringArray([self.read_string(rstream) for _ in range(alen)])
  File "/usr/local/lib/python2.7/dist-packages/ntcore/wire.py", line 198, in read_string_v3
    return rstream.read(slen).decode('utf-8')
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf0 in position 61: invalid continuation byte
INFO:nt:DISCONNECTED 10.35.28.32 port 1735 (Robot)

That way when you run smart dashboard locally you only get variables you want, not all of them. It becomes very cluttered after a while.

We left it out at some point, but the WPILib API uses it extensively so we should probably do this too.

As mentioned in #30, this will happen in 2018.

I would like to propose we drop support for Python 3.3 in 2019.

Python 3.3 is already EOL (as of September 2017), and is not distributed by major, currently supported GNU/Linux distros, so I don't anticipate this being a problem for any team.

We've already announced on CD that pynetworktables will continue to support Python 3.3 for this year, so I guess we're keeping it this way. I wouldn't imagine we'd have many more changes this year, so this probably won't be much of a problem for us anyway.

It's a bug in ntcore: wpilibsuite/ntcore#275

Due to the bug being pretty difficult to trigger and being almost the end of robot season, we should not push a fix for this until after Saturday.

The _NtProperty class should actually use the inst it's given...

In many matches, we've seen our vision system and other important variables drop out, and I now think I've figured out that this is due to pynetworktables (NT v3.0) not working. Indeed, during one match, I could see that isConnected() was continually reading false. The interesting thing is that in some matches it works and some matches it doesn't. When not connecting via FMS, it always works.

Since the LabVIEW dashboard was successfully showing network tables variables, I can conclude that this problem is one specific to pynetworktables.

We were using a raspberrypi running network tables for communication with the roborio, but when the roborio creates a table for microsoft camera which has a special character in its description the function NetworkTables.initialize(server=ip) cannot start connection because a UnicodeEncodeError during the handshake.

The debuging return the following errors:

DEBUG:nt:client connected
DEBUG:nt:NetworkConnection stopping (<ntcore.network_connection.NetworkConnection object at 0x74469330>)
ERROR:nt:Unhandled exception during handshake
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/ntcore/network_connection.py", line 240, in _readThreadMain
    handshake_success = self.m_handshake(self, _getMessage, self._sendMessages)
  File "/usr/local/lib/python2.7/dist-packages/ntcore/dispatcher.py", line 488, in _clientHandshake
    msg = get_msg()
  File "/usr/local/lib/python2.7/dist-packages/ntcore/network_connection.py", line 228, in _getMessage
    return Message.read(self.m_stream, decoder, self.m_get_entry_type)
  File "/usr/local/lib/python2.7/dist-packages/ntcore/message.py", line 123, in read
    value = codec.read_value(value_type, rstream)
  File "/usr/local/lib/python2.7/dist-packages/ntcore/wire.py", line 114, in read_value
    return Value.makeString(self.read_string(rstream))
  File "/usr/local/lib/python2.7/dist-packages/ntcore/value.py", line 51, in makeString
    return ValueType(NT_STRING, str(value))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xae' in position 9: ordinal not in range(128)
INFO:nt:DISCONNECTED roboRIO-3478-FRC.local port 1735 (Robot)
DEBUG:nt:write thread died (<ntcore.network_connection.NetworkConnection object at 0x74ce7230>)

It would be a good idea to use Hypothesis to verify round-trips work as intended.

It would also be a good idea to write some tests to verify that we can interact with ntcore correctly.

Started testing possibly using on our NVIDIA Jetson and am getting this error. Testing with other machines also throws this error with Python 2.7, but everything works with Python 3.5. Any ideas how I can fix this? I can add any additional information that may be useful.

root@tegra-ubuntu:/home/ubuntu/Desktop/Code/2016-Robot-Code/Vision/main# python networkTest.py
INFO:nt:Client 0xb601e918 entered connection state: CONNECTED_TO_SERVER
Exception in thread Client Connection Reader Thread:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/lib/python2.7/dist-packages/networktables2/connection.py", line 194, in run
    self.connection.read(self.adapter)
  File "/usr/local/lib/python2.7/dist-packages/networktables2/connection.py", line 102, in read
    serverName, (flags,) = SERVER_HELLO.read(self.rstream)
  File "/usr/local/lib/python2.7/dist-packages/networktables2/messages.py", line 71, in read
    nameLen = leb128.read_uleb128(rstream)
  File "/usr/local/lib/python2.7/dist-packages/networktables2/leb128.py", line 18, in read_uleb128
    result |= (b & 0x7f) << shift
TypeError: unsupported operand type(s) for &: 'str' and 'int'

Easy enough to do.

Our team is trying out GRIP, but I can't seem to get the data that is published over network tables by it. I have successfully printed the data in C++ with something like double x = grip->GetNumberArray("ball/x", llvm::ArrayRef<double>()) (where grip is the network table and ball is a subtable). However, it doesn't seem to work in Python:

getData(): no longer seems to exist, but still in docs
getDouble(): returns TypeError: Cannot get number for '/GRIP/ball/x', is a Array of [Double]
getNumber(): returns exact same thing as getDouble().

Is there anything else I should try? I confirmed that the data is in network tables with the TableViewer utility.

16:17:15:412 ERROR : robotpy : ---> The startCompetition() method (or methods called by it) should have handled the exception.
Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/wpilib/robotbase.py", line 185, in main
robot.startCompetition()
File "/usr/local/lib/python3.4/site-packages/wpilib/samplerobot.py", line 135, in startCompetition
LiveWindow.setEnabled(True)
File "/usr/local/lib/python3.4/site-packages/wpilib/livewindow.py", line 81, in setEnabled
LiveWindow.initializeLiveWindowComponents()
File "/usr/local/lib/python3.4/site-packages/wpilib/livewindow.py", line 56, in initializeLiveWindowComponents
table = LiveWindow.livewindowTable.getSubTable(c.subsystem).getSubTable(c.name)
File "/usr/local/lib/python3.4/site-packages/networktables/init.py", line 449, in getSubTable
return self.provider.getTable(self.absoluteKeyCache.get(key))
File "/usr/local/lib/python3.4/site-packages/networktables/init.py", line 342, in get
cachedValue = self.path + NetworkTable.PATH_SEPARATOR + key
TypeError: Can't convert 'int' object to str implicitly

Locals at innermost frame:

When i attempt to install on both Linux (Ubuntu 12.04) and Mac OS, there appears to be an issue between SIP and PyNetworkTables. Both are running Python3, however I tested on Multiple versions of SIP.

The traceback is posted below (With my username removed)

Somebody's-MacBook-Air:pynetworktables USERNAME$ ROBOTPY="../robotpy" python3 setup.py build
running build
running build_ext
building 'pynetworktables' extension
/usr/local/Cellar/sip/4.13.3/bin/sip -g -e -I /Users/USERNAME/pynetworktables/sip -I /Users/USERNAME/robotpy/Packages/wpilib/sip -c build/temp.macosx-10.8-x86_64-3.3 -b build/temp.macosx-10.8-x86_64-3.3/module.sbf -I /usr/local/share/sip /Users/USERNAME/pynetworktables/sip/module.sip
Traceback (most recent call last):
File "setup.py", line 119, in
cmdclass = {'build_ext': custom_build_ext}
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 917, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
cmd_obj.run()
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/command/build.py", line 126, in run
self.run_command(cmd_name)
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
cmd_obj.run()
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/command/build_ext.py", line 347, in run
self.build_extensions()
File "setup.py", line 101, in build_extensions
sipdistutils.build_ext.build_extensions(self)
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/command/build_ext.py", line 456, in build_extensions
self.build_extension(ext)
File "/usr/local/lib/python2.7/site-packages/sipdistutils.py", line 111, in build_extension
build_ext_base.build_extension(self, ext)
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/command/build_ext.py", line 483, in build_extension
sources = self.swig_sources(sources, ext)
File "/usr/local/lib/python2.7/site-packages/sipdistutils.py", line 153, in swig_sources
out = self._get_sip_output_list(sbf)
File "/usr/local/lib/python2.7/site-packages/sipdistutils.py", line 58, in _get_sip_output_list
for L in file(sbf):
NameError: global name 'file' is not defined

I think this will eliminate any possibility of deadlock, but it's more overhead. I'm not sure I want to do it, but maybe it makes sense to do this.

Otherwise you get into these weird situations:

self.foo = 5
if self.foo == 5:
     # this might not be true

This would make user connections more robust than using the 'server' keyword.

For example, removeWsConnectionListener, removeRobotConnectionListener, to introduce full event features.

Leaving this as a note to myself. Instead of

NetworkTable.setIPAddress(ip)
NetworkTable.setClientMode()
NetworkTable.initialize()

How about this instead?

NetworkTables.initialize(server='roborio-xxx-frc.local')

If no server argument is given, then it initializes in server mode instead. Can leave the old functions (setIPAddress, setClientMode) for backwards compatibility too.

Hi Dustin,

First of all, thanks for this! I have found it really useful.

Here's the issue:

The robot connects, prints CONNECTED, pynetworktables2js sends value ✅
The robot/network disconnects. Nothing happens ❌

Any idea why?

Thanks

When running the setup command 'ROBOTPY="/home/linaro/robotpy" python setup.py', the build fails in networktables2/thread/DefaultThreadManager.h. The relevant terminal output is as follows:

Adding 'public' to line 48 of DefaultThreadManager.cpp and 'public:' to line 23 of DefaultThreadManager.h
resolves these errors but the build still fails:

I tried this:

logging.basicConfig(level=logging.DEBUG)
NetworkTables.initialize(server="10.6.66.5")

And got this:

INFO:nt:NetworkTables initialized in client mode
DEBUG:nt.th:Started thread nt-dispatch-thread-0
DEBUG:nt.th:Started thread nt-client-thread-0
DEBUG:nt:client connected
DEBUG:nt.th:Started thread nt-net-write-0
DEBUG:nt.th:Started thread nt-net-read-0
ERROR:nt:Unhandled exception during handshake
Traceback (most recent call last):
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\network_connection.py", line 248, in _readThreadMain
    handshake_success = self.m_handshake(self, _getMessage, self._sendMessages)
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\dispatcher.py", line 571, in _clientHandshake
    msg = get_msg()
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\network_connection.py", line 236, in _getMessage
    return Message.read(self.m_stream, decoder, self.m_get_entry_type)
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\message.py", line 123, in read
    value = codec.read_value(value_type, rstream)
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\wire.py", line 114, in read_value
    return Value.makeString(self.read_string(rstream))
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\wire.py", line 199, in read_string_v3
    return rstream.read(slen).decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xea in position 9: invalid continuation byte
INFO:nt:DISCONNECTED 10.6.66.5 port 1735 (Robot)
DEBUG:nt.th:Thread nt-net-read-0 exited
DEBUG:nt.th:Thread nt-net-write-0 exited

TODO: investigate them

Example failure: https://travis-ci.org/robotpy/pynetworktables/jobs/179285410

As implemented in wpilibsuite/ntcore@77edf1e

Now that we're Python-3 only, it'd be useful for finding bugs I suspect.

Some kind of queue buildup or race condition during reconnects...

Reference: https://www.chiefdelphi.com/forums/showthread.php?t=164590