dfki-ni / rospy_message_converter Goto Github PK

Hi, first of all thank you for your wonderful work!

I wanted to ask if you support messages from packages like sensor_msgs, like JointState?

If not, how would I go about adding support for it?

Further, for completely custom messages, would this be the same?

Thank you very much!

Hello, are there plans to expand this to ROS2 and rclpy? By the package name (rospy) it may be misleading, although useful, to also support rclpy. Would PRs be accepted for this purpose?

Hi!

This is a very useful package, but I'm having trouble, since I've updated to:

$ dpkg -l | grep message-convert
ii  ros-melodic-rospy-message-converter          0.5.2-1bionic.20200801.235102                    amd64        Converts between Python dictionaries and JSON to rospy messages.

It seems like it can't handle numpy types anymore. Here is an example:

import numpy as np

from geometry_msgs.msg import PoseStamped
from rospy_message_converter.message_converter import convert_ros_message_to_dictionary, \
    convert_dictionary_to_ros_message

pose = PoseStamped()
pose.pose.position.x = np.float64(1)

d = convert_ros_message_to_dictionary(pose)
msg = convert_dictionary_to_ros_message('geometry_msgs/PoseStamped', d)

Running it, gives me this error:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-bb41facf3913>", line 1, in <module>
    runfile('/home/stelter/giskard_ws/src/giskardpy/scripts/muh4.py', wdir='/home/stelter/giskard_ws/src/giskardpy/scripts')
  File "/home/stelter/pycharm-2018.3.1/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/home/stelter/giskard_ws/src/giskardpy/scripts/muh4.py", line 11, in <module>
    msg = convert_dictionary_to_ros_message('geometry_msgs/PoseStamped', d)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy_message_converter/message_converter.py", line 115, in convert_dictionary_to_ros_message
    field_value = _convert_to_ros_type(field_type, field_value)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy_message_converter/message_converter.py", line 146, in _convert_to_ros_type
    field_value = convert_dictionary_to_ros_message(field_type, field_value)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy_message_converter/message_converter.py", line 115, in convert_dictionary_to_ros_message
    field_value = _convert_to_ros_type(field_type, field_value)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy_message_converter/message_converter.py", line 146, in _convert_to_ros_type
    field_value = convert_dictionary_to_ros_message(field_type, field_value)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy_message_converter/message_converter.py", line 115, in convert_dictionary_to_ros_message
    field_value = _convert_to_ros_type(field_type, field_value)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy_message_converter/message_converter.py", line 139, in _convert_to_ros_type
    raise TypeError("Wrong type: '{0}' must be {1}".format(field_value, field_type))
TypeError: Wrong type: '1.0' must be float64

That is odd, cause this 1.0 is of type float64.
Can you please look into it?
It does work with version

0.5.1-1bionic.20200530.021813 

Hello,

I am wondering if you could give an example of formatting the JSON to send a Twist message.

Thank you!

Hello Brandon,

I wonder if you still maintains this package or not. Recently, I had to patch rosbridge message parser to handle fixed size of uint array like uuid_msgs.UniqueID.

So I am just linking the related issue which you might be interested in RobotWebTools/rosbridge_suite#93

Hi, I have a dictionary that I want to send over ROS which looks like this:
dictionary = {'left_e0': 0.6745936036696302, 'left_e1': 1.3535597568240276, 'left_s0': -1.6707208611310596, 'left_s1': -1.5144475501651122, 'left_w0': 0.8013426068418519, 'left_w1': 0.982298380995793, 'left_w2': 1.2450093099285662}
How should I go about doing this, since applying message_converter.convert_dictionary_to_ros_message('std_msgs/String', dictionary) doesn't work?

Right now ros time objects will be converted to a dictionary of the form {'secs': sec_value, 'nsecs': nsecs_value}. In my group we usually convert ros time directly to a float with the to_sec() function.

It would be cool if this could be supported here as well, maybe by adding a flag time_as_float or time_to_sec. I am happy to submit a merge request if you think this would be a nice feature to have.

ROS message type "std_msgs/String" has no field named "StrSys"
But I can't find "StrSys" on the Internet

The character array created is of different length than the original byte array.

Hello,

I'm pretty new to ROS and was going about making some messages and service calls parameter based, but found it clunky. I found I just keep the same lines of code in the node, replacing some var = value pair with maybe var = rospy.get_param('foo'), or params = get_param('foo') and then var = params['bar'']. Anyway, I'm sure you get this, as you created this wonderful package!

I was wondering about extending your code to do service calls as well. I took a stab at adding a completely ripped off variant of your convert_dictionary_to_ros_message() function and it worked flawlessly! Honestly, other than changing message to service, there was only one call update to roslib.message.get_service_class().

Thoughts on this?

def convert_dictionary_to_ros_service(service_type, dictionary):
    """
    Takes in the service type and a Python dictionary and returns a ROS service request.

    Example:
        service_type = "std_srvs/SetBoolRequest"
        dict_message = { "data": "Hello, Robot" }
        ros_service = convert_dictionary_to_ros_service(service_type, dict_message)
    """
    service_class = roslib.message.get_service_class(service_type)
    service = service_class()
    service_fields = dict(_get_message_fields(service))

    for field_name, field_value in dictionary.items():
        if field_name in service_fields:
            field_type = service_fields[field_name]
            field_value = _convert_to_ros_type(field_type, field_value)
            setattr(service, field_name, field_value)
        else:
            error_message = 'ROS message type "{0}" has no field named "{1}"'\
                .format(service_type, field_name)
            raise ValueError(error_message)

    return service

In json_mesage_converter.py, I have to pass the message_type. That type is in the json isn't it? i think binney's code has to load the json and actually figure out the type and then call this function. can i get a version without the message_type. then i don't have to even think about what's going on in the json -- that's between you and argenteam. does that make sense?

I am currently using the rospy_message_converter to convert data points from a database into ROS. However, the database values have an _id field, which does not exist in the ROS message. After setting strict_mode=false in the convert_json_to_ros_message function, I get the following output:

...
ERROR - 2021-11-23 13:26:06,846 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,846 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,846 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,847 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,847 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,847 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,847 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,847 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,848 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,848 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,848 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,848 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,848 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,849 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,849 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,849 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,849 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,850 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,850 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,850 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,850 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,850 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,851 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,851 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,851 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
ERROR - 2021-11-23 13:26:06,851 - message_converter - ROS message type "MyMsg" has no field named "_id"! It will be ignored.
...

As I am importing thousands of data points, this becomes too verbose.

I am aware that I could filter out the "_id" field before calling the convert_json_to_ros_message, but this would kill the purpose of the strict_mode=false option.

I believe it would be good to either make this output as debug, or give the user the ability to change the log level.

First, this is really awesome. I think it should be merged into ros_comm and work implicitly with the rospy publisher.

In convert_dictionary_to_ros_message(), the API accepts a string name of a message here, then the classtype is retrieved from roslib. Is there any reason why the API doesn't just take the message type directly?

For example, instead of the current API, why not:

from rospy_message_converter import message_converter
from std_msgs.msg import String
dictionary = { 'data': 'Howdy' }
message = message_converter.convert_dictionary_to_ros_message(String, dictionary)

To provide verification and feedback via task manager, need to know if parsing failed or succeeded via an exception...suggested exception type = ParseException

Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from rospy_message_converter import message_converter
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pablo/src/freedomrobotics/freedom_python/env/local/lib/python2.7/site-packages/rospy_message_converter/message_converter.py", line 34, in <module>
    import roslib.message
ImportError: No module named roslib.message

I just created a simple virtualenv and installed the library with pip.

Is there anyother package that should be installed for running this? It would be nice to add it to the README.

I am trying to convert my nested json string to ros msg, but could not able to do that ?

{
        "position": {
            "latitude": 44.62,
            "longitude": -63.66
        },
        "state": "HALIFAX",
        "positional_uncertainty": 0,
        "classification": [
            {
            "type": "scanning",
            "confidence": 0.77
            },
            {
            "type": "scanning",
            "confidence": 0.8
            },
            {
            "type": "scanning",
            "confidence": 0.3
            }
        ]
        }

But when i try to convert this as ros msg, I am having an error indicating that AttributeError: 'str' object has no attribute 'items'
What i did is :

1. Created a custom msg for this.
2. For python this is just the another dictionary.
3. so i use json_message_converter.convert_json_to_ros_message(msg_type, dict_msg)
4. then try to publish the data.

What could go wrong ? Could you please point me in the right direction ?

Could you do a binary release into lunar and melodic? I've just tried it, and it compiles fine; also, all tests pass. So this should just be a matter of running bloom, no changes to the source code required.

Hi, thanks for the package but I am having trouble building it.
After running catkin_make I got the following error (I am using Noetic on Ubuntu 20.04 and I cloned from the master branch):

-- +++ processing catkin package: 'rospy_message_converter'
-- ==> add_subdirectory(rospy_message_converter)
-- Using these message generators: gencpp;geneus;genlisp;gennodejs;genpy
/opt/ros/noetic/lib/python3/dist-packages/_distutils_hack/init.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the distutils module in sys.modules. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/opt/ros/noetic/lib/python3/dist-packages/_distutils_hack/init.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

error: invalid command 'rospy_message_converter'
CMake Error at /opt/ros/noetic/share/catkin/cmake/safe_execute_process.cmake:11 (message):
execute_process(/home/milos/catkin_ws/build/catkin_generated/env_cached.sh
"/usr/bin/python3"
"/opt/ros/noetic/share/catkin/cmake/interrogate_setup_dot_py.py"
"rospy_message_converter"
"/home/milos/catkin_ws/src/rospy_message_converter/setup.py"
"/home/milos/catkin_ws/build/rospy_message_converter/catkin_generated/setup_py_interrogation.cmake")
returned error code 1
Call Stack (most recent call first):
/opt/ros/noetic/share/catkin/cmake/catkin_python_setup.cmake:46 (safe_execute_process)
rospy_message_converter/CMakeLists.txt:23 (catkin_python_setup)

-- Configuring incomplete, errors occurred!
See also "/home/milos/catkin_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/milos/catkin_ws/build/CMakeFiles/CMakeError.log".
make: *** [Makefile:4394: cmake_check_build_system] Error 1
Invoking "make cmake_check_build_system" failed

Is it possible to make this a pip library?

There should be an if(CATKIN_ENABLE_TESTING) around the add_rostest line in CMakeLists.txt. Otherwise, it throws an error when running catkin_make_isolated: The CMake code should only use it inside a conditional block which checks that testing is enabled.

Fix:

if(CATKIN_ENABLE_TESTING)
add_rostest(...)
endif()

Hi, I have installed this package on Ubuntu 16.04 (ROS kinetic) via: sudo apt install ros-kinetic-rospy-message-converter

I have found that this works as expected if I try and use a dictionary only containing a single item called 'data', as in the readme.md example eg.
message = message_converter.convert_dictionary_to_ros_message('std_msgs/String', {'data': 'some_data' })

but if I attempt to convert any other dictionary, eg. message = message_converter.convert_dictionary_to_ros_message('std_msgs/String', {'mydata': 'some_data' })

I get:

ValueError: ROS message type "std_msgs/String" has no field named "mydata"

Any help in getting this to work would be very much appreciated.

It looks like so, I'm experiencing this issue:

a custom ROS msg has a field like

uint8[8] intensities

and after

json_str = json_message_converter.convert_ros_message_to_json(data)

I see it is turned into a string (printing it with rospy.loginfo(json_str)).

Is it a bug or a feature?
Is it possible to force it to not consider it as a string?

Thank you.

when converting to ros string, should be decoded as utf-8, as unicode is not supported as specified by http://wiki.ros.org/msg

unicode strings are currently not supported as a ROS data type. utf-8 should be used to be compatible with ROS string serialization. In python 2, this encoding is automatic for unicode objects, but decoding must be done manually. In python 3, both encoding and decoding are automatic.

Proposed fix:
_convert_to_ros_primitive function in
src/rospy_message_converter/message_converter.py
line 129:

def _convert_to_ros_primitive(field_type, field_value):
    if field_type == 'string':
        field_value = field_value.encode('utf-8')
    return field_value

To reproduce this, rospkg and catkin-pkg also need to be installed to work with python3, e.g. using pip3:

$ pip3 install rospy-message-converter
Collecting rospy-message-converter
  Using cached https://files.pythonhosted.org/packages/c3/7d/5e153b0d7dea31faa4a12505ff04f26dce77716f97d963af2e165ac12984/rospy_message_converter-0.4.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-1zv7ys8z/rospy-message-converter/setup.py", line 8, in <module>
        package_dir={'' : 'src'},
      File "/usr/local/lib/python3.5/dist-packages/catkin_pkg/python_setup.py", line 74, in generate_distutils_setup
        package = parse_package(package_xml_path)
      File "/usr/local/lib/python3.5/dist-packages/catkin_pkg/package.py", line 513, in parse_package
        xml, filename = _get_package_xml(path)
      File "/usr/local/lib/python3.5/dist-packages/catkin_pkg/package.py", line 489, in _get_package_xml
        raise IOError('Path "%s" is neither a directory containing a "%s" file nor a file' % (path, PACKAGE_MANIFEST_FILENAME))
    OSError: Path "." is neither a directory containing a "package.xml" file nor a file
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-1zv7ys8z/rospy-message-converter/

In python3

The call for testing today revealed that this package hasn't been released yet for Noetic.

Would you mind also releasing it for Noetic? 🙂

So messages properly parsed before are not anymore.
I understand that this is reasonable option... so fill free to close if it has been an intended change.

I wonder if the documentations can be migrated to ros wiki..

Array in message data being read as string, the data of PointCloud2 is interpreted as a random code, not an array. how Can I do?

When running this package in python 3, the conversion of a string fails because of the ros implementation of std_msgs/msg/_String.py. They call encode() on the data field of a string object due to python3 being always True:

def serialize(self, buff):
    """
    serialize message into buffer
    :param buff: buffer, ``StringIO``
    """
    try
      _x = self.data
      length = len(_x)
      if python3 or type(_x) == str:
        _x = _x.encode('utf-8')

However, the rospy_message_converter already does this when it converts a string python object to a ros_primitive in rospy_message_converter/src/rospy_message_converter/message_converter.py:149:

def _convert_to_ros_primitive(field_type, field_value):
    if field_type == "string":
        field_value = field_value.encode('utf-8')
    return field_value

The fix is to add a check that we are not in python3 before doing the conversion.

def _convert_to_ros_primitive(field_type, field_value):
    if field_type == "string" and not python3:
        field_value = field_value.encode('utf-8')
    return field_value

This works for std_msgs/String and sensor_msgs/JointState but I am not sure if this breaks anything else as I don't have the infrastructure in place to test this.

I will create a PR shortly.

Hello, using this library I see that no error is raised during conversion a python dict to a ros msg, for these two scenarios:

1. a field in ros msg is not in python dict
2. python dict contain a <key, value> which value is not the same type of ros msg

While in the former case the library create a default value for the missing field, the latter is allowed (but must be unacceptable).
Anyway also for the first case I prefer an error instead of a silenting default value.

I personally customized library for theese two consraints that are mandatory for my purpose.
Are theese two in plan to be added, or they're out of the scope of this project?

(python 2.7, kinetic-ros)

If for example i want to publish a dictionary with integer keys it wouldn't let me do it, so i changed the keys to strings but it literally won't accept anything other than having 'data' as a key which kind of defeats the purpose in my opinion.

The dictionary i want to send:

self.cornerDict = {
    1: None,
    2: None,
    3: None,
    4: None,
    5: None,
    6: None,
    7: None,
    8: None}

Would be nice if you could send nested dictionaries too, like this one:

self.pandaPose1 = {
    0: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}},
    1: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}},
    2: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}}, 
    3: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}}, 
    4: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}}, 
    5: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}},
    6: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}}, 
    7: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}},
    8: {'POS': None,
        'PROX': {'head': None, 'L_shoulder': None, 'R_shoulder': None, 'L_elbow': None, 'R_elbow': None, 'L_wrist': None, 'R_wrist': None}},
        }