gbr1 / ros_openvino Goto Github PK

View Code? Open in Web Editor NEW

57.0 5.0 13.0 30.51 MB

A ROS package to wrap openvino inference engine and get it working with Myriad and GPU

Home Page: https://gbr1.github.io

License: GNU Affero General Public License v3.0

CMake 27.74% C++ 72.26%

ros ros-kinetic openvino realsense realsense2 opencv roscpp gbr1 erwhi myriad

ros_openvino's Introduction

ros_openvino

A ROS package to wrap openvino inference engine and get it working with Myriad and Intel CPU/GPUs
The main topic is to accelerate robotics developing using ROS with OpenVINO.
This package is designed on async api of Intel OpenVINO and allows an easy setup for object detection.

Entire package is born to support Myriad X, so it works perfectly with:

Movidius Neural Compute Stick 2
Ai Core X
Movidius Neural Compute Stick
Ai Core

object detection node allows you to use a standard image topic and obtain a new topic with live result image. If you enable depth analysis you can add depth image topic to obtain 3d boxes, visible as markers in rviz.

Wiki

Wiki home

Setup

You need some steps to get ros_openvino working.
If you are working on a Myriad based device (NCS, NCS2, AI Core, AI Core X) follow this guide because is faster.

If you have already this kind of stuffs, please take care to modify steps for your case.

- Prerequisites

Ubuntu 16.04/18.04
OpenVINO 2018 R5 or newer (latest officially tested is 2019 R3)
ROS Kinetic Kame/Melodic Morenia

- Setup ros_openvino

Clone repo in your source folder, assuming catkin_ws as your ROS workspace:
cd ~/catkin_ws/src
git clone http://github.com/gbr1/ros_openvino.git
Compile!
cd ~/catkin_ws
catkin_make
catkin_make install

all installation steps and a case of study configuration are described in Setup environment wiki page.

Remember that OpenVINO enviroment variables, ROS enviroment variables and ROS workspace need to be sourced in bashrc.

- Tests

There are some demo launch files.

Webcam and GPU
you need to install usb_cam node:
sudo apt-get install ros-kinetic-usb-cam
then type:
roslaunch ros_openvino gpu_demo_webcam.launch
Webcam and Myriad
you need to install usb_cam node:
sudo apt-get install ros-kinetic-usb-cam
then type:
roslaunch ros_openvino myriad_demo_webcam.launch
Realsense D435/D435i/D415 and Myriad
you need to install realsense package, follow here step-by-step according your environment setup.
After that, just type:
roslaunch ros_openvino myriad_demo_realsense.launch

Nodes

object_detection, a node to detect objects in enviroment using a RGB or RGBD camera using MobileNet-SSD

Note

Tests are done on UP2 with Intel Atom, 4GB of RAM and 64GB eMMC and AI Core X. Results with a complete analysis with 4-5 object detected is about 30fps for rgb output and 16-18Hz for spatial calculation.
On GPU analysis drop down the frame rate.

If OpenVINO works on Ubuntu 18.04 this package should be compatible with ROS Melodic Morenia.

Known issues

CPU device doesn't work. In the future yes, it is in "todo" list;
GPU is worst than Myriad;
resize of input image is not availble at the moment. It is in "todo" list.

Future steps

add face and body tracking
improve depth analysis

This package is distribuited under Affero GPLv3.
Copyright (c) 2019 Giovanni di Dio Bruno

Models:

mobilenet-ssd.bin
mobilenet-ssd.xml

are created using OpenVINO model downloader and converter.
Original mobilenet-ssd model are under Intel OpenVINO licenses.

ros_openvino's People

Contributors

Stargazers

Watchers

Forkers

iotman stefantasy sezan92 buoyancy99 sunmaxwll wntun kigs-mx mrjohnrobert simiasher corsair-cxs nmoduga kingwingshome irwanmazlin

ros_openvino's Issues

Using nodelet for faster inference?

Hey, I'm having a difficult time understanding the working of OpenVino. If I may, I would like to requests a feature to make a nodelet version of the object_detection node. I'm building a robot that have many image processing task, so to maximize performance, I use nodelet to publish the image from camera for further processing. Is it possible to do? Or maybe give me some pointer to how it could be implemented? Thank you so much for this awesome package!

can't run with openvino r2.1

InferenceEngine API changed a lot.

Compilation Issue when cmake

Description
i tried to use your project because it has 3D detection features, but i ran into compilation issue. I would really appreciate if you could help if you encounter this problem before.

Step to reproduce

i clone the project in ros working space
then after clone i try to compile the project in my working space using catkin_make then i got an error

Versions
OPENVINO 2019_R3
realsense version 2.50

Unable to launch object_detection with Realsense D435

Thank you for this detailed tutorial. Your code is a good reference for me, but I have an error while running your demo myriad_demo_realsense.launch.
I setup the development environment according to your tutorial.
My environment:
I am using my own laptop with Ubuntu 16.04 ROS Kinetic
Realsense D435
RealSense ROS v2.2.16
LibRealSense v2.37.0
OpenVINO 2019 R3
depth stream width: 640, height: 480, fps: 30, Format: Z16
color stream width: 640, height: 480, fps: 30, Format: RGB8

Issue:

[object_detection-4] process has died [pid 4751, exit code 255, cmd /home/agv/catkin_ws/devel/lib/ros_openvino/object_detection /object_detection/input_image:=/camera/color/image_raw /object_detection/input_depth:=/camera/aligned_depth_to_color/image_raw /object_detection/camera_info:=/camera/aligned_depth_to_color/camera_info __name:=object_detection __log:=/home/agv/.ros/log/ad10953a-eb81-11ea-896d-acd1b8383ba1/object_detection-4.log].
log file: /home/agv/.ros/log/ad10953a-eb81-11ea-896d-acd1b8383ba1/object_detection-4*.log

Pointcloud is visible in rviz

I tried to solve the error according to the other two issues, but to no avail.

Best regards,
Qixiang

unable to include AlexNet

I tried to change the CNN that is being used for object recognition.
I am using the Intel RealSense Camera and the VPU NCS2 on Ubuntu 16.04 (LTS).
The original, unmodified package works fine.

Steps taken:

Using the unmodified ros_openvino package
Download the bvlc_alexnet from the Intel-Website
Using the Model Optimizer with the given parameters (here) and generate the IR
Place the IR in the models - folder
Change the file params so that they fit to the folder I created
Starting the modified package

Issue:

Terminal output as follows:

[object_detection-4] process has died [pid 20105, exit code 255, cmd /home/USERNAME/catkin_ws/devel/lib/ros_openvino/object_detection /object_detection/input_image:=/camera/color/image_raw /object_detection/input_depth:=/camera/aligned_depth_to_color/image_raw /object_detection/camera_info:=/camera/aligned_depth_to_color/camera_info __name:=object_detection __log:=/home/USERNAME/.ros/log/da83b362-b530-11ea-87a9-e454e8a1df6c/object_detection-4.log].

No output image including bounding boxes visible in RVIZ.
But: depth image is visible (no boxes) in RVIZ.

All the best,
Robert

AttributeError: 'MultiDiGraph' object has no attribute 'node'

I have followed your instruction to install everything.
But when I run squeezenet demo as follows:

sudo ./demo_squeezenet_download_convert_run.sh -d MYRIAD

I got AttributeError: 'MultiDiGraph' object has no attribute 'node' when optimizing the model:

###################################################

Convert a model with Model Optimizer

Run python3 /opt/intel//computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo.py --input_model /home/shan/openvino_models/models/FP16/classification/squeezenet/1.1/caffe/squeezenet1.1.caffemodel /home/shan/openvino_models/ir/FP16//classification/squeezenet/1.1/caffe --data_type FP16

Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/home/shan/openvino_models/models/FP16/classification/squeezenet/1.1/caffe/squeezenet1.1.caffemodel
	- Path for generated IR: 	/home/shan/openvino_models/ir/FP16//classification/squeezenet/1.1/caffe
	- IR output name: 	squeezenet1.1
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	Not specified, inherited from the model
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP16
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	False
	- Reverse input channels: 	False
Caffe specific parameters:
	- Enable resnet optimization: 	True
	- Path to the Input prototxt: 	/home/shan/openvino_models/models/FP16/classification/squeezenet/1.1/caffe/squeezenet1.1.prototxt
	- Path to CustomLayersMapping.xml: 	Default
	- Path to a mean file: 	Not specified
	- Offsets for a mean file: 	Not specified
Model Optimizer version: 	1.5.12.49d067a0
[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  'MultiDiGraph' object has no attribute 'node'
[ ERROR ]  Traceback (most recent call last):
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/main.py", line 325, in main
    return driver(argv)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/main.py", line 279, in driver
    custom_layers_mapping_path=custom_layers_mapping_path)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/pipeline/caffe.py", line 74, in driver
    graph, original_shapes = loader.caffe_pb_to_nx(proto, model)
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/front/caffe/loader.py", line 314, in caffe_pb_to_nx
    if len(node.out_nodes()) == 0:
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/graph/graph.py", line 313, in out_nodes
    assert self.has('kind')
  File "/opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/mo/graph/graph.py", line 273, in has
    return k in self.graph.node[self.node]
AttributeError: 'MultiDiGraph' object has no attribute 'node'

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------
Error on or near line 199; exiting with status 1

Unable to use YOLOv3 model

I'm trying to change the model used from MobileNet SSD to YOLOv3.
I am using the Intel RealSense Camera D435 on Ubuntu 16.04 (LTS).

Steps taken:

Download tensorflow YOLOv3 here
Use the Model Optimizer to convert the model to IR representation (following this guide)
Place the IR in the models - folder
Change the launch file to use the models

Issue:

Terminal output as follows:

[object_detection-4] process has died [pid 20105, exit code 255, cmd /home/USERNAME/catkin_ws/devel/lib/ros_openvino/object_detection /object_detection/input_image:=/camera/color/image_raw /object_detection/input_depth:=/camera/aligned_depth_to_color/image_raw /object_detection/camera_info:=/camera/aligned_depth_to_color/camera_info __name:=object_detection __log:=/home/USERNAME/.ros/log/da83b362-b530-11ea-87a9-e454e8a1df6c/object_detection-4.log].

No output image is visible with boxes.

Best regards,
Thomas

cant compile after following instructions

I have already used Openvino with Intel Nural Computer Stick and tested it with some samples. But when i tried to build your package, it has a trouble in the link part of the object detection node

...
...
...
[ 96%] Built target filter_base
/opt/intel/openvino_2019.1.094/deployment_tools/inference_engine/lib/intel64/libinference_engine.so: undefined reference to `tbb::interface7::internal::task_arena_base::internal_max_concurrency(tbb::interface7::task_arena const*)'
collect2: error: ld returned 1 exit status
ros_openvino/CMakeFiles/object_detection.dir/build.make:130: recipe for target '/home/bot/catkin_ws/devel/lib/ros_openvino/object_detection' failed
make[2]: *** [/home/bot/catkin_ws/devel/lib/ros_openvino/object_detection] Error 1
CMakeFiles/Makefile2:6936: recipe for target 'ros_openvino/CMakeFiles/object_detection.dir/all' failed
make[1]: *** [ros_openvino/CMakeFiles/object_detection.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 96%] Built target urg_node_driver
[ 96%] Built target navsat_transform_node
[ 96%] Built target ukf
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

gbr1 / ros_openvino Goto Github PK

ros_openvino's Introduction

ros_openvino

Wiki

Setup

- Prerequisites

- Setup ros_openvino

- Tests

Nodes

Note

Known issues

Future steps

ros_openvino's People

Contributors

Stargazers

Watchers

Forkers

ros_openvino's Issues

Recommend Projects

Recommend Topics

Recommend Org