The Movidius™ Neural Compute Stick (NCS) is a tiny fanless deep learning device that you can use to learn AI programming at the edge. NCS is powered by the same low power high performance Movidius™ Vision Processing Unit (VPU) that can be found in millions of smart security cameras, gesture controlled drones, industrial machine vision equipment, and more.

This project is a ROS wrapper for NC API of NCSDK, providing the following features:

• A ROS service for classifying objects in a static image file
• A ROS publisher for classifying objects in a video stream from a RGB camera
• Demo applications to show the capabilities of ROS service and publisher
• Support multiple CNN models

There are 2 active branches in this project:

• master - stable branch
  The latest version on it is v0.4.0 which supports NCSDK v1.10.00. master branch is only updated when every milestone release ready.
• devel - default branch
  This branch is updated from time to time and maintain the latest code on it. Each pull request should be submitted based on devel branch. We will merge patches to master branch on every milestone release.

• An x86_64 computer running Ubuntu 16.04
• ROS Kinetic
• Movidius Neural Compute Stick (NCS)
• Movidius Neural Compute (MvNC) SDK
• RGB Camera, e.g. RealSense ZR300, RealSense D400 Series or standard USB camera

• Install ROS Kinetic Desktop-Full (guide)
• Create a catkin workspace (guide)
• Install NCSDK v1.10.00 (github)
• NCSDK should be installed in /opt/movidius by default. Create a symbol link in /opt/movidius to the supported CNN network from NCSDK release package.

sudo ln -s <your-workspace>/ncsdk/examples /opt/movidius/examples

After that, make sure you can find graph data in /opt/movidius/examples/caffe or /opt/movidius/examples/tensorflow and image data in /opt/movidius/examples/data/images

• Install ROS package for different cameras as needed. e.g.

  1. Standard USB camera

  sudo apt-get install ros-kinetic-usb-cam

  2. RealSense ZR300 camera
  • Install RealSense camera package (guide). Refer to Realsense ROS WiKi for more information.
  3. RealSense D400 series camera
  • Install Intel® RealSense™ SDK 2.0 (guide). Refer here for more details about Intel® RealSense™ SDK 2.0.
  • Install Intel® RealSense™ ROS (guide)

  cd ~/catkin_ws/src
  git clone https://github.com/intel-ros/realsense.git
  cd  realsense
  git checkout 2.0.0
  cd ~/catkin_ws
  catkin_make

cd ~/catkin_ws/src
git clone https://github.com/intel/ros_intel_movidius_ncs.git
cd ros_intel_movidius_ncs
git checkout v0.4.0
cd ~/catkin_ws
catkin_make
catkin_make install
source install/setup.bash

Copy categories.txt from this project to NCSDK installation location.

cp ~/catkin_ws/src/ros_intel_movidius_ncs/data/categories.txt /opt/movidius/examples/data/ilsvrc12/

Launch video streaming nodelet. Refer here for more parameter configurations.

roslaunch movidius_ncs_launch ncs_usbcam.launch

Make sure you can get result from the topic of object classification.

rostopic echo /movidius_ncs_nodelet/classified_object

Launch image viewer to show the inference result.

roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/usb_cam/image_raw"

Launch video streaming nodelet. Refer here for more parameter configurations.

roslaunch movidius_ncs_launch ncs_realsense_zr300.launch

Make sure you can get result from the topic of object classification.

rostopic echo /movidius_ncs_nodelet/classified_object

Launch image viewer to show the inference result. You can launch it directly without setting camera_topic, as its default value is /camera/color/image_raw

roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/camera/color/image_raw"
#or
roslaunch movidius_ncs_launch ncs_stream_example.launch

Launch video streaming nodelet. Refer here for more parameter configurations.

roslaunch movidius_ncs_launch ncs_realsense_d400.launch

Make sure you can get result from the topic of object classification.

rostopic echo /movidius_ncs_nodelet/classified_object

Launch image viewer to show the inference result. You can launch it directly without setting camera_topic, as its default value is /camera/color/image_raw

roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/camera/color/image_raw"
#or
roslaunch movidius_ncs_launch ncs_stream_example.launch

Launch your preferred camera node.

#launch ROS master
roscore
#launch camera node in another console
rosrun <your-camera-pkg> <your-camera-node>

e.g.

#launch ROS master
roscore
#launch camera node in another console
rosrun usb_cam usb_cam_node

Launch NCS nodelet and assign input_topic with the topic url of your rgb camera.

roslaunch movidius_ncs_launch ncs_nocam.launch input_topic:=<your_rgb_camera_topic>

e.g.

roslaunch movidius_ncs_launch ncs_nocam.launch input_topic:="/usb_cam/image_raw"

Launch image viewer to show the inference result.

roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:=<your_rgb_camera_topic>

e.g.

roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/usb_cam/image_raw"

Launch object classification service. Refer here for more parameter configurations.

roslaunch movidius_ncs_launch ncs_image.launch

Run the example application with an absolute path of an image

rosrun movidius_ncs_example movidius_ncs_example_image <image_path_to_be_inferred>

e.g.

rosrun movidius_ncs_example movidius_ncs_example_image /opt/movidius/examples/data/images/cat.jpg

You can choose different CNN Models for inference through the argument of graph_file_path. e.g.
Using RealSense camera

#one console
roslaunch movidius_ncs_launch ncs_realsense_d400.launch graph_file_path:="/opt/movidius/examples/caffe/SqueezeNet/graph"
#another console
roslaunch movidius_ncs_launch ncs_stream_example.launch

Using standard USB camera

#one console
roslaunch movidius_ncs_launch ncs_usbcam.launch graph_file_path:="/opt/movidius/examples/caffe/AlexNet/graph"
#another console
roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/usb_cam/image_raw"

/movidius_ncs_nodelet/classified_object

/movidius_ncs_image/classify_object

• Only absolute path of image file supported in image inference demo
• Only test on RealSense ZR300 camera, RealSense D400 series camera and Microsoft HD-300 USB camera
• Current v0.4.0 supporting NCSDK v1.10.00 is on master branch. devel branch is the development branch for the next release.

• Support multiple NCS devices
• Support more CNN models
• Support latest NCSDK