Giter VIP home page Giter VIP logo

sparse_dynamic_calibration's Introduction

sparse_dynamic_calibration

This is a ROS package to calibrate a camera network consisting of not only static overlapping cameras but also dynamic and non-overlapping cameras. It bridges separated camera views using a dynamic support camera with visual odometry, and estimates all the static and dynamic camera poses based on pose graph optimization. It has an optional depth image-based refinement step for RGB-D cameras.

The calibration method itself is designed for general camera networks. But, it also provides some scripts to incorporate with OpenPTrack, an RGB-D camera network-based human-machine interaction framework.

This package has been tested on ROS melodic on Ubuntu 18.04

video

Installation

Apriltag

git clone https://github.com/AprilRobotics/apriltags.git
cd apriltags && sudo make install

Visual odometry

In this work, we use Direct Sparse Odometry to obtain the camera motion. However, you can replace it with any visual odometry algortihm. Only the requirement is that it publishes the estimated odometry to "/vodom (geometry_msgs::PoseStamped)" topic.

DSO

# (Recommended) Install Pangolin (https://github.com/stevenlovegrove/Pangolin)

git clone https://github.com/koide3/dso.git
mkdir dso/build && cd dso/build
cmake ..
make -j4

echo "export DSO_PATH=/path/to/dso" >> ~/.bashrc
soruce ~/.bashrc


cd catkin_ws/src
git clone https://github.com/koide3/dso_ros.git
cd dso_ros && git checkout catkin
cd ../.. && catkin_make

sparse_dynamic_calibration

cd catkin_ws/src
git clone https://github.com/koide3/sparse_dynamic_calibration.git
cd .. && catkin_make

Usage

Tag placement

Print out apriltag 36h11 family, and place the in the environment so that each camera can see at least one tag.

Detecting tags from static cameras

Edit "data/tags.yaml" to specify the tag size.

%YAML:1.0
default_tag_size: 0.160

Then, launch "generate_tag_camera_network_conf.launch" to detect tags from the static cameras. It automatically detects cameras by findind topics which match with a regex pattern (e.g., "(/kinect.*)/rgb/image"), and then extracts image data from topics of the detected cameras (/rgb/camera_info, /rgb/image, /depth_ir/points). You can change the pattern and topic names by editing the launch file.

roslaunch sparse_dynamic_calibration generate_tag_camera_network_conf.launch

Recording dynamic camera image stream

Record an image stream and visual odometry data using a dynamic camera.

# in case you use a pointgrey camera
roslaunch sparse_dynamic_calibration camera.launch

#otherwise, use any other camera node like usb_cam
roslaunch dso_ros dso.launch

Although the calibration method itself is an online method, we recommend to run the calibration on a rosbag for testing.

rosbag record -O test.bag -e "/camera/(camera_info|image_raw/compressed)" /vodom /points

Running calibration

rosparam set use_sim_time true
roslaunch sparse_dynamic_calibration calibration.launch
roscd sparse_dynamic_calibration/config
rviz -d rviz.rviz
rosrun image_transport republish compressed in:=/camera/image_raw raw out:=/camera/image_raw
rosbag play --clock test.bag

After finishing to play the rosbag, save the estimate poses:

rostopic pub /sparse_dynamic_calibration/save std_msgs/Empty

You should be able to see the calibrated camera poses in "data/tag_camera_poses.yaml".

Depth image-based refinement (optional)

rosparam set use_sim_time false
roslaunch sparse_dynamic_calibration refinement.launch

Refined camera poses will be saved to "data/tag_camera_poses_refined.yaml". The accumulated point clouds before/after the refinement will be saved to /tmp/(original|refined).pcd

Copying estimated posed to OpenPTrack (optional)

After calibrating the camera network, copy the estimated camera poses with:

rosrun sparse_dynamic_calibration copy_to_openptrack.py

This script reads the estimated camera poses in "data/tag_camera_poses.yaml" and writes them into "opt_calibration/launch/opt_calibration_results.launch" and "opt_calibration/conf/camera_poses.yaml". Then, distribute the calibration result to each PC:

# On master PC
roslaunch opt_calibration detection_initializer.launch
# On each distributed PC
roslaunch opt_calibration listener.launch

Example

Static camera imageset
Dynamic camera rosbag
(will be available soon)

Detecting tags

tar xzvf sparse_dynamic_example.tar.gz
cp -R sparse_dynamic_example/data catkin_ws/src/sparse_dynamic_calibration/

roslaunch sparse_dynamic_calibration generate_tag_camera_network_conf.launch read_from_file:=true

Running calibration

rosparam set use_sim_time true
rosrun image_transport republish compressed in:=/camera/image_raw raw out:=/camera/image_raw
roscd sparse_dynamic_calibration/config
rviz -d rviz.rviz
roslaunch sparse_dynamic_calibration calibration.launch
rosbag play --clock real_30.bag
rostopic pub /sparse_dynamic_calibration_node/save std_msgs/Empty

Refinement

rosparam set use_sim_time false
roscd sparse_dynamic_calibration/config
rviz -d rviz.rviz
roslaunch sparse_dynamic_calibration refinement.launch

Related work

Kenji Koide and Emanuele Menegatti, Sparse and Dynamic RGB-D Camera Network Calibration with Visual Odometry, IROS2019 (under review).

sparse_dynamic_calibration's People

Contributors

koide3 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.