ethz-asl / kalibr Goto Github PK

Hey,

I just pulled down the latest version (added 3 commits) and I recompiled. After that I resourced the workspace and tried to run cam+imu calibration again, but I got this error:

Traceback (most recent call last):
  File "/home/xxx/kalibr_workspace/devel/bin/kalibr_calibrate_imu_camera", line 6, in <module>
    exec(fh.read())
  File "<string>", line 5, in <module>
  File "/home/xxx/kalibr_workspace/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/__init__.py", line 4, in <module>
    import IccSensors as sens
  File "/home/xxx/kalibr_workspace/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 16, in <module>
    import scipy.optimize
ImportError: No module named scipy.optimize

This is how I solved:

sudo apt-get install gfortran
sudo -H pip install scipy

These dependencies should be added to the installation guide IMHO.

When performing stereo calibration, using for instance aprilgrid pattern, is it possible to make intrinsic parameters fixed and only search for the baseline and convergence for the given intrinsic parameters?

Or, for example, force the algorithm to keep fx and fy identical?

I'm trying to run the Kalibr on ROS Jade. Compiled from source (used catkin_tools) and yes, I made sure that the workspace is sourced.

kabir@Kabir-MacBook:~/catkin_ws$ rosrun kalibr kalibr_calibrate_cameras --bag stereo-artemis.bag --topics /stereo/cam0/image_raw /stereo/cam1/image_raw --models pinhole-radtan pinhole-radtan --target checkerboard.yaml --show-extraction
importing libraries
Traceback (most recent call last):
  File "/home/kabir/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_cameras", line 10, in <module>
    import aslam_cv_backend as acvb
ImportError: No module named aslam_cv_backend

Help would be greatly appreciated.

I was following this documentation to use kalibr_camera_validator but when I ran it I received the error message:

  File "/home/andrija/ros_packages/devel/bin/kalibr_camera_validator", line 6, in <module>
    exec(fh.read())
  File "<string>", line 478, in <module>
  File "<string>", line 82, in __init__
  File "/home/andrija/ros_packages/src/kalibr-master/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 137, in func
    args[0].raiseError("Field {0} missing in file: {1}".format(e, args[0].yamlFile))
  File "/home/andrija/ros_packages/src/kalibr-master/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 183, in raiseError
    raise RuntimeError( "{0}{1}".format(header, message) )
RuntimeError: [CameraChainParameters Reader]: Field 'cam_overlaps' missing in file: /home/andrija/Documents/calibration_validation/camchain.yaml

How to solve this? I search github wiki page for cam_overlaps but I find nothing.

EDIT:

I saw someone somewere putting an array of one number in cam_overlaps. So, I did the same...

My camchain.yaml looks like this:

cam0:
    camera_model: pinhole
    intrinsics: [720.0, 720.0, 640.0, 350.0]
    distortion_model: radtan
    distortion_coeffs: [0.0, 0.0, 0.0, 0.0]
    cam_overlaps: [0]
    rostopic: /stereo/left/image
    resolution: [1280, 720]
cam1:
    camera_model: pinhole
    intrinsics: [720.0, 720.0, 640.0, 350.0]
    distortion_model: radtan
    distortion_coeffs: [0.0, 0.0, 0.0, 0.0]
    cam_overlaps: [1]
    T_cn_cm1:
    - [ 1, 0, 0, 120]
    - [ 0, 1, 0, 0]
    - [ 0, 0, 1, 0]
    - [ 0, 0, 0, 1]
    rostopic: /stereo/right/image
    resolution: [1280, 720]

Using this camchain.yaml I can run kalibr_camera_validator and this is what I get:

andrija@andrija-T500:~$ kalibr_camera_validator --cam ~/Documents/calibration_validation/camchain.yaml --target ~/Documents/calibration_validation/target.yaml 
importing libraries
/usr/lib/python2.7/dist-packages/pkg_resources.py:1031: UserWarning: /home/andrija/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Consider a more secure location (set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).
    warnings.warn(msg, UserWarning)
Initializing calibration target:
    Type: aprilgrid
    Tags: 
      Rows: 6
      Cols: 6
      Size: 0.0375 [m]
      Spacing 0.0111 [m]
initializing camera geometry
Camera /stereo/left/image:
    Camera model: pinhole
    Focal length: [720.0, 720.0]
    Principal point: [640.0, 350.0]
    Distortion model: radtan
    Distortion coefficients: [0.0, 0.0, 0.0, 0.0]
initializing camera geometry
Camera /stereo/right/image:
    Camera model: pinhole
    Focal length: [720.0, 720.0]
    Principal point: [640.0, 350.0]
    Distortion model: radtan
    Distortion coefficients: [0.0, 0.0, 0.0, 0.0]
/home/andrija/ros_packages/devel/bin/kalibr_camera_validator:326: RuntimeWarning: invalid value encountered in divide
/home/andrija/ros_packages/devel/bin/kalibr_camera_validator:327: RuntimeWarning: invalid value encountered in divide
/home/andrija/ros_packages/devel/bin/kalibr_camera_validator:328: RuntimeWarning: invalid value encountered in divide

And it stays like this, presumably waiting for messages to start arriving through specified channels.
When images do start arriving, an error occurs:

[ERROR] [WallTime: 1447415308.430469] bad callback: <bound method Subscriber.callback of <message_filters.Subscriber instance at 0x7f968fd71b48>>
Traceback (most recent call last):
    File "/opt/ros/indigo/lib/python2.7/dist-packages/rospy/topics.py", line 711, in _invoke_callback
      cb(msg)
    File "/opt/ros/indigo/lib/python2.7/dist-packages/message_filters/__init__.py", line 74, in callback
      self.signalMessage(msg)
    File "/opt/ros/indigo/lib/python2.7/dist-packages/message_filters/__init__.py", line 56, in signalMessage
      cb(*(msg + args))
    File "/opt/ros/indigo/lib/python2.7/dist-packages/message_filters/__init__.py", line 195, in add
      self.signalMessage(*msgs)
    File "/opt/ros/indigo/lib/python2.7/dist-packages/message_filters/__init__.py", line 56, in signalMessage
      cb(*(msg + args))
    File "<string>", line 134, in synchronizedCallback
TypeError: Conversion is only valid for arrays with 1 or 2 dimensions. Argument has 3 dimensions

It seems like images are not released internally s.t. the executable consumes vast amounts of memory until the system is running out. Smaller datasets work.

Hi,
I get a matrics in the result,but it seems unreasonable.Is it right that the last column [0.00448395,-0.00498745,-0.0022667]-1 represents the translation matrics[x,y,z]-1?I think the value is too small,the real distance of IMU and Camera is about [0.07,0.035,0.02] by measurement.but other values are ok!
T_ci: (imu to cam0): [m]
[[ 0.97904792 0.13214156 -0.15493151 0.00448395]
[ 0.17046857 -0.94803377 0.26864929 -0.00498745]
[-0.11138056 -0.28943148 -0.95069647 -0.0022667 ]
[ 0. 0. 0. 1. ]]
I am looking for your advises! Thanks!

See https://groups.google.com/forum/#!topic/kalibr-users/wYt0uhedT4Y

Hi,

I followed the installation instructions, but when compiling Kalibr in the final step, I get this error:

Errors     << numpy_eigen:make /home/ale/kalibr_workspace/logs/numpy_eigen/build.make.000.log
In file included from /usr/include/python2.7/numpy/ndarraytypes.h:1761:0,
                 from /usr/include/python2.7/numpy/ndarrayobject.h:17,
                 from /usr/include/python2.7/numpy/arrayobject.h:4,
                 from /home/xxx/kalibr_workspace/src/Kalibr/Schweizer-Messer/numpy_eigen/include/numpy_eigen/NumpyEigenConverter.hpp:20,
                 from /home/xxx/kalibr_workspace/src/Kalibr/Schweizer-Messer/numpy_eigen/src/autogen_module/numpy_eigen_export_module.cpp:2:
/usr/include/python2.7/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/numpy_eigen.dir/src/autogen_module/numpy_eigen_export_module.cpp.o] Error 4
make[1]: *** [CMakeFiles/numpy_eigen.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/numpy_eigen_test.dir/src/autogen_test_module/test_1_1_int.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/numpy_eigen_test.dir/all] Error 2
make: *** [all] Error 2

I checked that Eigen is installed fine, and it is. The only thing I did differently is running sudo rosdep init instead of rosdep init (sudo is needed and also reported in official ROS instructions).

Any help?

Hi,

What is the AprilTag family used in the default calibration target PDFs available for download?

Kabir

Hi,

The current kalibr_camera_validator uses the strict time synchronizer included in message_filters. Since Indigo, message_filters for Python includes ApproximateTimeSynchronizer which will enlarge the usage of kalibr_camera_validator to a larger community of users.

The modification seems simple. I have replaced, in line 105,
sync_sub = message_filters.TimeSynchronizer([val.image_sub for val in self.monovalidators],1)
by
sync_sub = message_filters.ApproximateTimeSynchronizer([val.image_sub for val in self.monovalidators],2,0.01)

Best regards.

Martial Sanfourche (ONERA, France)

I am compiling Kalibr from source using ROS Indigo. I had to add cmake_modules to package.xml and CMakeLists.txt as described in https://github.com/ros/cmake_modules/blob/0.3-devel/README.md , whereever Eigen was required. Catkin configuration runs so far for Kalibr, but fails now for PTAM:

-- +++ processing catkin package: 'ptam'
-- ==> add_subdirectory(ethzasl_ptam/ptam)
-- Using these message generators: gencpp;genlisp;genpy
CMake Error at ethzasl_ptam/ptam/CMakeLists.txt:116 (set_target_properties):
set_target_properties called with incorrect number of arguments.

PTAM used to compile properly before installing Kalibr.
Has anybody an idea what the problem might be?
Regards,
Stefan

Hi,
I'm trying to run the calibration of the pixhawk and ueye camera setup with the cde version, but run into the following error: Failed to obtain orientation prior!

The example dataset runs fine though. Any guidance on where to look for the problem would be very appreciated!

./kalibr_calibrate_imu_camera --bag px4-dataset/imu-ueye.bag --cam px4-dataset/camchain.yaml 
--imu px4-dataset/px4_imu.yaml --target px4-dataset/april_6x6_A3.yaml
importing libraries
Initializing IMUs:
  Update rate: 207.0
  Accelerometer:
    Noise density: 0.013 
    Noise density (discrete): 0.187037429409 
    Random walk: 0.000108
  Gyroscope:
    Noise density: 0.00065
    Noise density (discrete): 0.00935187147046 
    Random walk: 2.12e-06
Initializing imu rosbag dataset reader:
    Dataset:          /home/artyom/kalibr-cde/px4-dataset/imu-ueye.bag
    Topic:            /mavros/imu/data
    Number of messages: 18914
Reading IMU data (/mavros/imu/data)
  Read 18914 imu readings over 90.5 seconds                          
Initializing calibration target:
  Type: aprilgrid
  Tags: 
    Rows: 6
    Cols: 6
    Size: 0.035 [m]
    Spacing 0.01225 [m]
Initializing camera chain:
Camera chain - cam0:
  Camera model: pinhole
  Focal length: [416.754436, 416.592321]
  Principal point: [376.0, 240.0]
  Distortion model: equidistant
  Distortion coefficients: [-0.273119, 0.061682, -0.000351, -0.00108]
  baseline: no data available
Initializing camera rosbag dataset reader:
    Dataset:          /home/artyom/kalibr-cde/px4-dataset/imu-ueye.bag
    Topic:            /camera/image_raw
    Number of images: 631
Extracting calibration target corners
  Extracted corners for 631 images (of 631 images)                              

Building the problem
    Spline order: 6
    Pose knots per second: 70
    Do pose motion regularization: False
        xddot translation variance: 1000000.000000
        xddot rotation variance: 100000.000000
    Bias knots per second: 50
    Do bias motion regularization: True
    Blake-Zisserman on reprojection errors -1
    Acceleration Huber width (m/s^2): -1.000000
    Gyroscope Huber width (rad/s): -1.000000
    Do time calibration: False
    Max iterations: 30
    Time offset padding: 0.020000

Estimating imu-camera rotation prior

Initializing a pose spline with 21011 knots (100.000000 knots per second over 210.105000 seconds)
[FATAL] [1444575368.082328]: Failed to obtain orientation prior!

Hi!
When I try to run kalibr_calibrate_cameras ...
I get the error
ImportError: No module named apport.fileutils
I use the cde binary on ubuntu 14.04.
Unfortunatelly I do not know anything about cde. What can I do? Ubuntu has apport installed.

The Stacktrace is
importing libraries Traceback (most recent call last): File "/var/kalibr-build/devel/bin/kalibr_calibrate_cameras", line 5, in <module> exec(fh.read()) File "<string>", line 444, in <module> File "<string>", line 149, in main File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 397, in __init__ ParametersBase.__init__(self, yamlFile, "CalibrationTargetConfig", createYaml) File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 135, in __init__ self.data = self.readYaml() File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 146, in readYaml self.raiseError( "Could not read configuration from {0}".format(self.yamlFile) ) File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 169, in raiseError raise RuntimeError( "{0}{1}".format(header, message) ) RuntimeError: [CalibrationTargetConfig Reader]: Could not read configuration from /mnt/256GBSDCard/schachbrett_kalibr.yamls Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes ImportError: No module named apport.fileutils

The idea would be to get intrinsic parameters from camera_info topic. This would allow using monocular calibration performed beforehand, which has already good low reprojection error.

Building with catkin tools (https://github.com/catkin/catkin_tools) instead of catkin_make gives "missing include" errors.

[aslam_cameras] ==> '/home/den2pal/work/uvm/ros_ws/build/aslam_cameras/build_env.sh /usr/bin/make -j8 -l8' in '/home/den2pal/work/uvm/ros_ws/build/aslam_cameras'
[ 10%] [ 20%] [ 20%] [ 20%] [ 40%] [ 40%] [ 40%] [ 40%] Building CXX object CMakeFiles/aslam_cameras.dir/src/GridCalibrationTargetCheckerboard.cpp.o
Building CXX object CMakeFiles/aslam_cameras.dir/src/GridCalibrationTargetCirclegrid.cpp.o
Building CXX object CMakeFiles/aslam_cameras.dir/src/GridCalibrationTargetObservation.cpp.o
Building CXX object CMakeFiles/aslam_cameras.dir/src/CameraGeometryBase.cpp.o
Building CXX object CMakeFiles/aslam_cameras.dir/src/KeypointBase.cpp.o
Building CXX object CMakeFiles/aslam_cameras.dir/src/GridDetector.cpp.o
Building CXX object CMakeFiles/aslam_cameras.dir/src/FrameBase.cpp.o
Building CXX object CMakeFiles/aslam_cameras.dir/src/Triangulation.cpp.o
In file included from /home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/include/aslam/FrameBase.hpp:6:0,
                 from /home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/src/FrameBase.cpp:1:
/home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/include/aslam/KeypointBase.hpp:10:45: fatal error: sm/kinematics/UncertainVector.hpp: No such file or directory
compilation terminated.
In file included from /home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/src/KeypointBase.cpp:1:0:
/home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/include/aslam/KeypointBase.hpp:10:45: fatal error: sm/kinematics/UncertainVector.hpp: No such file or directory
compilation terminated.
In file included from /home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/src/GridCalibrationTargetObservation.cpp:3:0:
/home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/include/aslam/cameras/GridCalibrationTargetObservation.hpp:12:44: fatal error: sm/kinematics/Transformation.hpp: No such file or directory
compilation terminated.
In file included from /home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/include/aslam/cameras.hpp:5:0,
                 from /home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/src/CameraGeometryBase.cpp:2:
/home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/include/aslam/cameras/CameraGeometry.hpp:6:55: fatal error: sm/kinematics/UncertainHomogeneousPoint.hpp: No such file or directory
compilation terminated.
In file included from /home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/include/aslam/cameras/GridDetector.hpp:12:0,
                 from /home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/src/GridDetector.cpp:9:
/home/den2pal/work/uvm/ros_ws/src/Kalibr/aslam_cv/aslam_cameras/include/aslam/cameras/GridCalibrationTargetObservation.hpp:12:44: fatal error: sm/kinematics/Transformation.hpp: No such file or directory
compilation terminated.
[ 45%] Building CXX object CMakeFiles/aslam_cameras.dir/src/DescriptorBase.cpp.o
make[2]: *** [CMakeFiles/aslam_cameras.dir/src/GridCalibrationTargetObservation.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [CMakeFiles/aslam_cameras.dir/src/KeypointBase.cpp.o] Error 1
make[2]: *** [CMakeFiles/aslam_cameras.dir/src/FrameBase.cpp.o] Error 1
make[2]: *** [CMakeFiles/aslam_cameras.dir/src/CameraGeometryBase.cpp.o] Error 1
make[2]: *** [CMakeFiles/aslam_cameras.dir/src/GridDetector.cpp.o] Error 1
make[1]: *** [CMakeFiles/aslam_cameras.dir/all] Error 2
make: *** [all] Error 2
[aslam_cameras] <== '/home/den2pal/work/uvm/ros_ws/build/aslam_cameras/build_env.sh /usr/bin/make -j8 -l8' failed with return code '2'

And then later also for other packages such as aslam_backend... Something with the way catkin_simple works, but I am a bit at loss what the correct fix is.

A workaround e.g. for the first error in aslam_cameras is adding a manual include_directories(${catkin_INCLUDE_DIRS}, but I was under the impression that catkin simple is supposed to do that under-the-hood.

Hi,
I've tested 5 different dataset acquired today, all failed, but not the sample dataset nor the ones I tested yesterday. Is it caused by the dataset and how to avoid it?

The Jacobian matrix is 194404 x 25179
[0.0]: J: 1.45862e+07
[1]: J: 2.21318e+06, dJ: 1.2373e+07, deltaX: 0.991163, LM - lambda:100 mu:2
[2]: J: 33328.9, dJ: 2.17985e+06, deltaX: 0.521266, LM - lambda:33.3333 mu:2
[3]: J: 28162.1, dJ: 5166.75, deltaX: 0.0966487, LM - lambda:11.1111 mu:2
[4]: J: 13295.2, dJ: 14866.9, deltaX: 0.0136785, LM - lambda:12.7412 mu:2
Exception in thread block: Exception in thread block: [aslam::Exception] /home/louis/catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [15106.3 <= 15106.3 < 15106.4]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
[aslam::Exception] /home/louis/catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [15118.2 <= 15118.2 < 15118.2]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
Exception in thread block: [aslam::Exception] /home/louis/catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [15094.8 <= 15094.8 < 15094.9]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
[ERROR] [1454555378.858709]: Optimization failed!
Traceback (most recent call last):
  File "/home/louis/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 206, in <module>
    main()
  File "/home/louis/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 182, in main
    iCal.optimize(maxIterations=parsed.max_iter, recoverCov=parsed.recover_cov)
  File "/home/louis/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 183, in optimize
    raise RuntimeError("Optimization failed!")
RuntimeError: Optimization failed!

I'm following https://github.com/ethz-asl/kalibr/wiki/IMU-Noise-Model

How long should I sample to be able to have a reliable base to calculate the Allan deviation needed to extract the noise parameters needed by Kalibr?

Hi, could you point me to a resource/paper explaining the equidistant distortion model? It seems to me the paper reference on [1] is not discussing that.

Am I right to assume that the standard ROS tools (image_proc) do not support that distortion model, but rather only the pinhole camera model + radtan distorition model (called plumb_bob in CameraInfo) [2,3]?

[1] https://github.com/ethz-asl/kalibr/wiki/supported-models
[2] http://wiki.ros.org/image_pipeline/CameraInfo
[3] http://docs.ros.org/api/sensor_msgs/html/msg/CameraInfo.html

Hello,

Once again, I am not sure, but comparing AccelerometerError.cpp and GyroscopeError.cpp under kalibr/aslam_offline_calibration/kalibr/src, why do you pre-multiply by C_b_w in the first file, while you don't in the 2nd file?

Using RGB images creates the following error message:-

[ERROR] [WallTime: 1404422763.895539] bad callback: <bound method Subscriber.callback of <message_filters.Subscriber instance at 0x3f0c680>>
Traceback (most recent call last):
  File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/topics.py", line 682, in _invoke_callback
    cb(msg)
  File "/opt/ros/hydro/lib/python2.7/dist-packages/message_filters/__init__.py", line 73, in callback
    self.signalMessage(msg)
  File "/opt/ros/hydro/lib/python2.7/dist-packages/message_filters/__init__.py", line 55, in signalMessage
    cb(*(msg + args))
  File "/opt/ros/hydro/lib/python2.7/dist-packages/message_filters/__init__.py", line 142, in add
    self.signalMessage(*msgs)
  File "/opt/ros/hydro/lib/python2.7/dist-packages/message_filters/__init__.py", line 55, in signalMessage
    cb(*(msg + args))
  File "<string>", line 134, in synchronizedCallback
TypeError: Conversion is only valid for arrays with 1 or 2 dimensions. Argument has 3 dimensions

Simple fix by reading in images as grayscale ("mono8") format using cv_bridge.

kalibr / aslam_offline_calibration / kalibr / python / kalibr_camera_calibration / CameraCalibrator.py
in line 97
selfgrid should be self.grid

Not really an issue but:
Is there a way to convert the intrinsics and distortion coefficients to pinhole model after calibration?

The problem is this: I need the omni model for the calibration to work with 8 (4 stereo in all directions) cameras (small overlap over a 90 deg corner). But afterwards I want to triangulate in the centre of the images (which is more or less pinhole) and for that I need the pinhole model. I want to avoid calibrating it twice (first all 8 with omni and then stereo with pinhole).

I had a look at the paper "A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses" but could find anything helpful...

Hi,

I’m trying to use Kalibr to calibrate a monocular visual-inertial setup consisting of a 1280x1024 camera with Asctec autopilot's IMU, but so far unsuccessfully. I was wondering if you could provide a few pointers about good conditions for IMU-cam calibration. Any help would be greatly appreciated.

The intrinsics of the camera were calibrated with Kalibr with equidistant distortion model, which I read were recommended here: #31
I ran the calibration at 4 Hz with the camera static and moving an April grid in front of it. The focus was adjusted using a Siemens star. The static sequence is 3-minute long and I tried to vary the grid poses significantly. The reprojection errors are below 0.2 pixels (see intrinsics calibration report here: results-cam-static.txt ), and results with the calibration validator on the live stream also look correct. So I assume the intrinsics calibration is good enough. Moving on to IMU-cam calibration, running at 20 Hz on the cam and 200 Hz on the IMU, I get the orientation prior fail error like in the issue in the link above and here: http://pkok.github.io/2015/02/17/
The exposure was fixed at 3 ms under good and even illumination to prevent motion blur (see screenshot below), while still detecting the target consistently.

I have tried to follow everything as recommended but I must be missing something. From what I read in previous issue, it seems that this error has been solved by using a larger pattern, while I’m currently using an A4 format (2cm april tags). I also read that you suggested the cam-imu synchronisation as a possible source of error. I’m not so familiar with ROS synchronisation but I know we’ve run successfully other visual-inertial code using the same ROS sensor packages.

In advance, thanks.

hello,
I am currently trying to use kalibr,I choose "building from source" when installing the toolbox,
but the project building failed with the error: virtual memory exhausted.
I find the reason is " /Kalibr/aslam_cv/aslam_cv_backend_python/src/module.cpp"
when I comment module.cpp ,building will success.
but i need build module.cpp,otherwise I can't use tools .

running catkin_make yields the following error:

/home/y22ma/ros/hydro_catkin/src/kalibr/aslam_incremental_calibration/incremental_calibration/src/algorithms/marginalize.cpp: In function ‘Eigen::MatrixXd aslam::calibration::marginalJacobian(cholmod_sparse_, cholmod_sparse_, cholmod_common_)’:
/home/y22ma/ros/hydro_catkin/src/kalibr/aslam_incremental_calibration/incremental_calibration/src/algorithms/marginalize.cpp:74:45: error: invalid conversion from ‘std::ptrdiff_t_ {aka int_}’ to ‘long int_’ [-fpermissive]
/home/y22ma/ros/hydro_catkin/devel/share/suitesparse/cmake/../../../include/suitesparse/cholmod_matrixops.h:194:17: error: initializing argument 4 of ‘cholmod_sparse* cholmod_l_submatrix(cholmod_sparse_, long int_, long int, long int_, long int, int, int, cholmod_common_)’ [-fpermissive]
/home/y22ma/ros/hydro_catkin/src/kalibr/aslam_incremental_calibration/incremental_calibration/src/algorithms/marginalize.cpp: In function ‘double aslam::calibration::marginalize(const aslam::backend::CompressedColumnMatrix&, std::size_t, Eigen::MatrixXd&, Eigen::MatrixXd&, Eigen::MatrixXd&, Eigen::MatrixXd&, Eigen::MatrixXd&, double, double)’:
/home/y22ma/ros/hydro_catkin/src/kalibr/aslam_incremental_calibration/incremental_calibration/src/algorithms/marginalize.cpp:156:20: error: invalid conversion from ‘std::ptrdiff_t* {aka int_}’ to ‘long int_’ [-fpermissive]
/home/y22ma/ros/hydro_catkin/devel/share/suitesparse/cmake/../../../include/suitesparse/cholmod_matrixops.h:194:17: error: initializing argument 4 of ‘cholmod_sparse* cholmod_l_submatrix(cholmod_sparse_, long int_, long int, long int_, long int, int, int, cholmod_common_)’ [-fpermissive]
/home/y22ma/ros/hydro_catkin/src/kalibr/aslam_incremental_calibration/incremental_calibration/src/algorithms/marginalize.cpp:170:36: error: invalid conversion from ‘std::ptrdiff_t* {aka int_}’ to ‘long int_’ [-fpermissive]
/home/y22ma/ros/hydro_catkin/devel/share/suitesparse/cmake/../../../include/suitesparse/cholmod_matrixops.h:194:17: error: initializing argument 4 of ‘cholmod_sparse* cholmod_l_submatrix(cholmod_sparse_, long int_, long int, long int_, long int, int, int, cholmod_common_)’ [-fpermissive]

Hello, I am following the steps at https://github.com/ethz-asl/kalibr/wiki/camera-imu-calibration
I have successfully installed kalibr from source and also sourced it in the bash file. When I try to run the dataset example using the following command
kalibr_calibrate_imu_camera --target april_6x6.yaml --cam camchain.yaml --imu imu_adis16448.yaml --bag dynamic.bag --bag-from-to 5 45

It says " kalibr_calibrate_imu_camera: command not found "
Can anyone tell me what am I doing wrong?

Thanks

The wiki page describing the YAML formats describes all required fields for the YAML files except for rostopic for the IMU configuration.

During a run of rosrun kalibr kalibr_calibrate_imu_camera --time-calibration --bag some.bag --cam cam_config.yaml --imu imu_config.yaml --target target_config.yaml I get the output:

importing libraries
Initializing IMUs:
  Update rate: 100
  Accelerometer:
    Noise density: 0.004 
    Noise density (discrete): 0.04 
    Random walk: 0.02
  Gyroscope:
    Noise density: 0.1
    Noise density (discrete): 1.0 
    Random walk: 1.0
Traceback (most recent call last):
  File "/home/pkok/ros/repos/kalibr/src/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 206, in <module>
    main()
  File "/home/pkok/ros/repos/kalibr/src/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 138, in main
    imus.append( sens.IccImu(imuConfig, parsed) )
  File "/home/pkok/ros/repos/kalibr/src/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 511, in __init__
    self.dataset = initImuBagDataset(parsed.bagfile[0], imuConfig.getRosTopic(), parsed.bag_from_to)
  File "/home/pkok/ros/repos/kalibr/src/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 123, in func
    args[0].raiseError("Field {0} missing in file: {1}".format(e, args[0].yamlFile))
  File "/home/pkok/ros/repos/kalibr/src/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 169, in raiseError
    raise RuntimeError( "{0}{1}".format(header, message) )
RuntimeError: [ImuConfig Reader]: Field 'rostopic' missing in file: repos/kalibr/config/imu/MTx.yaml

Adding a field rostopic in imu_config.yaml which refers to a topic with message type sensor_msgs/Imu resolves this error.

Suggested solution: Add some documentation about this field in Yaml formats.

Hello,

I am using camera calibration and further camera-imu calibration on 3 different smartphones. However, only one gives me the right position of imu vs. camera. The other two say the IMU is more than 15cm away from camera in x and y axes and more than 3 cm in z axis. I am not sure why is this not working only on the two of them (iphone 5 and nexus 5). Do you have some suggestions?

Thank you

Hi,
Nice work of this camera imu calibration tool. It's easy to use, thanks guys.
I managed to get several calibration results, but I'm confused to choose one of them, since the transformation matrices are quite different.
There is something wierd that the translation is too large for a mobile phone, like this
T_ci: (imu to cam0): [m]
[[-0.01867876 -0.99980259 -0.00677455 0.09131875]
[-0.99980497 0.01872141 -0.00628717 0.28932638]
[ 0.00641276 0.00665579 -0.99995729 -0.14433917]
[ 0. 0. 0. 1. ]]
By the way, the camera intrinsic calculated by kalibr_calibrate_camera is quite different from the result of the calibration toolbox for matlab under the link below, especially the distortion，does it matter?
http://www.vision.caltech.edu/bouguetj/calib_doc/

Hi,

I would like to print the calibration target, such as an Aprilgrid or a checkerboard, on A4 paper and use it for calibration. Could anyone kindly advice how to use kalibr_create_target_pdf to specify the size?

After we have the calibration target, how could we set the size in the yaml accordingly? Should we use the size values in the kalibr_create_target_pdf directly or measure it on the actual calibration target printed on the A4 paper?

Any help is appreciated!

I am not 100% sure, but can someone check whether there should be a .transpose() in line 160 of the file: kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py

Since it only affects the prior, the calibration results should still be correct but it would take longer for the optimization to converge

I had to install libv4l-dev to compile kalibr. I suggest it is added to the dependency list in the installation instructions.

Hi,

Thanks for sharing the package with the community! It helps a lot!

Currently we are trying to find the corresponding parameters of our IMU according to:
https://github.com/ethz-asl/kalibr/wiki/IMU-Noise-Model

According to the datasheet of ADIS 16448 (http://www.analog.com/media/en/technical-documentation/data-sheets/ADIS16448.pdf)
The parameters are:
Gyroscopes:
Angular Random Walk: 0.66 deg/sqrt(hr)
Rate Noise Density (f=25Hz): 0.0135 deg/sec/sqrt(hz)
Accelerometer:
Velocity Random Walk: 0.11 m/sec/sqrt(hr)
Noise Density: 0.23 mg/sqrt(hz)

And the parameters for the same IMU in the Kalibr example (https://github.com/ethz-asl/kalibr/wiki/yaml-formats) are:
Gyroscopes:
Random Walk: 4.0e-6 rad/(s^2)/sqrt(hz)
Noise Density(continuous time): 0.005 rad/s/sqrt(hz)
Accelerometer:
Random Walk: 0.0002 m/(s^3)/sqrt(hz)
Noise Density(continuous time): 0.01m/(s^2)/sqrt(hz)

Could you please specify the relationship between the two sets of parameters? This may be add to the tutorial as an example.

Thank you so much!

• Xinke

Issue for the image drag/drop hack.

http://solutionoptimist.com/2013/12/28/awesome-github-tricks/

I faced this issue and resolved this by change

#import Image
from PIL import Image

in kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_calibration/MulticamGraph.py
Hope this helps those who face the same issue.
Cheers,

Can anyone tell me how to solve this problem:

Exception in thread block: [aslam::Exception] /var/kalibr-build/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [763.696 <= 763.746 < 763.746]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!

https://github.com/ethz-asl/kalibr/wiki/IMU-Noise-Model-and-Intrinsics

Basically the symbols for the noise params

Hi,

The kalibr_calibrate_cameras command seems to fail when called with the fov distortion. I have been able to repeat the error message below on 3 different computers, with Kalibr installed from 3 different people. This issue doesn't appear if the --show-extraction option is enabled or with the other distortion models.

kalibr_calibrate_cameras --bag ~/Desktop/2016-07-12-23-25-53.bag --topics /camera/image_raw --models pinhole-fov --target ~/april_6x6.yaml importing libraries Initializing cam0: Camera model: pinhole-fov Dataset: /home/cbrommer/Desktop/2016-07-12-23-25-53.bag Topic: /camera/image_raw Number of images: 671 Extracting calibration target corners python: /usr/include/boost/archive/detail/oserializer.hpp:436: static void boost::archive::detail::save_pointer_type<Archive>::polymorphic::save(Archive&, T&) [with T = aslam::cameras::CameraGeometryBase; Archive = boost::archive::text_oarchive]: Assertion__null != bpos' failed.
Aborted (core dumped)`

Hi all,

I am looking into the calibration of two imu's using KALIBR to obtain an estimation of their relative spacial transform. (Imu1 with respect to imu0)

My question is: For performing the multi imu calibration: do I move the robot and bag the two IMU-topics (imu0 and imu1) and run the calibration or do I move the robot and bag the two IMU-topics as well as two camera topics (with an aprilgrid in the field of view) thus -> imu0, imu1, cam0, cam1.

I think this might need more clarification in the Wiki.

Thanks!

T

Hello,I have successfully installed kalibr from source and downloaded your sample datasets to run the example using the following command
kalibr_calibrate_imu_camera --target april_6x6.yaml --cam camchain.yaml --imu imu_adis16448.yaml --bag dynamic.bag --bag-from-to 5 45

but there is an error:
RuntimeError: [ImuConfig Reader]: Could not read configuration from imu_adis16448.yaml

this is the log:

 Initializing IMUs:
 Traceback_ (most recent call last):
  File "/home/yanyan/kalibr_workspace/devel/bin/kalibr_calibrate_imu_camera", line 6, in <module>
    exec(fh.read())
  File "<string>", line 236, in <module>
  File "<string>", line 147, in main
  File "/home/yanyan/kalibr_workspace/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 320, in __init__
    ParametersBase.__init__(self, yamlFile, "ImuConfig", createYaml)
  File "/home/yanyan/kalibr_workspace/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 149, in __init__
    self.data = self.readYaml()
  File "/home/yanyan/kalibr_workspace/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 160, in readYaml
    self.raiseError( "Could not read configuration from {0}".format(self.yamlFile) )
  File "/home/yanyan/kalibr_workspace/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 183, in raiseError
    raise RuntimeError( "{0}{1}".format(header, message) )\

Can anyone tell me why it happened and how to solve it.

Thanks!

Hi @rehderj,

thanks for merging in RS and the dataset, I was able to run your 6Hz dataset with latest master.

LineDelay:
-6.00622067443e-07

With a framerate of 20 it completes successfully, while if I try 6 if fails (Spline Coefficient Buffer Exceeded).

Unfortunately, it doesn't work with any of my datasets, with any framerate. I made a dataset around the same length as yours, same movements, but I just get a NaN Jacobian from the optimization at the first step and then it fails.

I wonder if I'm creating the bag in some kind of wrong way: do you use JPEG files or raw images? I pass JPEGs to the bagcreater script. Configuration for the target is as follows:

target_type: 'aprilgrid' #gridtype
tagCols: 6               #number of apriltags
tagRows: 6               #number of apriltags
tagSize: 0.088           #size of apriltag, edge to edge [m]
tagSpacing: 0.3          #ratio of space between tags to tagSize

I measured the tag manually and it is 88mm indeed.

What else is left to check? How should I calculate the estimated inverse variance of the feature detector?

Thanks a lot

Hi there,

I am currently trying to use kalibr with the Invensense MPU-9250 IMU.
As described here (https://github.com/ethz-asl/kalibr/wiki/IMU-Noise-Model), I want to obtain the random walk parameters from the Allan Standard Deviation plot.

On that page you mention an AD script in the following ToDo: [TODO(nikolicj) add AD script]
Would it be possible to get access to that script? This would really be amazingly helpful.

Thanks so much for providing this great software!

Cheers,
Daniel

Hi,
On line 97 of kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_calibration/CameraCalibrator.py
a dot is missing in selfgrid.

Everything works fine, I compiled your code from source on ubuntu 14.04.

I recommend to use a fresh catkin workspace.

Sorry, cannot delete this issue.

Hi Paul,

Your work on rolling shutter camera calibration is cool. The great thing is that the source code is open in Kalibr. By reading your CVPR'13 paper, I have a quick question about the frame time $\bar{t}$ which is defined as the starting time of the integration (exposure) of the first row of an image. But for an image i, we only know its raw timestamp $t_i$.

So I wonder, how did you determine $bar{t}$ given $t_i$? Did you assume there is a constant offset between the two and optimize for this offset in calibration?

As discussed in #64 I am integrating RS code into cam + IMU calibration.

One of the first things I noted is a possible error in the code. At https://github.com/othlu/kalibr/blob/master/aslam_offline_calibration/kalibr/python/kalibr_rs_camera_calibration/RsCalibrator.py#L316 we see there is a test for successful observation, which only checks if there is at least one successful corner observation (see GridCalibrationTargetObservation::hasSuccessfulObservation).

But above at https://github.com/othlu/kalibr/blob/master/aslam_offline_calibration/kalibr/python/kalibr_rs_camera_calibration/RsCalibrator.py#L276 it was stated that the order of observations has to be preserved. As an observation is only a vector of corners (see GridCalibrationTargetObservation::getCornersImageFrame), if there is any missing corner it means the indexes of the following corners will be changed, in the worst case (first corner missing), all of them will be changed and all the errors for the frame will be setup wrong in the jacobian.

The confirmation of this can be seen at https://github.com/ethz-asl/kalibr/blob/master/aslam_cv/aslam_cameras/src/GridCalibrationTargetObservation.cpp#L75-L78

I think the fix is either:

• the test should discard the frame if any of the observations are missing;
• the observations should be passed in a hash so that their order is not impacted by missing detections at a certain frame.

That would make the dataset creation and camera calibration much more robust.

Do you think it makes sense?

The radial-tangantial model in image_proc requires 5 vector, and kalibr provides a 4 vector. The equidist model is not implement in image_proc as well. Is there a quick way to use calibration data obtained through kalibr with image-proc ?

Running the calibration with the provided data fails with the following error:

user@psc:$ kalibr_calibrate_imu_camera --bag dynamic.bag --cam camchain.yaml --imu imu_adis16448.yaml --target april_6x6.yaml 
...
Extracting calibration target corners
Traceback (most recent call last):
File "/Workspace/Catkin/devel/bin/kalibr_calibrate_imu_camera", line 5, in <module>
exec(fh.read())
File "<string>", line 206, in <module>
File "<string>", line 149, in main
File "/Workspace/Catkin/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 391, in __init__
showOneStep=parsed.extractionstepping) )  
File "/Workspace/Catkin/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 59, in __init__
self.targetObservations = kc.extractCornersFromDataset(self.dataset, self.detector, multithreading=multithreading)
File "/Workspace/Catkin/src/Kalibr/aslam_offline_calibration/kalibr/python
/kalibr_common/TargetExtractor.py", line 73, in extractCornersFromDataset
raise RuntimeError("Exception during multithreaded extraction: {0}".format(e))
RuntimeError: Exception during multithreaded extraction: CvBridge instance has no attribute 'imgmsg_to_cv'

This error seems to be similar to
http://answers.ros.org/question/170971/issues-with-cv_bridge-when-using-cameracalibratorpy-in-indigo/

findChessboardCorners uses cv::Size patternSize(cols(), rows());

but drawChessboardCorners uses cv::Size(rows(), cols())

This causes wrong drawing of chessboard corners with different colors in same line like this:

instead of this: