Good conditions for IMU-cam calibration about kalibr HOT 7 CLOSED

Hi,

thank you for using our toolbox.

For determining the orientation prior (more correctly, the initial guess for the relative rotation between IMU and camera), we align angular velocities perceived by the gyroscopes with those perceived by the camera from observing the visual target. (https://github.com/ethz-asl/kalibr/blob/master/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py#L104-L172)
When temporal calibration is activated in addition (through the option --time-calibration), this step is preceded by estimating a static temporal offset from aligning the (frame-independent) absolute angular velocity perceived by the two modalities. (https://github.com/ethz-asl/kalibr/blob/master/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py#L174-L235) This method also provides some plotting functionality that can be activated with the option "--verbose".

So, in your problem only the camera and the gyroscopes are involved. We observed this alignment to work robustly even for low-price gyroscopes (that might exhibit scale or misalignment issues). Common sources of error hence are misdetections of the visual target, spurious measurements in the gyroscopes from incorrectly reading from these devices, and synchronization issues.
You can visualize the detections with --verbose, --show-extraction and --extraction-stepping to make sure that the target is detected correctly. For spurious measurements, you might want to use rqt_plot or some similar tool to look for spikes in the data. Finally for incorrect synchronization, you might want to look at the difference in header times of succeeding IMU and images messages (should be constant).
In kalibr, you could also plot the correlation of the absolute angular velocities (using the --verbose option). This should show two well aligned curves and a correlation plot with a distinct peak.

The absolute size of the target should not play a major role. The size of its projection relative to the camera image will affect the accuracy to which orientations can be estimated by the camera. In turn, it might have implications on the procedure for obtaining an initial guess of the relative orientation between camera and IMU. However, unless it is an extreme case (which is not supported by your image), this should not be the cause of your problem.

Finally and as a side note, varying exposure times induce varying delays which, unaccounted for, may cause problems for high precision applications. While this should not be the cause for your problem, you might want to address this issue if system accuracy is crucial.

Cheers!

from kalibr.

As a small addition: Problems in the gyroscope measurements are not always obvious. For many IMUs, measurement ranges can be selected, and in many cases this yields a change in the conversion from the digital output to SI units. Not accounting for this change in scale factors will result in incorrect measurements, which will cause the alignment process to fail. However, the scale factor change is rarely subtle, so it can be assessed by means of rather simple experiments.

from kalibr.

Thanks for the suggested tests. I ran them. Unfortunately the problem is still on, but here are my observations.

Target detection was successful in all frames. Verbose mode tells me that some outlier corners are eliminated in each image due to too high reprojection errors, 3-4 on average and always less than 10 though.

Spurious measurements: plotting gyros' measurements in rqt_plot, I did not see any major anomaly. I can see spikes of the order of 0.02 rad/sec when I zoom in the plot. This is 2 orders of magnitude than the angular velocities I'm observing, so I'm assuming this is normal.

IMU header times show a time difference between IMU mostly vary of a few microseconds around the 5 milliseconds period expect at 200 Hz. I could notice a few missing IMU message though, up to 4 in a row, by observing delays which are multiples of 5 milliseconds (see plot below). Could that be the problem?

I actually wasn't enabling the --time-calibration in my previous attempts. By doing this now, I get a different error, which I report in the attached text file. I never get to see the plots of absolute angular velocities you're mentioning, this error probably happens before that then.

error.txt

Finally, I also checked the angular velocity measurement by putting my setup on a rotating table with a known angular velocity and the output matched.

from kalibr.

Thank you for the updated information. In general, the image header times would be just as interesting as the IMU header times. Sorry for not being clearer on this before.

I agree with you that the results of all tests you ran look pretty good. So I am wildly guessing from the error message you posted: Is it possible that IMU and camera header timestamps are massively off? Unfortunately, rqt_bag will only plot ticks for arrival timestamps, so you will have to directly compare the header timestamps.

If this is not the case, maybe you could instrument the angular velocity correlation code with outputs to see what kind of input data is provided ( https://github.com/ethz-asl/kalibr/blob/master/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py#L208-L209)

Cheers!

from kalibr.

Hi, I don't know if the issue has been resolved yet. Using Kalibr to calibrate a stereo pair and an Imu,
I got the same error saying:

return multiarray.correlate2(a, v, mode)

ValueError: negative dimensions are not allowed

I was running the cameras and the imu on different computers of a multiple machine ros network.
I found, that the cam1 and cam0 (running on the same machine) topics were rather well synchronized, but the timestamps of the imu were 61.5 seconds (!) off (in the future). Apparently, the computers had different system times.

When running the cameras and the imu on the same machine, the timestamps of imu, cam0 and cam1 were closer together, and calibration was successful.

Hope this helps,
Cheers!

from kalibr.

Hi,

maybe some clarifications about kalibr are helpful here:

• kalibr is a tool for calibrating the intrinsics and extrinsics of multiple cameras with respect to each other as well as the extrinsics and a (small) fixed time delay of multiple cameras with respect to an IMU.
• kalibr uses header timestamps, while rqt_bag displays timestamps when messages were recoded. Please see https://github.com/ethz-asl/dataset_tools/blob/master/python/restamp_bag.py for a tool that assigns header timestamps to recording timestamps, facilitating inspection of correct timing in rqt_bag.
• kalibr was not designed to compensate for inadequate system design. For the correlation (that obtains an initial guess for the time offset) to be successful, the timespan covered by IMU and camera timestamps has to match approximately. If your system exhibits an offset in timestamps of a minute, you will likely benefit substantially from improving your setup. If this is legacy data and you cannot do anything about it, please modify the aforementioned script so that it gets you into the ballpark from where kalibr can pick up.
• kalibr does not fiddle with your timestamps. It just uses header timestamps. For best results, you probably want to clean up your timestamps. See TICSync by Harrison & Newman or the work by Moon et al. for approaches to achieve this.

Also, please let me know in case your verify that this error is related to non-overlapping timespans in IMU and camera timestamps and whether it is fixed by appropriate timing, so that this issue can be closed.

Cheers!

Update kalibr has since been updated to include a --perform-synchronization option for assessing issues with the timestamping. Please see the wiki for details.

from kalibr.

The camera and IMU header timestamps were indeed off by... 105 min in my bag file. This was found using the restamp_bag python script suggested @rehderj and plotting the timestamps in rqt_bag.

I have been able to run successful calibration after ensuring my camera and IMU were synchronized, like @paul2746 . I'm closing this issue.

Thank you very much for your time and help!

from kalibr.