urbste / openimucameracalibrator Goto Github PK
View Code? Open in Web Editor NEWCamera calibration tool
License: GNU Affero General Public License v3.0
openimucameracalibrator's People
Contributors
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Stargazers
Watchers
Forkers
booler ogsdave mfkiwl benpaobutingla vino6520 jenkinlu324 mxjtufei mppcasc wk199 tigerxiaoyg wolf943134497 androidslam zzy0713 honuaye sevenchao sensor-calibration-room adas-eye xiandenis maalouf xp-ji seanxyz woodpeckerjiang gatsby23 xinbin818 cheng-chi liujiacheng1009 renwuli chengwei920412 poormanslaserscanner xiaohulugo altaykacan urbane6679 yaoli1992 fickrie67 peterzs liaozihang drawhamxh aod321 mbdebock firefly-hub zgr2194 xinsuinizhuanopenimucameracalibrator's Issues
Remove SSE2 on ARM64
I am trying to compile without SSE2 but I get errors.
camera_calibrator.cc:307] Calibration failed.
I0619 12:06:18.381163 20784 camera_calibrator.cc:132] Using 61 in bundle adjustment
I0619 12:06:18.381170 20784 camera_calibrator.cc:151] Bundle adjusting focal length and radial distortion.
W0619 12:06:18.388887 20784 residual_block.cc:129]
Error in evaluating the ResidualBlock.
There are two possible reasons. Either the CostFunction did not evaluate and fill all
residual and jacobians that were requested or there was a non-finite value (nan/infinite)
generated during the or jacobian computation.
Residual Block size: 3 parameter blocks x 2 residuals
For each parameter block, the value of the parameters are printed in the first column
and the value of the jacobian under the corresponding residual. If a ParameterBlock was
held constant then the corresponding jacobian is printed as 'Not Computed'. If an entry
of the Jacobian/residual array was requested but was not written to by user code, it is
indicated by 'Uninitialized'. This is an error. Residuals or Jacobian values evaluating
to Inf or NaN is also an error.
Residuals: -2.45432e+156 inf
Parameter Block 0, size: 6
0.18807 | 1.24771e+158 -inf
0.0127992 | -1.88731e+158 -inf
0.15303 | -9.35615e+156 -inf
1.19109 | 1.92433e+157 -inf
2.00557 | 2.59522e+157 -inf
0.228032 | -8.45544e+156 -inf
Parameter Block 1, size: 7
470.504 | -3.55744e+154 inf
1 | 0 inf
0 | 1.37804e+154 -nan
480 | 3.79231e+154 -nan
270 | 0 inf
0 | -1.6939e+157 inf
0 | -1.71425e+157 inf
Parameter Block 2, size: 4
0.189 | Not Computed Not Computed
0.021 | Not Computed Not Computed
0 | Not Computed Not Computed
1 | Not Computed Not Computed
E0619 12:06:18.388916 20784 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed.
I0619 12:06:18.389778 20784 bundle_adjuster.cc:210]
Solver Summary (v 2.0.0-eigen-(3.3.4)-lapack-suitesparse-(5.1.2)-cxsparse-(3.1.9)-eigensparse-no_openmp)
Original Reduced
Parameter blocks 125 62
Parameters 625 373
Effective parameters 619 367
Residual blocks 3201 3201
Residuals 6402 6402
Minimizer TRUST_REGION
Dense linear algebra library EIGEN
Trust region strategy LEVENBERG_MARQUARDT
Given Used
Linear solver DENSE_QR DENSE_QR
Threads 1 1
Linear solver ordering 1,61 1,61
Cost:
Initial -1.000000e+00
Minimizer iterations 0
Successful steps 0
Unsuccessful steps 0
Line search steps 0
Time (in seconds):
Preprocessor 0.005364
Residual only evaluation 0.000000 (0)
Line search cost evaluation 0.000000
Jacobian & residual evaluation 0.000957 (1)
Line search gradient evaluation 0.000000
Linear solver 0.000000 (0)
Line search polynomial minimization 0.000000
Minimizer 0.001009
Postprocessor 0.000062
Total 0.006435
Termination: FAILURE (Residual and Jacobian evaluation failed.)
I0619 12:06:18.391957 20784 camera_calibrator.cc:71] Removed view: 0 with RMSE reproj error: 25.539
I0619 12:06:18.391988 20784 camera_calibrator.cc:71] Removed view: 1 with RMSE reproj error: 14.2904
I0619 12:06:18.392002 20784 camera_calibrator.cc:71] Removed view: 2 with RMSE reproj error: 42.0994
I0619 12:06:18.392014 20784 camera_calibrator.cc:71] Removed view: 3 with RMSE reproj error: 11.9774
I0619 12:06:18.392026 20784 camera_calibrator.cc:71] Removed view: 4 with RMSE reproj error: 12.1681
I0619 12:06:18.392038 20784 camera_calibrator.cc:71] Removed view: 5 with RMSE reproj error: 5.15406
I0619 12:06:18.392050 20784 camera_calibrator.cc:71] Removed view: 6 with RMSE reproj error: 19.5144
I0619 12:06:18.392062 20784 camera_calibrator.cc:71] Removed view: 8 with RMSE reproj error: 7.57894
I0619 12:06:18.392072 20784 camera_calibrator.cc:71] Removed view: 9 with RMSE reproj error: 30.3194
I0619 12:06:18.392083 20784 camera_calibrator.cc:71] Removed view: 10 with RMSE reproj error: 17.9262
I0619 12:06:18.392093 20784 camera_calibrator.cc:71] Removed view: 11 with RMSE reproj error: 38.9146
I0619 12:06:18.392103 20784 camera_calibrator.cc:71] Removed view: 13 with RMSE reproj error: 18.793
I0619 12:06:18.392110 20784 camera_calibrator.cc:71] Removed view: 14 with RMSE reproj error: 38.3689
I0619 12:06:18.392118 20784 camera_calibrator.cc:71] Removed view: 15 with RMSE reproj error: 42.9921
I0619 12:06:18.392125 20784 camera_calibrator.cc:71] Removed view: 16 with RMSE reproj error: 28.1177
I0619 12:06:18.392135 20784 camera_calibrator.cc:71] Removed view: 17 with RMSE reproj error: 19.4586
I0619 12:06:18.392143 20784 camera_calibrator.cc:71] Removed view: 18 with RMSE reproj error: 158.078
I0619 12:06:18.392153 20784 camera_calibrator.cc:71] Removed view: 19 with RMSE reproj error: 45.5234
I0619 12:06:18.392161 20784 camera_calibrator.cc:71] Removed view: 20 with RMSE reproj error: 475.489
I0619 12:06:18.392169 20784 camera_calibrator.cc:71] Removed view: 21 with RMSE reproj error: 10.9856
I0619 12:06:18.392177 20784 camera_calibrator.cc:71] Removed view: 22 with RMSE reproj error: 106.876
I0619 12:06:18.392185 20784 camera_calibrator.cc:71] Removed view: 23 with RMSE reproj error: 46.0012
I0619 12:06:18.392194 20784 camera_calibrator.cc:71] Removed view: 24 with RMSE reproj error: 42.8112
I0619 12:06:18.392202 20784 camera_calibrator.cc:71] Removed view: 25 with RMSE reproj error: 26947.6
I0619 12:06:18.392208 20784 camera_calibrator.cc:71] Removed view: 26 with RMSE reproj error: 64.9799
I0619 12:06:18.392215 20784 camera_calibrator.cc:71] Removed view: 27 with RMSE reproj error: 87.0539
I0619 12:06:18.392223 20784 camera_calibrator.cc:71] Removed view: 28 with RMSE reproj error: 87.4256
I0619 12:06:18.392231 20784 camera_calibrator.cc:71] Removed view: 29 with RMSE reproj error: 97.3909
I0619 12:06:18.392238 20784 camera_calibrator.cc:71] Removed view: 30 with RMSE reproj error: 141.518
I0619 12:06:18.392246 20784 camera_calibrator.cc:71] Removed view: 31 with RMSE reproj error: 165.175
I0619 12:06:18.392254 20784 camera_calibrator.cc:71] Removed view: 32 with RMSE reproj error: 146.289
I0619 12:06:18.392262 20784 camera_calibrator.cc:71] Removed view: 33 with RMSE reproj error: 185.255
I0619 12:06:18.392271 20784 camera_calibrator.cc:71] Removed view: 34 with RMSE reproj error: 165.962
I0619 12:06:18.392278 20784 camera_calibrator.cc:71] Removed view: 35 with RMSE reproj error: 74.9524
I0619 12:06:18.392287 20784 camera_calibrator.cc:71] Removed view: 36 with RMSE reproj error: 65.7859
I0619 12:06:18.392297 20784 camera_calibrator.cc:71] Removed view: 37 with RMSE reproj error: 5.12021
I0619 12:06:18.392305 20784 camera_calibrator.cc:71] Removed view: 38 with RMSE reproj error: 73.3327
I0619 12:06:18.392314 20784 camera_calibrator.cc:71] Removed view: 39 with RMSE reproj error: 493.203
I0619 12:06:18.392323 20784 camera_calibrator.cc:71] Removed view: 40 with RMSE reproj error: 130.537
I0619 12:06:18.392331 20784 camera_calibrator.cc:71] Removed view: 41 with RMSE reproj error: 22.767
I0619 12:06:18.392340 20784 camera_calibrator.cc:71] Removed view: 42 with RMSE reproj error: 80.1456
I0619 12:06:18.392349 20784 camera_calibrator.cc:71] Removed view: 43 with RMSE reproj error: 10.3255
I0619 12:06:18.392359 20784 camera_calibrator.cc:71] Removed view: 44 with RMSE reproj error: 52.5553
I0619 12:06:18.392369 20784 camera_calibrator.cc:71] Removed view: 45 with RMSE reproj error: 29.6967
I0619 12:06:18.392377 20784 camera_calibrator.cc:71] Removed view: 46 with RMSE reproj error: 57.7268
I0619 12:06:18.392385 20784 camera_calibrator.cc:71] Removed view: 47 with RMSE reproj error: 22.0158
I0619 12:06:18.392392 20784 camera_calibrator.cc:71] Removed view: 48 with RMSE reproj error: 30.8915
I0619 12:06:18.392400 20784 camera_calibrator.cc:71] Removed view: 49 with RMSE reproj error: 8.7679
I0619 12:06:18.392407 20784 camera_calibrator.cc:71] Removed view: 50 with RMSE reproj error: 8.96917
I0619 12:06:18.392416 20784 camera_calibrator.cc:71] Removed view: 52 with RMSE reproj error: 5.8264
I0619 12:06:18.392423 20784 camera_calibrator.cc:71] Removed view: 53 with RMSE reproj error: 35.295
I0619 12:06:18.392432 20784 camera_calibrator.cc:71] Removed view: 54 with RMSE reproj error: 59.337
I0619 12:06:18.392441 20784 camera_calibrator.cc:71] Removed view: 55 with RMSE reproj error: 27.2382
I0619 12:06:18.392448 20784 camera_calibrator.cc:71] Removed view: 56 with RMSE reproj error: 45.9613
I0619 12:06:18.392457 20784 camera_calibrator.cc:71] Removed view: 57 with RMSE reproj error: 47.9316
I0619 12:06:18.392474 20784 camera_calibrator.cc:71] Removed view: 58 with RMSE reproj error: 14.216
I0619 12:06:18.392480 20784 camera_calibrator.cc:71] Removed view: 59 with RMSE reproj error: 9.56113
I0619 12:06:18.392488 20784 camera_calibrator.cc:71] Removed view: 60 with RMSE reproj error: 7.33861
I0619 12:06:18.392493 20784 camera_calibrator.cc:161] Optimizing principal point.
W0619 12:06:18.392649 20784 residual_block.cc:129]
Error in evaluating the ResidualBlock.
There are two possible reasons. Either the CostFunction did not evaluate and fill all
residual and jacobians that were requested or there was a non-finite value (nan/infinite)
generated during the or jacobian computation.
Residual Block size: 3 parameter blocks x 2 residuals
For each parameter block, the value of the parameters are printed in the first column
and the value of the jacobian under the corresponding residual. If a ParameterBlock was
held constant then the corresponding jacobian is printed as 'Not Computed'. If an entry
of the Jacobian/residual array was requested but was not written to by user code, it is
indicated by 'Uninitialized'. This is an error. Residuals or Jacobian values evaluating
to Inf or NaN is also an error.
Residuals: -1.64821e+155 -inf
Parameter Block 0, size: 6
0.255604 | Not Computed Not Computed
0.187156 | Not Computed Not Computed
0.196285 | Not Computed Not Computed
-0.259704 | Not Computed Not Computed
2.54699 | Not Computed Not Computed
-0.446241 | Not Computed Not Computed
Parameter Block 1, size: 7
470.504 | -5.22339e+153 -inf
1 | 0 -inf
0 | -1.28911e+154 -nan
480 | 3.79231e+154 -nan
270 | 0 inf
0 | -3.30605e+155 -inf
0 | -4.44737e+154 -inf
Parameter Block 2, size: 4
0.189 | Not Computed Not Computed
0.021 | Not Computed Not Computed
0 | Not Computed Not Computed
1 | Not Computed Not Computed
E0619 12:06:18.392655 20784 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed.
I0619 12:06:18.392663 20784 bundle_adjuster.cc:210]
Solver Summary (v 2.0.0-eigen-(3.3.4)-lapack-suitesparse-(5.1.2)-cxsparse-(3.1.9)-eigensparse-no_openmp)
Original Reduced
Parameter blocks 67 1
Parameters 277 7
Effective parameters 272 2
Estimated focal length: 424.931
Number of Ransac inliers: 26
Not enough views left for proper calibration!
Residual blocks 145 145
Residuals 290 290
Minimizer TRUST_REGION
Dense linear algebra library EIGEN
Trust region strategy LEVENBERG_MARQUARDT
Given Used
Linear solver DENSE_QR DENSE_QR
Threads 1 1
Linear solver ordering 1,3 1
Cost:
Initial -1.000000e+00
Minimizer iterations 0
Successful steps 0
Unsuccessful steps 0
Line search steps 0
Time (in seconds):
Preprocessor 0.000019
Residual only evaluation 0.000000 (0)
Line search cost evaluation 0.000000
Jacobian & residual evaluation 0.000034 (1)
Line search gradient evaluation 0.000000
Linear solver 0.000000 (0)
Line search polynomial minimization 0.000000
Minimizer 0.000042
Postprocessor 0.000002
Total 0.000063
Termination: FAILURE (Residual and Jacobian evaluation failed.)
E0619 12:06:18.392709 20784 camera_calibrator.cc:307] Calibration failed.
program gets stuck
IPFS Dataset downloading
I'm trying to download the dataset from the link, but it doesn't seem to work.
Error in continuous_time_imu_to_camera_calibration.cc
Issue running GOPRO example
Hello,
I am trying to run the GOPRO example, I get this error
(openicc) ubuntu@ip-172-31-32-249:~/OpenImuCameraCalibrator$ python python/run_gopro_calibration.py --path_calib_dataset=~/data/GoPro9/dataset3/ --checker_size_m=0.021 --image_downsample_factor=2 --camera_model=DIVISION_UNDISTORTION
Error! Could not find cam calibration video file with MP4 ending in path ~/data/GoPro9/dataset3/cam
any hint?
thanks in advance
Various files missing when calibrating.
The corners are extracted, lots of stuff is generated, but I see quite a few errors towards the end.
Command within the docker container is: python3 python/run_gopro_calibration.py --path_calib_dataset ../dataset/calibrationDataset/ --checker_size_m=0.05115 --path_to_build ../OpenImuCameraCalibrator/build/applications/
Command to start up the docker container: sudo docker run -it --rm -v pwd:/home -v /home/Downloads/GoPro9:/dataset openicc
Endless state during GoPro calibration
Hi,
I'm running OpenICC in the built docker container and the calibration script seems to be in an endless state (waited more than 20 minutes).
I have tested with the example GoPro dataset and with my own GoPro dataset and in both cases its acting the same.
Does this step take so long or is that a known problem?
It does not end with an error.
Command to execute (own dataset):
python3 python/run_gopro_calibration.py --path_calib_dataset /dataset/dataset/ --path_to_build ../OpenImuCameraCalibrator/build/applications/
Command to execute (sample dataset):
python3 python/run_gopro_calibration.py --path_calib_dataset /dataset/dataset3/ --path_to_build ../OpenImuCameraCalibrator/build/applications/
Here the console output:
root@0de212363e6f:/home# python3 python/run_gopro_calibration.py --path_calib_dataset /dataset/dataset/ --path_to_build ../OpenImuCameraCalibrator/build/applications/
['/dataset/dataset/cam/GH016906.MP4']
==================================================================
Running corner extraction.
==================================================================
Extracing corners for camera calibration.
I1115 14:43:33.048960 785 extract_board_to_json.cc:92] Starting board extraction. This might take a while...
Total number of frames: 2539
I1115 14:43:34.021831 785 board_extractor.cc:443] Extracting corners from frame 60 / 2539
I1115 14:43:34.924288 785 board_extractor.cc:443] Extracting corners from frame 120 / 2539
I1115 14:43:35.943917 785 board_extractor.cc:443] Extracting corners from frame 180 / 2539
I1115 14:43:36.992697 785 board_extractor.cc:443] Extracting corners from frame 240 / 2539
I1115 14:43:37.956604 785 board_extractor.cc:443] Extracting corners from frame 300 / 2539
I1115 14:43:38.761646 785 board_extractor.cc:443] Extracting corners from frame 360 / 2539
I1115 14:43:39.692451 785 board_extractor.cc:443] Extracting corners from frame 420 / 2539
I1115 14:43:40.487787 785 board_extractor.cc:443] Extracting corners from frame 480 / 2539
I1115 14:43:41.260596 785 board_extractor.cc:443] Extracting corners from frame 540 / 2539
I1115 14:43:42.189363 785 board_extractor.cc:443] Extracting corners from frame 600 / 2539
I1115 14:43:43.157529 785 board_extractor.cc:443] Extracting corners from frame 660 / 2539
I1115 14:43:44.065414 785 board_extractor.cc:443] Extracting corners from frame 720 / 2539
I1115 14:43:44.917310 785 board_extractor.cc:443] Extracting corners from frame 780 / 2539
I1115 14:43:45.797626 785 board_extractor.cc:443] Extracting corners from frame 840 / 2539
I1115 14:43:46.697059 785 board_extractor.cc:443] Extracting corners from frame 900 / 2539
I1115 14:43:47.701277 785 board_extractor.cc:443] Extracting corners from frame 960 / 2539
I1115 14:43:48.727213 785 board_extractor.cc:443] Extracting corners from frame 1020 / 2539
I1115 14:43:49.715231 785 board_extractor.cc:443] Extracting corners from frame 1080 / 2539
I1115 14:43:50.589349 785 board_extractor.cc:443] Extracting corners from frame 1140 / 2539
I1115 14:43:51.423588 785 board_extractor.cc:443] Extracting corners from frame 1200 / 2539
I1115 14:43:52.253571 785 board_extractor.cc:443] Extracting corners from frame 1260 / 2539
I1115 14:43:52.975440 785 board_extractor.cc:443] Extracting corners from frame 1320 / 2539
I1115 14:43:53.597079 785 board_extractor.cc:443] Extracting corners from frame 1380 / 2539
I1115 14:43:54.238117 785 board_extractor.cc:443] Extracting corners from frame 1440 / 2539
I1115 14:43:54.945288 785 board_extractor.cc:443] Extracting corners from frame 1500 / 2539
I1115 14:43:55.674548 785 board_extractor.cc:443] Extracting corners from frame 1560 / 2539
I1115 14:43:56.470479 785 board_extractor.cc:443] Extracting corners from frame 1620 / 2539
I1115 14:43:57.278043 785 board_extractor.cc:443] Extracting corners from frame 1680 / 2539
I1115 14:43:58.086393 785 board_extractor.cc:443] Extracting corners from frame 1740 / 2539
I1115 14:43:58.903590 785 board_extractor.cc:443] Extracting corners from frame 1800 / 2539
I1115 14:43:59.738869 785 board_extractor.cc:443] Extracting corners from frame 1860 / 2539
I1115 14:44:00.531898 785 board_extractor.cc:443] Extracting corners from frame 1920 / 2539
I1115 14:44:01.362118 785 board_extractor.cc:443] Extracting corners from frame 1980 / 2539
I1115 14:44:02.158274 785 board_extractor.cc:443] Extracting corners from frame 2040 / 2539
I1115 14:44:02.951296 785 board_extractor.cc:443] Extracting corners from frame 2100 / 2539
I1115 14:44:03.819614 785 board_extractor.cc:443] Extracting corners from frame 2160 / 2539
I1115 14:44:04.764062 785 board_extractor.cc:443] Extracting corners from frame 2220 / 2539
I1115 14:44:05.641790 785 board_extractor.cc:443] Extracting corners from frame 2280 / 2539
I1115 14:44:06.472656 785 board_extractor.cc:443] Extracting corners from frame 2340 / 2539
I1115 14:44:07.325987 785 board_extractor.cc:443] Extracting corners from frame 2400 / 2539
I1115 14:44:08.152597 785 board_extractor.cc:443] Extracting corners from frame 2460 / 2539
I1115 14:44:08.963240 785 board_extractor.cc:443] Extracting corners from frame 2520 / 2539
Extracing corners for imu camera calibration.
I1115 14:44:09.637115 809 extract_board_to_json.cc:92] Starting board extraction. This might take a while...
Total number of frames: 3408
I1115 14:44:10.598726 809 board_extractor.cc:443] Extracting corners from frame 60 / 3408
I1115 14:44:11.385028 809 board_extractor.cc:443] Extracting corners from frame 120 / 3408
I1115 14:44:12.214113 809 board_extractor.cc:443] Extracting corners from frame 180 / 3408
I1115 14:44:13.084995 809 board_extractor.cc:443] Extracting corners from frame 240 / 3408
I1115 14:44:13.891249 809 board_extractor.cc:443] Extracting corners from frame 300 / 3408
I1115 14:44:14.750679 809 board_extractor.cc:443] Extracting corners from frame 360 / 3408
I1115 14:44:15.618247 809 board_extractor.cc:443] Extracting corners from frame 420 / 3408
I1115 14:44:16.486579 809 board_extractor.cc:443] Extracting corners from frame 480 / 3408
I1115 14:44:17.348230 809 board_extractor.cc:443] Extracting corners from frame 540 / 3408
I1115 14:44:18.159485 809 board_extractor.cc:443] Extracting corners from frame 600 / 3408
I1115 14:44:18.995193 809 board_extractor.cc:443] Extracting corners from frame 660 / 3408
I1115 14:44:19.828387 809 board_extractor.cc:443] Extracting corners from frame 720 / 3408
I1115 14:44:20.721172 809 board_extractor.cc:443] Extracting corners from frame 780 / 3408
I1115 14:44:21.598981 809 board_extractor.cc:443] Extracting corners from frame 840 / 3408
I1115 14:44:22.506908 809 board_extractor.cc:443] Extracting corners from frame 900 / 3408
I1115 14:44:23.389768 809 board_extractor.cc:443] Extracting corners from frame 960 / 3408
I1115 14:44:24.260164 809 board_extractor.cc:443] Extracting corners from frame 1020 / 3408
I1115 14:44:25.133056 809 board_extractor.cc:443] Extracting corners from frame 1080 / 3408
I1115 14:44:26.009479 809 board_extractor.cc:443] Extracting corners from frame 1140 / 3408
I1115 14:44:26.861279 809 board_extractor.cc:443] Extracting corners from frame 1200 / 3408
I1115 14:44:27.661465 809 board_extractor.cc:443] Extracting corners from frame 1260 / 3408
I1115 14:44:28.492956 809 board_extractor.cc:443] Extracting corners from frame 1320 / 3408
I1115 14:44:29.303337 809 board_extractor.cc:443] Extracting corners from frame 1380 / 3408
I1115 14:44:30.128628 809 board_extractor.cc:443] Extracting corners from frame 1440 / 3408
I1115 14:44:30.940163 809 board_extractor.cc:443] Extracting corners from frame 1500 / 3408
I1115 14:44:31.775705 809 board_extractor.cc:443] Extracting corners from frame 1560 / 3408
I1115 14:44:32.661419 809 board_extractor.cc:443] Extracting corners from frame 1620 / 3408
I1115 14:44:33.587129 809 board_extractor.cc:443] Extracting corners from frame 1680 / 3408
I1115 14:44:34.523890 809 board_extractor.cc:443] Extracting corners from frame 1740 / 3408
I1115 14:44:35.500397 809 board_extractor.cc:443] Extracting corners from frame 1800 / 3408
I1115 14:44:36.407727 809 board_extractor.cc:443] Extracting corners from frame 1860 / 3408
I1115 14:44:37.349107 809 board_extractor.cc:443] Extracting corners from frame 1920 / 3408
I1115 14:44:38.231518 809 board_extractor.cc:443] Extracting corners from frame 1980 / 3408
I1115 14:44:39.202426 809 board_extractor.cc:443] Extracting corners from frame 2040 / 3408
I1115 14:44:40.089824 809 board_extractor.cc:443] Extracting corners from frame 2100 / 3408
I1115 14:44:40.894716 809 board_extractor.cc:443] Extracting corners from frame 2160 / 3408
I1115 14:44:41.716318 809 board_extractor.cc:443] Extracting corners from frame 2220 / 3408
I1115 14:44:42.510680 809 board_extractor.cc:443] Extracting corners from frame 2280 / 3408
I1115 14:44:43.319604 809 board_extractor.cc:443] Extracting corners from frame 2340 / 3408
I1115 14:44:44.134963 809 board_extractor.cc:443] Extracting corners from frame 2400 / 3408
I1115 14:44:44.912284 809 board_extractor.cc:443] Extracting corners from frame 2460 / 3408
I1115 14:44:45.741256 809 board_extractor.cc:443] Extracting corners from frame 2520 / 3408
I1115 14:44:46.590600 809 board_extractor.cc:443] Extracting corners from frame 2580 / 3408
I1115 14:44:47.425009 809 board_extractor.cc:443] Extracting corners from frame 2640 / 3408
I1115 14:44:48.209347 809 board_extractor.cc:443] Extracting corners from frame 2700 / 3408
I1115 14:44:48.990257 809 board_extractor.cc:443] Extracting corners from frame 2760 / 3408
I1115 14:44:49.776077 809 board_extractor.cc:443] Extracting corners from frame 2820 / 3408
I1115 14:44:50.579536 809 board_extractor.cc:443] Extracting corners from frame 2880 / 3408
I1115 14:44:51.346032 809 board_extractor.cc:443] Extracting corners from frame 2940 / 3408
I1115 14:44:52.106858 809 board_extractor.cc:443] Extracting corners from frame 3000 / 3408
I1115 14:44:52.877943 809 board_extractor.cc:443] Extracting corners from frame 3060 / 3408
I1115 14:44:53.641984 809 board_extractor.cc:443] Extracting corners from frame 3120 / 3408
I1115 14:44:54.413209 809 board_extractor.cc:443] Extracting corners from frame 3180 / 3408
I1115 14:44:55.206188 809 board_extractor.cc:443] Extracting corners from frame 3240 / 3408
I1115 14:44:56.027237 809 board_extractor.cc:443] Extracting corners from frame 3300 / 3408
I1115 14:44:56.887868 809 board_extractor.cc:443] Extracting corners from frame 3360 / 3408
Finished corner extraction.
==================================================================
Corner extraction took 85.07s.
==================================================================
==================================================================
Running camera calibration.
==================================================================
Calibrating camera.
View: 0/2539 initialized for calibration.
View: 100/2539 initialized for calibration.
View: 200/2539 initialized for calibration.
View: 300/2539 initialized for calibration.
View: 400/2539 initialized for calibration.
View: 500/2539 initialized for calibration.
View: 600/2539 initialized for calibration.
View: 700/2539 initialized for calibration.
View: 800/2539 initialized for calibration.
View: 900/2539 initialized for calibration.
View: 1000/2539 initialized for calibration.
View: 1100/2539 initialized for calibration.
View: 1200/2539 initialized for calibration.
View: 1300/2539 initialized for calibration.
View: 1400/2539 initialized for calibration.
View: 1500/2539 initialized for calibration.
View: 1600/2539 initialized for calibration.
View: 1700/2539 initialized for calibration.
View: 1800/2539 initialized for calibration.
View: 1900/2539 initialized for calibration.
View: 2000/2539 initialized for calibration.
View: 2100/2539 initialized for calibration.
View: 2200/2539 initialized for calibration.
View: 2300/2539 initialized for calibration.
View: 2400/2539 initialized for calibration.
View: 2500/2539 initialized for calibration.
Using 150 views for camera calibration.
Current files in directories (in this state):
root@0de212363e6f:/dataset/dataset# ls cam
GH016906.MP4 cam_calib_GH016906_di_2_ransac_poses.ply cam_corners_GH016906.uson
root@0de212363e6f:/dataset/dataset# ls cam_imu/
GH016904.MP4 cam_imu_corners_GH016904.uson
root@0de212363e6f:/dataset/dataset# ls imu_bias/
GH016905.MP4
Missing nodejs packages in docker container
Hi,
gopro-telemetry and mp4box are not installed by default and have to be installed manually.
Can it not be added into the Dockerfile?
Repository/Folder for IMU calibration outputs
Just a thought, but maybe have a nested folder for various common camera calibrations, so that people can buy a camera model and just use the output with reasonable success. While there might be subtle manufacturing differences between units, having base config would help people like me out, especially to compare to with their own calculated configurations.
Issues running your example with GoPro9
Hi, I am trying to follow your calibration example with your data as explained here:
https://github.com/urbste/OpenImuCameraCalibrator/blob/master/docs/gopro_calibration.md
If I extract your data and delete all the files except the mp4 files, I get following error:
==================================================================
Optimizing IMU to Camera calibration using Spline Fusion.
WARNING: Logging before InitGoogleLogging() is written to STDERR
F1123 11:35:26.019022 925709 continuous_time_imu_to_camera_calibration.cc:167] Check failed: ReadIMU2CamInit( FLAGS_gyro_to_cam_initial_calibration, imu2cam, time_offset_imu_to_cam) Could not read: /home/francisco/Downloads/GoPro9/dataset3/cam_imu/imu_to_cam_calibration_GH010164.json
So basically the file "imu_to_cam_calibration_GH010164.json" does not get written out, but I do not see any related error previously in the output text other than a few of these here:
I1123 11:55:45.336704 926842 pose_estimator.cc:254] Removing view 2022 due to large z coordinate: -0.387445 vs median z coordinate -0.183183
If I extract the files provided and do not delete the calibration files, the program runs through with no problems.
I hope somebody can help, this error might be related to some issues already mentioned here with the pyTheia lib?
Execution time
Can someone tell how long it takes do do the calibration?
I'm wating over 200 min and its still executing the calibration.
Iteration time in spline fusion
Hello, thanks for the great repo. I attempted to perform the camera to IMU calibration using the provided GoPro9 dataset. However, it has taken up to 44 iterations so far and still going. Is this duration considered normal?
FilterBadPoses removes all views
Hi,
I'm trying to calibrate my Hero9 and have followed the instructions. However, during estimate camera poses from checkerboard, I realized that FilterBadPoses removes all the views because the difference of the z-values is always greater than median-z.
It turns out that my median-z is a value that is < 0. I tried to modify the if-condition to std::abs(diff) - std::abs(median_z) in PoseEstimator::FilterBadPoses(), but after doing so, I found that after optimizing IMU to camera calibration using spline fusion, my reprojection error for each images after imshow() is of an absurd number (>200 pixels).
I'm not too familiar with Theia, but I'm wondering if there's any way I can visualize my camera poses after camera calibration process?
Any guidance and advice will be appreciated.
Many thanks,
Kenny
static_calib_result.json
Can you provide static_calib_result.json or dataset2?
Failed in pose optimization
Hi urbste,
I have met error std::bad_alloc during pose optimization w/ your MP4 files provided in this link. Do you have any idea about this error? Thanks in advance.
Track Id: 55 std dev: 0.0409686 0.0265254 0.0545904 mm
Track Id: 56 std dev: 0.0385919 0.0249822 0.0543944 mm
Track Id: 57 std dev: 0.0358216 0.0243497 0.0538741 mm
Track Id: 58 std dev: 0.0335099 0.0238545 0.053015 mm
Track Id: 59 std dev: 0.0322875 0.0234659 0.0525263 mm
Track Id: 60 std dev: 0.0322966 0.0234475 0.0527493 mm
Track Id: 61 std dev: 0.0328023 0.0236162 0.0530119 mm
Track Id: 62 std dev: 0.034797 0.0250347 0.0553444 mm
Mean board point standard deviation after optimization: 0.0281578 0.0246573 0.0531155 mm
I0903 05:02:06.131096 249378 pose_estimator.cc:231] Optimizing all estimated poses.
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
==================================================================
Pose estimation estimation took 43.33s.
mono_inertial_gopro_vi tracking lost.
I use the https://github.com/urbste/ORB_SLAM3/tree/master code to run orbslam3. But the tracking is lost.
Can you give some video examples video files(mp4) like this.
The input video is Open ICC calibration data, and the slam setting is gopro9_wide_setting.yaml or gopro9_linear_setting.yaml.
I can not get the gopro9 example result. The slam3 log is :
Examples/Monocular-Inertial/mono_inertial_gopro_vi Vocabulary/ORBvoc.txt Examples/Monocular-Inertial/gopro9_wide_setting.yaml video.mp4 imu_data.json
RB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.
Input sensor was set to: Monocular-Inertial
Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!
Initialization of Atlas from scratch
Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:
Camera Parameters:
- Camera: Fisheye
- Image scale: 1
- fx: 294.398
- fy: 294.268
- cx: 320.341
- cy: 184.266
- k1: 0.0532949
- k2: 0.0025238
- k3: 0.0478167
- k4: -0.042767
- fps: 50
- color order: RGB (ignored if grayscale)
ORB Extractor Parameters:
- Number of Features: 1250
- Scale Levels: 6
- Scale Factor: 1.2
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7
Left camera to Imu Transform (Tbc):
[-0.99996901, 0.00167755, -0.0076600099, -0.00083036401;
0.0076709599, 0.0066179801, -0.99994898, -0.016419901;
-0.00162677, -0.99997699, -0.00663064, -0.00567071;
0, 0, 0, 1]
IMU frequency: 200 Hz
IMU gyro noise: 0.0015 rad/s/sqrt(Hz)
IMU gyro walk: 5e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.017 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.0055 m/s^3/sqrt(Hz)
There are 1 cameras in the atlas
Camera 0 is fisheye
Starting the Viewer
First KF:0; Map init KF:0
New Map created with 406 points
Trying to initialize IMU
Empty frame...
Empty frame...
Empty frame...
Empty frame...
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Video FPS: 59.940059940059932
ORB-SLAM 3 running at: 71.061498183881284 FPS
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Init mbg: -0.049462951922201466
-0.048096000704198311
-0.30794716759215124
Init mba: -1.048252110912111e-09
-2.4094664177576587e-09
-2.4174839976317348e-08
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 14
mnInitialFrameId = 0
125 Frames set to lost
First KF:12; Map init KF:0
New Map created with 407 points
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Video FPS: 59.940059940059932
ORB-SLAM 3 running at: 70.934532675495944 FPS
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Init mbg: 0.036148811787984243
-0.0027985820598631977
0.014331851673705209
Init mba: 2.5153335930231403e-08
-1.079735859818857e-08
2.8651734883282952e-08
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 142
mnInitialFrameId = 138
250 Frames set to lost
First KF:23; Map init KF:12
New Map created with 370 points
Trying to initialize IMU
Trying to initialize IMU
Video FPS: 59.940059940059932
ORB-SLAM 3 running at: 69.572878877722673 FPS
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Trying to initialize IMU
Init mbg: -0.010273129165816786
0.0047093377012380313
-0.0019212091660355615
Init mba: 3.5785226136105223e-08
2.6513270357983744e-09
1.9580616119964778e-08
Video FPS: 59.940059940059932
ORB-SLAM 3 running at: 87.114744841347928 FPS
Not enough motion for initializing. Reseting...
TRACK: Reset map because local mapper set the bad imu flag
SYSTEM-> Reseting active map in monocular case
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 274
mnInitialFrameId = 266
380 Frames set to lost
First KF:35; Map init KF:23
New Map created with 333 points
Trying to initialize IMU
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 407
mnInitialFrameId = 404
395 Frames set to lost
First KF:37; Map init KF:35
New Map created with 343 points
Trying to initialize IMU
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
At the same time, I also used my own GoPro 9 to test the mono_inertial_gopro_vi , but the results were similarly not very good, as the track was always lost. You can download my test data and results from here. Thank you very much for your help.
GFLAGS error while compiling
Hi i get these errors when compiling.
-- Failed to find installed gflags CMake configuration, searching for gflags build directories exported with CMake.
-- Failed to find an installed/exported CMake configuration for gflags, will perform search for installed gflags components.
CMake Error at cmake/FindGflags.cmake:121 (message):
Failed to find gflags - Could not find gflags include directory, set
GFLAGS_INCLUDE_DIR to directory containing gflags/gflags.h
Call Stack (most recent call first):
cmake/FindGflags.cmake:464 (gflags_report_not_found)
CMakeLists.txt:162 (find_package)
E0610 11:26:11.893433 23973 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed. Calibration failed.
Hi,
Any help on the following error. Thanks in advance.
pc@pc:~/work/vendor/linux/OpenImuCameraCalibrator$ python3 python/run_gopro_calibration.py --path_calib_dataset=/home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/ --checker_size_m=0.021 --image_downsample_factor=2 --camera_model=PINHOLE --path_to_build=/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/build/applications/ --path_to_src=/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/
['/home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/GH010162.MP4']
==================================================================
Running corner extraction.
==================================================================
Extracing corners for camera calibration.
I0610 11:22:34.732122 23971 extract_board_to_json.cc:57] Skipping corner extraction. Already extracted for: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/GH010162.MP4
Extracing corners for imu camera calibration.
I0610 11:22:34.778587 23972 extract_board_to_json.cc:57] Skipping corner extraction. Already extracted for: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.MP4
Finished corner extraction.
==================================================================
Corner extraction took 0.09s.
==================================================================
==================================================================
Running camera calibration.
==================================================================
Calibrating camera.
View: 0/1463 initialized for calibration.
View: 100/1463 initialized for calibration.
View: 200/1463 initialized for calibration.
View: 300/1463 initialized for calibration.
View: 400/1463 initialized for calibration.
View: 500/1463 initialized for calibration.
View: 600/1463 initialized for calibration.
View: 700/1463 initialized for calibration.
View: 800/1463 initialized for calibration.
View: 900/1463 initialized for calibration.
View: 1000/1463 initialized for calibration.
View: 1100/1463 initialized for calibration.
View: 1200/1463 initialized for calibration.
View: 1300/1463 initialized for calibration.
View: 1400/1463 initialized for calibration.
E0610 11:26:11.893433 23973 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed.
E0610 11:26:11.898849 23973 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed.
Not enough views left for proper calibration!
E0610 11:26:11.898969 23973 camera_calibrator.cc:307] Calibration failed.
Focal Length:482.086px Principal Point: 480/270px.
Finished camera calibration.
==================================================================
Camera calibration took 217.15s.
==================================================================
==================================================================
Extracting GoPro telemetry for imu bias and camera imu calibration.
==================================================================
Input video path: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.MP4
Output file will be: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.MP4_telemetry.json
Skipping GH010164 as telemetry file already exists: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.json
Done
==================================================================
Telemetry extraction took 0.13s.
==================================================================
Input video path: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/imu_bias/GH010163.MP4
Output file will be: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/imu_bias/GH010163.MP4_telemetry.json
Skipping GH010163 as telemetry file already exists: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/imu_bias/GH010163.json
Done
==================================================================
Estimating IMU biases.
==================================================================
Estimated biases:
gyroscope bias: -0.00161 rad/s, 0.00187 rad/s, -0.00759 rad/s
accelerometer bias: -0.02881 m/s2, -0.03653 m/s2, 0.01007 m/s2
Writing result to: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/imu_bias/imu_bias_GH010163.json
==================================================================
IMU bias estimation took 0.41s.
==================================================================
==================================================================
Estimating camera poses for IMU - CAM calibration.
==================================================================
Could not open: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/cam_calib_GH010162_ph_2.0.json
F0610 11:26:13.795512 24156 estimate_camera_poses_from_checkerboard.cc:52] Check failed: read_camera_calibration(FLAGS_camera_calibration_json, camera, fps) Could not read camera calibration: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/cam_calib_GH010162_ph_2.0.json
*** Check failure stack trace: ***
@ 0x7fa7420b90cd google::LogMessage::Fail()
@ 0x7fa7420baf33 google::LogMessage::SendToLog()
@ 0x7fa7420b8c28 google::LogMessage::Flush()
@ 0x7fa7420bb999 google::LogMessageFatal::~LogMessageFatal()
@ 0x55619acb5dab (unknown)
@ 0x7fa73fed1bf7 __libc_start_main
@ 0x55619acb56aa (unknown)
==================================================================
Pose estimation estimation took 0.98s.
==================================================================
==================================================================
Estimating Spline error weighting and knot spacing.
==================================================================
Knot spacing SO3: 0.050 seconds at quality level q_so3=0.99
Knot spacing R3: 0.090 seconds at quality level q_r3=0.99
Gyroscope weighting factor: 22.374 at quality level q_so3=0.99
Accelerometer weighting factor: 3.852 at quality level q_r3=0.99
Writing result to: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/spline_info_GH010164.json
==================================================================
Spline weighting and knot spacing estimation took 0.83s.
==================================================================
==================================================================
Initializing IMU to camera rotation.
==================================================================
E0610 11:26:14.644371 24181 reconstruction_reader.cc:65] Could not open the file: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/pose_calib_GH010164.calibdata for reading.
F0610 11:26:14.644461 24181 estimate_imu_to_camera_rotation.cc:55] Check failed: theia::ReadReconstruction(FLAGS_input_pose_calibration_dataset, &pose_dataset)
*** Check failure stack trace: ***
@ 0x7f58705300cd google::LogMessage::Fail()
@ 0x7f5870531f33 google::LogMessage::SendToLog()
@ 0x7f587052fc28 google::LogMessage::Flush()
@ 0x7f5870532999 google::LogMessageFatal::~LogMessageFatal()
@ 0x55d6c36d0bb0 (unknown)
@ 0x7f586ed6cbf7 __libc_start_main
@ 0x55d6c36d052a (unknown)
==================================================================
Spline weighting and knot spacing estimation took 0.01s.
==================================================================
==================================================================
Optimizing IMU to Camera calibration using Spline Fusion.
==================================================================
WARNING: Logging before InitGoogleLogging() is written to STDERR
E0610 11:26:14.685009 24182 reconstruction_reader.cc:65] Could not open the file: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/pose_calib_GH010164.calibdata for reading.
F0610 11:26:14.685065 24182 continuous_time_imu_to_camera_calibration.cc:87] Check failed: theia::ReadReconstruction(FLAGS_input_pose_dataset, &pose_dataset) Could not read Reconstruction file.
*** Check failure stack trace: ***
==================================================================
Spline weighting and knot spacing estimation took 0.04s.
==================================================================
==================================================================
Print results.
==================================================================
Traceback (most recent call last):
File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/print_result_stats.py", line 75, in <module>
main()
File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/print_result_stats.py", line 24, in main
data = read_calib_json(args.path_results)
File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/print_result_stats.py", line 9, in read_calib_json
with open(file, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/cam_imu_calib_result_GH010164.json'
pc@pc:~/work/vendor/linux/OpenImuCameraCalibrator$ ls /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/
cam_calib_GH010162_ph_2.0_ransac_pose.ply cam_corners_GH010162.uson GH010162.MP4
more precise Image timestamp from video after HERO8 Black
Hi Steffen:
Sorry for my delayed action on GoPro max's calibration mr because of my busy work.
I think there is a better way to calculate each frame's timestamp than the method used below:
For cameras after Hero fusion, we can use the STMP in streams to calculate measure's precise timestamp. I have tested the imu's stamps calculated by STMP and gpmf-telemetry in your project, they are almost the same.
As you can see in this issue: gopro/gpmf-parser#160, for cameras after HERO8 Black (v2.5), we can use STMP in CORI streams to calculate more precise frame timestamp.(STMP in CORI does not start from 0.)
SyntaxError: Unexpected token = while running extract_metadata.js
Hi,
Since I have no knowledge in JS, I am finding it hard to debug this error:
node $js_path/extract_metadata.js $base_path GH010164.MP4 $base_path
/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/javascript/helpers/mp4reader.js:96
onSamples = (id, user, samples) => {
^
SyntaxError: Unexpected token =
at createScript (vm.js:80:10)
at Object.runInThisContext (vm.js:139:10)
at Module._compile (module.js:616:28)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/javascript/extract_metadata.js:5:25)
I have already installed nodejs(v8.10.0) though apt and did npm install in the directory OpenImuCameraCalibrator/javascript/
This same error is stopping python/run_gopro_calibration.py from executing.
==================================================================
Extracting GoPro telemetry for imu bias and camera imu calibration.
==================================================================
/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/javascript/helpers/mp4reader.js:96
onSamples = (id, user, samples) => {
^
SyntaxError: Unexpected token =
at createScript (vm.js:80:10)
at Object.runInThisContext (vm.js:139:10)
at Module._compile (module.js:616:28)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/javascript/extract_metadata.js:5:25)
/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/javascript/helpers/mp4reader.js:96
onSamples = (id, user, samples) => {
^
SyntaxError: Unexpected token =
at createScript (vm.js:80:10)
at Object.runInThisContext (vm.js:139:10)
at Module._compile (module.js:616:28)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/javascript/extract_metadata.js:5:25)
==================================================================
Telemetry extraction took 0.28s.
==================================================================
Traceback (most recent call last):
File "python/run_gopro_calibration.py", line 315, in <module>
main()
File "python/run_gopro_calibration.py", line 200, in main
telemetry_conv.convert_gopro_telemetry_file(gopro_telemetry, gopro_telemetry_gen)
File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/telemetry_converter.py", line 217, in convert_gopro_telemetry_file
input_telemetry_json, skip_seconds=skip_seconds)
File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/telemetry_converter.py", line 58, in read_gopro_telemetry
self.telemetry = self._read_gopro_telemetry(path_to_jsons, skip_seconds=skip_seconds)
File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/telemetry_converter.py", line 68, in _read_gopro_telemetry
json_file = open(path_to_json, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.json'
System: Kubuntu 18.04, 64-bit
Сalibration from a set of pictures
Is it possible to calibrate using a set of pictures like the kalibr does?
error: ‘PnPType’ is not a member of ‘theia’
Hi @urbste Thanks for your works!
An error occurred when use the new version of OpenICC and pyTheiaSFM .
/OpenImuCameraCalibrator/include/OpenCameraCalibrator/core/pose_estimator.h:80:10: error: ‘PnPType’ in namespace ‘theia’ does not name a type
theia::PnPType pnp_type_ = theia::PnPType::DLS;
I noticed that you are using the new pnp method, when I use last version, The gopro calibration process seems to be stuck, it seems that there is a problem with pose optimization.
I20220119 21:06:09.460232 137055 pose_estimator.cc:132] Skipping view at timestamp : 55.8892s. Not enough points found.
I20220119 21:06:09.460340 137055 pose_estimator.cc:132] Skipping view at timestamp : 55.9059s. Not enough points found.
I20220119 21:06:09.510980 137055 pose_estimator.cc:132] Skipping view at timestamp : 56.69s. Not enough points found.
I20220119 21:06:09.514223 137055 pose_estimator.cc:132] Skipping view at timestamp : 56.7233s. Not enough points found.
I20220119 21:06:09.541810 137055 pose_estimator.cc:132] Skipping view at timestamp : 57.2906s. Not enough points found.
I20220119 21:06:09.562834 137055 pose_estimator.cc:132] Skipping view at timestamp : 57.8411s. Not enough points found.
I20220119 21:06:09.570690 137055 pose_estimator.cc:132] Skipping view at timestamp : 58.0914s. Not enough points found.
I20220119 21:06:09.600827 137055 pose_estimator.cc:146] Pose estimation failed for view at timestamp 58.6085s.
I20220119 21:06:09.702355 137055 pose_estimator.cc:132] Skipping view at timestamp : 60.0934s. Not enough points found.
I20220119 21:06:09.741844 137055 pose_estimator.cc:132] Skipping view at timestamp : 60.744s. Not enough points found.
I20220119 21:06:09.742926 137055 pose_estimator.cc:132] Skipping view at timestamp : 60.7774s. Not enough points found.
I20220119 21:06:09.742964 137055 pose_estimator.cc:132] Skipping view at timestamp : 60.7941s. Not enough points found.
I20220119 21:06:09.744585 137055 pose_estimator.cc:132] Skipping view at timestamp : 60.9108s. Not enough points found.
I20220119 21:06:09.778606 137055 pose_estimator.cc:132] Skipping view at timestamp : 61.3613s. Not enough points found.
I20220119 21:06:09.783015 137055 pose_estimator.cc:132] Skipping view at timestamp : 61.428s. Not enough points found.
I20220119 21:06:09.783051 137055 pose_estimator.cc:132] Skipping view at timestamp : 61.4447s. Not enough points found.
I20220119 21:06:09.783080 137055 pose_estimator.cc:132] Skipping view at timestamp : 61.4781s. Not enough points found.
I20220119 21:06:09.813349 137055 pose_estimator.cc:132] Skipping view at timestamp : 61.8284s. Not enough points found.
I20220119 21:06:09.843387 137055 pose_estimator.cc:132] Skipping view at timestamp : 62.4123s. Not enough points found.
I20220119 21:06:09.843415 137055 pose_estimator.cc:132] Skipping view at timestamp : 62.5124s. Not enough points found.
I20220119 21:06:09.876546 137055 pose_estimator.cc:132] Skipping view at timestamp : 63.0129s. Not enough points found.
I20220119 21:06:09.878156 137055 pose_estimator.cc:132] Skipping view at timestamp : 63.0797s. Not enough points found.
I20220119 21:06:09.878191 137055 pose_estimator.cc:132] Skipping view at timestamp : 63.0964s. Not enough points found.
I20220119 21:06:09.878226 137055 pose_estimator.cc:132] Skipping view at timestamp : 63.1297s. Not enough points found.
I20220119 21:06:09.897560 137055 pose_estimator.cc:132] Skipping view at timestamp : 63.3466s. Not enough points found.
I20220119 21:06:09.897598 137055 pose_estimator.cc:132] Skipping view at timestamp : 63.3633s. Not enough points found.
I20220119 21:06:09.900799 137055 pose_estimator.cc:132] Skipping view at timestamp : 63.4634s. Not enough points found.
I20220119 21:06:09.962488 137055 pose_estimator.cc:132] Skipping view at timestamp : 64.3643s. Not enough points found.
I20220119 21:06:09.963955 137055 pose_estimator.cc:132] Skipping view at timestamp : 64.4477s. Not enough points found.
I20220119 21:06:09.963982 137055 pose_estimator.cc:132] Skipping view at timestamp : 64.5144s. Not enough points found.
I20220119 21:06:10.001494 137055 pose_estimator.cc:132] Skipping view at timestamp : 64.8982s. Not enough points found.
I20220119 21:06:10.001520 137055 pose_estimator.cc:132] Skipping view at timestamp : 64.9482s. Not enough points found.
I20220119 21:06:10.001549 137055 pose_estimator.cc:132] Skipping view at timestamp : 64.9649s. Not enough points found.
I20220119 21:06:10.004623 137055 pose_estimator.cc:132] Skipping view at timestamp : 65.115s. Not enough points found.
I20220119 21:06:10.004660 137055 pose_estimator.cc:132] Skipping view at timestamp : 65.1317s. Not enough points found.
I20220119 21:06:10.025790 137055 pose_estimator.cc:132] Skipping view at timestamp : 65.3486s. Not enough points found.
I20220119 21:06:10.027387 137055 pose_estimator.cc:132] Skipping view at timestamp : 65.432s. Not enough points found.
I20220119 21:06:10.030544 137055 pose_estimator.cc:132] Skipping view at timestamp : 65.4988s. Not enough points found.
I20220119 21:06:10.031658 137055 pose_estimator.cc:132] Skipping view at timestamp : 65.5321s. Not enough points found.
I20220119 21:06:10.033329 137055 pose_estimator.cc:132] Skipping view at timestamp : 65.5822s. Not enough points found.
I20220119 21:06:10.058063 137055 pose_estimator.cc:132] Skipping view at timestamp : 65.8825s. Not enough points found.
I20220119 21:06:10.060990 137055 pose_estimator.cc:132] Skipping view at timestamp : 66.0994s. Not enough points found.
I20220119 21:06:10.081813 137055 pose_estimator.cc:132] Skipping view at timestamp : 66.3496s. Not enough points found.
I20220119 21:06:10.113782 137055 pose_estimator.cc:132] Skipping view at timestamp : 66.8501s. Not enough points found.
I20220119 21:06:11.379074 137055 estimate_camera_poses_from_checkerboard.cc:58] Finished pose estimation.
use my gopro calibration file to generate ***.yaml for ORB_slam3
I use openicc to calibrate my gopro9 and I get the calibration files: cam_imu_calib_result_GX010022.json and cam_calib_GX010020_fi_2.json, how to convert the calibration data to ***.yaml(slam configfile) to run orbslam3.
I think the key parameter is Tbc: !!opencv-matrix data and camera intrinsic.
Here is my calibration data.
cam_calib_GX010020_fi_2.json
cam_imu_calib_result_GX010022.json
The target yaml file like this:
gopro9_maxlens_fisheye_setting.json
Thanks.
Docker Image
Hi,
Is there a docker image for this tool? I am having issues with dependencies.
How to config the quality value in the spline weight estimation?
@urbste Hi author, Thank you for share your project, How should I determine the sensor quality parameters of the get_sew_for_dataset.py. I found that different parameters produce different calibration results because they affect the weights in the final spline optimization process.
Stereo Support?
Might be asking a lot, but is there any way to use this to calibrate stereo cameras with an IMU?
example dataset
is it available one example dataset for IMU/camera calibration?
ORB_SLAM3 fork bugs
Hi Steffen, thank you so much for your work on GoPro SLAM! I'm a robotics PhD student trying to track a custom hand-held device with mounted GoPro.
While trying to use your https://github.com/urbste/ORB_SLAM3 repo, I found a lot of changes regarding to the DoubleSphere camera model was erased by a merge in master. Not sure if this is intentional.
Sorry about creating issue for another repo. Unforuntatley the issue tab on your ORB_SLAM3 repo is disabled.
Problems in the beginning building: cannot find ' opencv2/aruco/dictionary.hpp '
thanks for your job
Here is a problem in the beginning when I try to build this project
OpenImuCameraCalibrator/src/core/camera_calibrator.cc:19:10: fatal error: opencv2/aruco/dictionary.hpp: no that direction or index
19 | #include <opencv2/aruco/dictionary.hpp>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
I want to test some data base on the gopro12, so do you have free time to help me handle with it
Equirectangular image and IMU extrinsics calibration
Hi, Steffen, i have a question and wish to get your advice.
Now i have got a gopro-max camera whose output 360 video is equirectangular(from mp4 file) or EAC(extracted from ffmpeg) and i have no ways to extract original dual fisheye images and calibrate lenses with common camera models.
So the only practicable way to run VIO on gopro-max is to develop a equirectangular VIO system based on other open source code which i have almost done.
However, there exists a problem calibrating extrinsics between the equirectangular 'lens' and imu as both your code, kalibr or so on do not support such projection models. I got two ideas solving the problem. One is to write code to support such model. And the other is to project equirectangular into cubemaps and use front cube image to estimate the extrinsics. I prefer the second as it could save much time. However, i wonder if this method make sense as will equirectangular 'lens' and front cubemap 'lens' share the same orientation and location?
Look forward to your advice and thanks!!
question about the concatenation of metadata
Hi @urbste , thanks for sharing this great work!
I'm also trying to use the video file recorded by GoPro to run vio. I'm so happy when finding someone already done so much research!
My question is about this line
Is it reasonable to just concatenate the metadata directly? Shouldn't there be some time gap between adjacent video segments?
Theia path given but it sets to false
Hi,
When building the project it seems it cannot locate Theia files even after i set the path to required files it finds it but sets to False. error below:
CMake Error at /home/openin/pyTheiaSfM/build/TheiaConfig.cmake:47 (message):
Failed to find Theia - Theia install root: /home/openin, determined from
relative path from TheiaConfg.cmake install location:
/home/openin/pyTheiaSfM/build, does not contain Theia headers. Either the
install directory was deleted, or the install tree was only partially
relocated outside of CMake after Theia was built.
Call Stack (most recent call first):
/home/openin/pyTheiaSfM/build/TheiaConfig.cmake:91 (theia_report_not_found)
CMakeLists.txt:35 (find_package)
CMake Error at CMakeLists.txt:35 (find_package):
Found package configuration file:
/home/openin/pyTheiaSfM/build/TheiaConfig.cmake
but it set Theia_FOUND to FALSE so package "Theia" is considered to be NOT
FOUND.
EstimateRadialDistUncalibratedAbsolutePose error
EstimateRadialDistUncalibratedAbsolutePose double free or corruption (out)
Build error with tbb library
Thanks for the great work.
When I was building the project, I found some packages that I think you missed in README:
opencv_contribshould be built together with OpenCV since you use aruco APIs in your code which is in theopencv_contribpackage. It should be stated explicitly in the installation instruction.- In the installation,
libtbb-devis required but it didn't mention. I installed the library from APT but it didn't work.
The libtbb-dev causes an error as the log below. I use Ubuntu 20.04.
Do you know how could I fix it? Thanks.
biendltb@bienpc:/data/Projects/tmp/OpenImuCameraCalibrator/build$ make -j15
[ 2%] Building CXX object CMakeFiles/OpenImuCameraCalibrator.dir/src/core/imu_camera_calibrator.cc.o
In file included from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/core/imu_camera_calibrator.h:22,
from /data/Projects/tmp/OpenImuCameraCalibrator/src/core/imu_camera_calibrator.cc:16:
/data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:129:10: warning: ‘template<class Key> class tbb::tbb_hash’ is deprecated: tbb::tbb_hash is deprecated, use std::hash [-Wdeprecated-declarations]
129 | tbb::tbb_hash<std::pair<TimeCamId, TimeCamId>>,
| ^~~~~~~~
In file included from /usr/include/tbb/internal/_concurrent_unordered_impl.h:48,
from /usr/include/tbb/concurrent_unordered_map.h:26,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:41,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/core/imu_camera_calibrator.h:22,
from /data/Projects/tmp/OpenImuCameraCalibrator/src/core/imu_camera_calibrator.cc:16:
/usr/include/tbb/internal/_tbb_hash_compare_impl.h:86:74: note: declared here
86 | class __TBB_DEPRECATED_MSG("tbb::tbb_hash is deprecated, use std::hash") tbb_hash
| ^~~~~~~~
In file included from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/core/imu_camera_calibrator.h:22,
from /data/Projects/tmp/OpenImuCameraCalibrator/src/core/imu_camera_calibrator.cc:16:
/data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:304:8: warning: ‘template<class Key> class tbb::tbb_hash’ is deprecated: tbb::tbb_hash is deprecated, use std::hash [-Wdeprecated-declarations]
304 | struct tbb_hash<TimeCamId> : public std::hash<TimeCamId> {};
| ^~~~~~~~
In file included from /usr/include/tbb/internal/_concurrent_unordered_impl.h:48,
from /usr/include/tbb/concurrent_unordered_map.h:26,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:41,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/core/imu_camera_calibrator.h:22,
from /data/Projects/tmp/OpenImuCameraCalibrator/src/core/imu_camera_calibrator.cc:16:
/usr/include/tbb/internal/_tbb_hash_compare_impl.h:86:74: note: declared here
86 | class __TBB_DEPRECATED_MSG("tbb::tbb_hash is deprecated, use std::hash") tbb_hash
| ^~~~~~~~
In file included from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/core/imu_camera_calibrator.h:22,
from /data/Projects/tmp/OpenImuCameraCalibrator/src/core/imu_camera_calibrator.cc:16:
/data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:307:8: warning: ‘template<class Key> class tbb::tbb_hash’ is deprecated: tbb::tbb_hash is deprecated, use std::hash [-Wdeprecated-declarations]
307 | struct tbb_hash<std::pair<TimeCamId, TimeCamId>>
| ^~~~~~~~
In file included from /usr/include/tbb/internal/_concurrent_unordered_impl.h:48,
from /usr/include/tbb/concurrent_unordered_map.h:26,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:41,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/core/imu_camera_calibrator.h:22,
from /data/Projects/tmp/OpenImuCameraCalibrator/src/core/imu_camera_calibrator.cc:16:
/usr/include/tbb/internal/_tbb_hash_compare_impl.h:86:74: note: declared here
86 | class __TBB_DEPRECATED_MSG("tbb::tbb_hash is deprecated, use std::hash") tbb_hash
| ^~~~~~~~
In file included from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:41,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/core/imu_camera_calibrator.h:22,
from /data/Projects/tmp/OpenImuCameraCalibrator/src/core/imu_camera_calibrator.cc:16:
/usr/include/tbb/tbb.h:21:154: note: #pragma message: TBB Warning: tbb.h contains deprecated functionality. For details, please see Deprecated Features appendix in the TBB reference manual.
21 | contains deprecated functionality. For details, please see Deprecated Features appendix in the TBB reference manual.")
| ^
[ 5%] Linking CXX static library libOpenImuCameraCalibrator.a
[ 56%] Built target OpenImuCameraCalibrator
Scanning dependencies of target estimate_imu_to_camera_rotation
Scanning dependencies of target static_imu_calibration
Scanning dependencies of target extract_board_to_json
Scanning dependencies of target fit_allan_variance
Scanning dependencies of target estimate_camera_poses_from_checkerboard
Scanning dependencies of target calibrate_camera
Scanning dependencies of target continuous_time_imu_to_camera_calibration
Scanning dependencies of target create_charuco_board
[ 62%] Building CXX object applications/CMakeFiles/estimate_imu_to_camera_rotation.dir/estimate_imu_to_camera_rotation.cc.o
[ 62%] Building CXX object applications/CMakeFiles/extract_board_to_json.dir/extract_board_to_json.cc.o
[ 64%] Building CXX object applications/CMakeFiles/create_charuco_board.dir/create_charuco_board.cc.o
[ 67%] Building CXX object applications/CMakeFiles/estimate_camera_poses_from_checkerboard.dir/estimate_camera_poses_from_checkerboard.cc.o
[ 70%] Building CXX object applications/CMakeFiles/calibrate_camera.dir/calibrate_camera.cc.o
[ 72%] Building CXX object applications/CMakeFiles/fit_allan_variance.dir/fit_allan_variance.cc.o
[ 78%] Building CXX object applications/CMakeFiles/continuous_time_imu_to_camera_calibration.dir/continuous_time_imu_to_camera_calibration.cc.o
[ 78%] Building CXX object applications/CMakeFiles/static_imu_calibration.dir/static_imu_calibration.cc.o
[ 81%] Linking CXX executable create_charuco_board
In file included from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/applications/continuous_time_imu_to_camera_calibration.cc:27:
/data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:129:10: warning: ‘template<class Key> class tbb::tbb_hash’ is deprecated: tbb::tbb_hash is deprecated, use std::hash [-Wdeprecated-declarations]
129 | tbb::tbb_hash<std::pair<TimeCamId, TimeCamId>>,
| ^~~~~~~~
In file included from /usr/include/tbb/internal/_concurrent_unordered_impl.h:48,
from /usr/include/tbb/concurrent_unordered_map.h:26,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:41,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/applications/continuous_time_imu_to_camera_calibration.cc:27:
/usr/include/tbb/internal/_tbb_hash_compare_impl.h:86:74: note: declared here
86 | class __TBB_DEPRECATED_MSG("tbb::tbb_hash is deprecated, use std::hash") tbb_hash
| ^~~~~~~~
In file included from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/applications/continuous_time_imu_to_camera_calibration.cc:27:
/data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:304:8: warning: ‘template<class Key> class tbb::tbb_hash’ is deprecated: tbb::tbb_hash is deprecated, use std::hash [-Wdeprecated-declarations]
304 | struct tbb_hash<TimeCamId> : public std::hash<TimeCamId> {};
| ^~~~~~~~
In file included from /usr/include/tbb/internal/_concurrent_unordered_impl.h:48,
from /usr/include/tbb/concurrent_unordered_map.h:26,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:41,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/applications/continuous_time_imu_to_camera_calibration.cc:27:
/usr/include/tbb/internal/_tbb_hash_compare_impl.h:86:74: note: declared here
86 | class __TBB_DEPRECATED_MSG("tbb::tbb_hash is deprecated, use std::hash") tbb_hash
| ^~~~~~~~
In file included from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/applications/continuous_time_imu_to_camera_calibration.cc:27:
/data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:307:8: warning: ‘template<class Key> class tbb::tbb_hash’ is deprecated: tbb::tbb_hash is deprecated, use std::hash [-Wdeprecated-declarations]
307 | struct tbb_hash<std::pair<TimeCamId, TimeCamId>>
| ^~~~~~~~
In file included from /usr/include/tbb/internal/_concurrent_unordered_impl.h:48,
from /usr/include/tbb/concurrent_unordered_map.h:26,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/common_types.h:41,
from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:38,
from /data/Projects/tmp/OpenImuCameraCalibrator/applications/continuous_time_imu_to_camera_calibration.cc:27:
/usr/include/tbb/internal/_tbb_hash_compare_impl.h:86:74: note: declared here
86 | class __TBB_DEPRECATED_MSG("tbb::tbb_hash is deprecated, use std::hash") tbb_hash
| ^~~~~~~~
In file included from /data/Projects/tmp/OpenImuCameraCalibrator/include/OpenCameraCalibrator/basalt_spline/calib_helpers.h:41,
from /data/Projects/tmp/OpenImuCameraCalibrator/applications/continuous_time_imu_to_camera_calibration.cc:27:
/usr/include/tbb/tbb.h:21:154: note: #pragma message: TBB Warning: tbb.h contains deprecated functionality. For details, please see Deprecated Features appendix in the TBB reference manual.
21 | #pragma message("TBB Warning: tbb.h contains deprecated functionality. For details, please see Deprecated Features appendix in the TBB reference manual.")
| ^
[ 83%] Linking CXX executable fit_allan_variance
/usr/bin/ld: CMakeFiles/create_charuco_board.dir/create_charuco_board.cc.o: in function `main':
create_charuco_board.cc:(.text+0xb2): undefined reference to `cv::aruco::getPredefinedDictionary(cv::aruco::PREDEFINED_DICTIONARY_NAME)'
/usr/bin/ld: create_charuco_board.cc:(.text+0x12d): undefined reference to `cv::aruco::CharucoBoard::create(int, int, float, float, cv::Ptr<cv::aruco::Dictionary> const&)'
/usr/bin/ld: create_charuco_board.cc:(.text+0x1a2): undefined reference to `cv::aruco::CharucoBoard::draw(cv::Size_<int>, cv::_OutputArray const&, int, int)'
collect2: error: ld returned 1 exit status
make[2]: *** [applications/CMakeFiles/create_charuco_board.dir/build.make:125: applications/create_charuco_board] Error 1
make[1]: *** [CMakeFiles/Makefile2:272: applications/CMakeFiles/create_charuco_board.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 83%] Built target fit_allan_variance
[ 86%] Linking CXX executable extract_board_to_json
[ 89%] Linking CXX executable calibrate_camera
[ 91%] Linking CXX executable estimate_camera_poses_from_checkerboard
[ 94%] Linking CXX executable estimate_imu_to_camera_rotation
/usr/bin/ld: ../libOpenImuCameraCalibrator.a(board_extractor.cc.o): in function `OpenICC::core::BoardExtractor::InitializeCharucoBoard(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, float, float, int, int, int)':
board_extractor.cc:(.text+0x60c): undefined reference to `cv::aruco::DetectorParameters::create()'
/usr/bin/ld: board_extractor.cc:(.text+0x6cd): undefined reference to `cv::aruco::getPredefinedDictionary(cv::aruco::PREDEFINED_DICTIONARY_NAME)'
/usr/bin/ld: board_extractor.cc:(.text+0x719): undefined reference to `cv::aruco::CharucoBoard::create(int, int, float, float, cv::Ptr<cv::aruco::Dictionary> const&)'
/usr/bin/ld: ../libOpenImuCameraCalibrator.a(board_extractor.cc.o): in function `OpenICC::core::BoardExtractor::ExtractBoard(cv::Mat const&, std::vector<Eigen::Matrix<double, 2, 1, 0, 2, 1>, Eigen::aligned_allocator<Eigen::Matrix<double, 2, 1, 0, 2, 1> > >&, std::vector<int, std::allocator<int> >&)':
board_extractor.cc:(.text+0xb08): undefined reference to `cv::aruco::detectMarkers(cv::_InputArray const&, cv::Ptr<cv::aruco::Dictionary> const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::Ptr<cv::aruco::DetectorParameters> const&, cv::_OutputArray const&, cv::_InputArray const&, cv::_InputArray const&)'
/usr/bin/ld: board_extractor.cc:(.text+0xb57): undefined reference to `cv::aruco::DetectorParameters::create()'
/usr/bin/ld: board_extractor.cc:(.text+0xc29): undefined reference to `cv::aruco::refineDetectedMarkers(cv::_InputArray const&, cv::Ptr<cv::aruco::Board> const&, cv::_InputOutputArray const&, cv::_InputOutputArray const&, cv::_InputOutputArray const&, cv::_InputArray const&, cv::_InputArray const&, float, float, bool, cv::_OutputArray const&, cv::Ptr<cv::aruco::DetectorParameters> const&)'
/usr/bin/ld: board_extractor.cc:(.text+0xd75): undefined reference to `cv::aruco::interpolateCornersCharuco(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::Ptr<cv::aruco::CharucoBoard> const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::_InputArray const&, cv::_InputArray const&, int)'
collect2: error: ld returned 1 exit status
make[2]: *** [applications/CMakeFiles/extract_board_to_json.dir/build.make:125: applications/extract_board_to_json] Error 1
make[1]: *** [CMakeFiles/Makefile2:299: applications/CMakeFiles/extract_board_to_json.dir/all] Error 2
[ 94%] Built target estimate_imu_to_camera_rotation
[ 97%] Linking CXX executable static_imu_calibration
[ 97%] Built target estimate_camera_poses_from_checkerboard
[ 97%] Built target static_imu_calibration
[ 97%] Built target calibrate_camera
[100%] Linking CXX executable continuous_time_imu_to_camera_calibration
/usr/bin/ld: CMakeFiles/continuous_time_imu_to_camera_calibration.dir/continuous_time_imu_to_camera_calibration.cc.o: in function `tbb::interface7::task_arena::initialize()':
continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb10interface710task_arena10initializeEv[_ZN3tbb10interface710task_arena10initializeEv]+0x27): undefined reference to `tbb::interface7::internal::task_arena_base::internal_initialize()'
/usr/bin/ld: CMakeFiles/continuous_time_imu_to_camera_calibration.dir/continuous_time_imu_to_camera_calibration.cc.o: in function `tbb::interface7::task_arena::terminate()':
continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb10interface710task_arena9terminateEv[_ZN3tbb10interface710task_arena9terminateEv]+0x24): undefined reference to `tbb::interface7::internal::task_arena_base::internal_terminate()'
/usr/bin/ld: CMakeFiles/continuous_time_imu_to_camera_calibration.dir/continuous_time_imu_to_camera_calibration.cc.o: in function `tbb::interface7::task_arena::current_thread_index()':
continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb10interface710task_arena20current_thread_indexEv[_ZN3tbb10interface710task_arena20current_thread_indexEv]+0x9): undefined reference to `tbb::interface7::internal::task_arena_base::internal_current_slot()'
/usr/bin/ld: CMakeFiles/continuous_time_imu_to_camera_calibration.dir/continuous_time_imu_to_camera_calibration.cc.o: in function `tbb::flow::interface10::graph::wait_for_all()':
continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb4flow11interface105graph12wait_for_allEv[_ZN3tbb4flow11interface105graph12wait_for_allEv]+0x7b): undefined reference to `tbb::task_group_context::is_group_execution_cancelled() const'
/usr/bin/ld: continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb4flow11interface105graph12wait_for_allEv[_ZN3tbb4flow11interface105graph12wait_for_allEv]+0xb5): undefined reference to `tbb::task_group_context::reset()'
/usr/bin/ld: continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb4flow11interface105graph12wait_for_allEv[_ZN3tbb4flow11interface105graph12wait_for_allEv]+0xfd): undefined reference to `tbb::task_group_context::reset()'
/usr/bin/ld: CMakeFiles/continuous_time_imu_to_camera_calibration.dir/continuous_time_imu_to_camera_calibration.cc.o: in function `tbb::flow::interface10::graph::~graph()':
continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb4flow11interface105graphD2Ev[_ZN3tbb4flow11interface105graphD5Ev]+0x4c): undefined reference to `tbb::interface5::internal::task_base::destroy(tbb::task&)'
/usr/bin/ld: continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb4flow11interface105graphD2Ev[_ZN3tbb4flow11interface105graphD5Ev]+0x6d): undefined reference to `tbb::task_group_context::~task_group_context()'
/usr/bin/ld: CMakeFiles/continuous_time_imu_to_camera_calibration.dir/continuous_time_imu_to_camera_calibration.cc.o: in function `void tbb::interface7::task_arena::execute_impl<void, tbb::flow::interface10::graph::wait_functor const>(tbb::flow::interface10::graph::wait_functor const&)':
continuous_time_imu_to_camera_calibration.cc:(.text._ZN3tbb10interface710task_arena12execute_implIvKNS_4flow11interface105graph12wait_functorEEET_RT0_[_ZN3tbb10interface710task_arena12execute_implIvKNS_4flow11interface105graph12wait_functorEEET_RT0_]+0x52): undefined reference to `tbb::interface7::internal::task_arena_base::internal_execute(tbb::interface7::internal::delegate_base&) const'
collect2: error: ld returned 1 exit status
make[2]: *** [applications/CMakeFiles/continuous_time_imu_to_camera_calibration.dir/build.make:125: applications/continuous_time_imu_to_camera_calibration] Error 1
make[1]: *** [CMakeFiles/Makefile2:191: applications/CMakeFiles/continuous_time_imu_to_camera_calibration.dir/all] Error 2
make: *** [Makefile:84: all] Error 2Make GPS data optional?
I don't think GPS data is used, and when I don't have it recorded in the imu calibration video, stuff blows up.
estimate_imu_to_camera_rotation failed
Hi, I'm running your examples of GoPro camera & imu calibration but I failed with following mistake. Looks like std::vector<double> cams_dt_s or timestamps_images is empty?
==================================================================
Spline weighting and knot spacing estimation took 2.31s.
==================================================================
==================================================================
Initializing IMU to camera rotation.
==================================================================
I0602 14:01:41.985631 3844 estimate_imu_to_camera_rotation.cc:72] Load IMU bias file: /home/Downloads/GoPro/GoPro9/dataset3/imu_bias/imu_bias_GH010163.json
I0602 14:01:42.436834 3844 estimate_imu_to_camera_rotation.cc:87] Using supplied initial imu 2 camera time offset: -0.078202
I0602 14:01:42.453464 3844 estimate_imu_to_camera_rotation.cc:105] Mean IMU data rate: 202.914Hz
estimate_imu_to_camera_rotation: /home/Downloads/GoPro/OpenImuCameraCalibrator/src/utils/utils.cc:78: double OpenICC::utils::MedianOfDoubleVec(std::vector<double, std::allocator<double> >&): Assertion `!double_vec.empty()' failed.
==================================================================
timestamp quality for go pro camera
Thanks for your good work!
I find this project from https://www.youtube.com/watch?v=Phw_OVP6sxI and i want to use go-pro max in my vio system.
I have some questions about the go-pro data you used in the video, and sincerely wish to get your help.
- image timestamp: is it a real raw element in bin file, or infered via ffmpeg or so on.
- the time offset(image and imu) repeatability of go-pro device, that's to say, once i calibrate the time offset of a go-pro camera, can i use it as initial value for the following data i record(I will also online calibrate it)? (time offset repeatability of Insta360 one x2 is quite terrible, quite different for different video)
- Is the go-pro camera suitable for vio job in your opinion?
Wish your answer!
Question about orbslam3 setting yaml
I try to use Gopro10 black.
i run python script file(run_gopro_calibration.py), and i get some result json files.
anyway, i have to setup my setting yaml for orbslam3.
where i can find IMU.NoiseGyro, IMU.NoiseAcc, IMU.GyroWalk, IMU.AccWalk, IMU.Frequency values?
Thanks
An issue when running a self-recording GoPro10 video
Is your project supporting GoPro10?
After I recorded the GoPro10 video according to your data collection standards, the following issues occurred:"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.