Evaluation Metric about semantic_suma HOT 3 CLOSED

Hey @MohamedAfifii,

Thanks for following our work.

As far as I know, the final results are the average of all the subsequences, which you could find more detailed information in the official evaluation code: https://s3.eu-central-1.amazonaws.com/avg-kitti/devkit_odometry.zip.

Note that, for kitti road dataset, we use the old lengths which are = {5,10,50,100,150,200,250,300,350,400}, since some of the road sequences are less than 800 frames.

I hope my answer helps!

from semantic_suma.

Thanks for your reply.

So is the overall total error the micro average (average of individual subsequence errors) or the macro average (average of sequence errors)?

from semantic_suma.

Thanks for your reply.

So is the overall total error the micro average (average of individual subsequence errors) or the macro average (average of sequence errors)?

For the KITTI odometry benchmark, we will in the end get only one evaluation result, which is the average over all sequences and subsequences.

For more information about the KITTI odometry benchmark, we refer to the original website: http://www.cvlibs.net/datasets/kitti/eval_odometry.php

Since the benchmark is not a part of this work, I would like to close this issue.
Please feel free to drop me messages if there is any other question about SuMa++.

from semantic_suma.