mAP calculation in evaluate.py about pointpillars HOT 4 CLOSED

Hello @hongduc2307,

11 recall positions are calculated on validation set here.

Because sampling precisions every 4 intervals as follows, so "11" is hard-coded here.

For evaluation, we keep consistent with mmdet3d, as implemented in the following lines.

https://github.com/open-mmlab/mmdetection3d/blob/eb5a5a2d166d2d60dfbdffe63b925cb37a6541e3/mmdet3d/core/evaluation/kitti_utils/eval.py#L573-L577

from pointpillars.

@zhulf0804 Thank you for quick reply.

Yes I see your point here, but when I print len(score_thresholds) it is not always 41 so the loop size is not 11.

I see the score_thresholds could be taken from the following mmdet3d function. The third argument num_sample_pts=41 may not ensure that len(score_thresholds) is 41.

https://github.com/open-mmlab/mmdetection3d/blob/eb5a5a2d166d2d60dfbdffe63b925cb37a6541e3/mmdet3d/core/evaluation/kitti_utils/eval.py#L10

from pointpillars.

Yes, you are right.

P-R curve draws (precision, recall) point at the specific scores. However, the recall can't be 100% even if given a very small score. Just like the following figure, the highest recall is slightly higher than 0.8. So it's not always 41 or 11.

from pointpillars.

@zhulf0804 Yes you're alright.

Now it's clear that the hard-code "11" should be changed depending on len(score_thresholds) so the result AP will change a bit.
Thank you so much for kind answers. This repo is a really great work.

from pointpillars.