Why multi object pose estimation is not possible with linemod dataset? about efficientpose HOT 2 CLOSED

Hi danikhani,

I already answered your question here.
To give you a better understanding, I took the following images from the Linemod dataset including their 6D annotation.

You can see that in every image only a single object is annotated in the Linemod dataset but the other objects are also visible in the images.
If you want to train a single model for all objects the problem is that if you feed e.g. the annotated cat image from above to your model, you can only provide the annotations of the cat but not for the ape, duck, etc.
So the ground truth for all the other objects without annotations is just "background". This will unnecessarily penalize your model a lot during training when actually correctly detecting the other objects.

The Occlusion dataset (subset of Linemod) instead provides annotations for all objects in the image and is therefore better suited for multi 6d pose estimation.

Furthermore, I don't think it is impossible to train a single model for all Linemod objects because you can try using synthetic rendered data and the cut&paste strategy like PVNet but you probably can't use the vanilla Linemod images which could worsen the performance of your model on the test dataset.
But I didn't do any experiments here so this is just a guess.

Sincerely,
Yannick

from efficientpose.

Hi!

Sorry for re-mentioning the issue, but I still have a question here, for settings upon multi-object pose estimation.

One of your competitor, DPOD (Zakharov, Sergey, Ivan Shugurov, and Slobodan Ilic. "Dpod: 6d pose object detector and refiner." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.), is also a holistic method (if its refiner is not used) for multi-object pose estimation. As is stated in its paper, DPOD has conducted corresponding experiments tested on Occlusion dataset, and clearly all objects are trained in one single model. So why don't you conduct this experiment in the same setting?

P.S. One thing I slightly suspect about this experiment of DPOD is that the training dataset might not include real dataset from LineMOD. However, if it were trained with synthetic images only, this crutial settings should have been pointed out in the paper. Also, the synthetic dataset generated by DPOD is of really low quality, I highly doubt that if they can achieve 47.25% ADD on Occlusion if trained only with synthetic images.

from efficientpose.