falling to recover detailed local shapes at predicted points, Single categorie ShapeNet-55 about pointr HOT 3 CLOSED

Hi, @alekovargas

Thanks for your interest in our work. I haven't tried to train PoinTr on Single category before, but it is nice to see the results from you.
From your results and descriptions, PoinTr can work well on the coarse(center) prediction in your setting. However, the detail local geometry can not be correctly inferred. I think there maybe two reasons:

• In the training pipeline, local geometry is weakly supervised by an CD between the complete shapes, not the local regions. And CD loss, to some extend, is suboptimal in shape reconstruction problems.
• The problem is too challenge. I am curious about the drop ratio when you evaluate the model ( and visualize the results). The model trained on fewer samples may lack of generalization ability to all incomplete patterns. And for those situations where the missing parts contain complex geometry patterns (beyond simple flat surfaces), the model may need more samples or more parameters. (the head and wings of airplane can be completed well by PoinTr, but for earphone, bottle, PoinTr performes worse)

For the first point, can we add a local shape reconstruction loss to constrain more on the local geometry? And for loss between overall shapes, I haven't try to optimize EMD loss during the training of PoinTr, which worths a try.
For the second point, if your situation does not allow you to have many samples, i think more augmentations maybe helpful. In may codebase, i only use random-crop for the samples in ShapeNet-55.

Hope it can help you.

Best!

from pointr.

Close it since no response. Feel free to re-open it if problems still exist

from pointr.

sorry for the late replay,

Yes more augmentation data solved the problem, thanks for such nice work.

regards

from pointr.