Interpreting the training outputs about meshtransformer HOT 2 CLOSED

Wow, that was an instant feedback. Cheers for that.

Regarding the extreme case that the entire body is occluded. The reason why I have asked is that it actually works good, considering that it sees nothing. also the score is scarily high.

Finally, regarding the spikes, you are right, it is the early training stage, I was checking it just because it happens that in some evaluations it actually works fine, then in the following one, the spikes appear again, but I assume it also depends from random evaluation images.

For example after 238000 iterations:

After 28600 iterations:

and it goes back and forth.

But your feedback, that it is because image-mesh alignment is not good, makes absolute sense. As is too early to make other assumption.

Have a good one,
Cheers

from meshtransformer.

Thanks for the questions!

We usually use (partially-) occluded training samples to improve model training. In many datasets, their 2D keypoint annotations have the visibility flags. If a specific keypoint is invisible (i.e., occluded), we will simply ignore it. However, in the examples you mentioned above, I see there is one extreme case that the entire human body is occluded. We may need to remove these kind of training samples in future. I believe the training sample is from MuCo-3DHP dataset.

I also observed the spike issue. I usually see the spikes when it is in an early training stage. This is probably because the image-mesh alignment is still not good. Could you please let me know the number of iterations/epoches you tried to obtain these examples?

from meshtransformer.