Question about the Self-supervised pre-trained models (MoCo) about imbalanced-semi-self HOT 3 CLOSED

Good question. I think in this case, your actual problem is to apply MoCo on an imbalanced dataset. There could be many factors. One possible explanation is on hyper-parameters --- for example, the number of pre-training epochs, as I only trained for 200 epochs, longer training might bring better performance. Also, during the fine-tuning stage on linear classifier, you might want to follow the hyper-parameter in MoCo's repo, i.e., the learning rate (they usually use very large lr, e.g., 30, for the linear classifier).

And indeed, the performance might be actually reasonable, indicating that current contrastive self-supervised learning might have large performance drop when facing imbalanced data. This problem is an independent problem, and might be intereting to the self-supervised learning community.

from imbalanced-semi-self.

thank you so much!

from imbalanced-semi-self.

Hi, I'm very interested in this paper. Recently, I'm trying to reproduce the results on ImageNet-LT with this code. However, I found that the loss value can only decrease from 9.5 to 6.9 for the moco pretraining. Is it correct? Can you post your training log for reference ? Thank you very much.

from imbalanced-semi-self.