About the optimizers and hyper-parameters about relabel_imagenet HOT 3 CLOSED

According to the paper (Section 4.1), you mentioned the best result (80.2%) was achieved by training ResNet-50 with an SGD optimizer. However, from the config files provided in this repo, AdamP is used for ResNet-50+ReLabel+CutMix. I am wondering that which optimizer did you use for training with CutMix? Looking forward to your reply. Thanks!

from relabel_imagenet.

@yxgeee

Thank you for the inspiring work! I have read the paper and the code, and would like to raise a question about the selection of the training hyper-parameters. Specifically, I found that you use custom setups for optimizers (e.g., SGD v.s. AdamP, learning rate, weight decay, etc.) in three configs (baseline, relabel, relabel+cutmix). I am wondering that how did you tune it? Are there any policies? Thanks!

According to the paper (Section 4.1), you mentioned the best result (80.2%) was achieved by training ResNet-50 with an SGD optimizer. However, from the config files provided in this repo, AdamP is used for ResNet-50+ReLabel+CutMix. I am wondering that which optimizer did you use for training with CutMix? Looking forward to your reply. Thanks!

Yes, we used AdamP optimizer when applying CutMix regularization for further improvement.
Otherwise, we used SGD optimizer.

For your information,

• ResNet-50+ReLabel+CutMix with SGD: 79.8%
• ResNet-50+ReLabel+CutMix with AdamP: 80.2%

The hyperparameter for AdamP is described in our config file (relabel_train_resnet50_cutmix.yaml). We tuned the lr and wd for AdamP by grid search.

We will add these details in Section 4.1.
Thank you for pointing it out!

from relabel_imagenet.

Thanks a lot for your reply!

from relabel_imagenet.