Some questions about the training process about magnet HOT 4 CLOSED

Hi,

You should train the segmentation backbone before training the refinement. The guideline for training backbone is already here https://github.com/VinAIResearch/MagNet#training-backbone-networks

from magnet.

I made some attempts to retrain the backbone and refinement. As the model performance after training, the best_mIou of the backbone and the epoch_IoU of the refinement model are around 0.6 and 0.25, respectively. But the epoch_IoU can go up to 0.9 when the refinement model is retrained with the pretrained backbone. I think the difference is caused by the backbone not being trained well. So I try different random seed, batch_size, learning rate, and distributed training to get backbone to work well, but all failed. I guess that some tricks are used in the training process. Could the checkpoint.pth.tar or training log be made publiced?

from magnet.

Recently, I separated the validate part from backbone/train.py to evaluate the performance of pretrained backbone. Through comparing the performance of backbone with pretrained and retrained parameters, it would be cleared where the abnormalities occured in the process of retraining on deepglobe dataset. The fpn and hrnet_ocr backbone with pretrained parameters have been evaluated. The best_Iou of retrained fpn and hrnet_ocr are 0.61, 0.63 learned from the training log, while the best_Iou of pretrained fpn and hrnet_ocr are unknown. As the evaluate result, the mIou of pretrained fpn and hrnet_ocr are 0.07 and 0.61. The 0.61 mIou of pretrained hrnet_ocr could prove the upper best_Iou limit of retrained fpn is around 0.6. The retraining of fpn backbone seems work well. However, in the retraining of refinement model on deepglobe dataset, the epoch_IoU only rises to 0.3 with 0.61 best_iou retrained fpn, while the epoch_Iou with 0.07 mIou pretrained fpn could go up to 0.9; in the retraining of refinement model on cityscapes dataset, the epoch_IoU could rise to 0.87 with 0.63 best_iou retrained hrnet_ocr, and the epoch_Iou with pretrained hrnet_ocr could go up to 0.93. The performance between pretrained and retrained fpn is opposite to the evaluate result, but the performance between pretrained and retrained hrnet_ocr is matching with the evaluate result. Have you ever encountered this difference?

from magnet.

Hi,
It would help if you break down your issues into multiple questions. I'm still not understanding your problem here.

Thank you.

from magnet.