The result of RPN is close to 0 for zero-shot inference of own dataset, however, the result of GT is very good. What should I do? about regionclip HOT 5 CLOSED

What I do is to merge the weights of RPN and pretrained models and don't provide RPN checkpoint, i.e., only provide one model checkpoint with MODEL.WEIGHTS. Then the mAP in paper can be reproduced.

from regionclip.

Thanks for your interest in our work. According to your descriptions, the low performance is caused by localization errors. You could try training an RPN on your own dataset. This should provide a reasonable performance between 0 and GT boxes.

from regionclip.

g

I have also encountered the same situation. Would you be willing to further explain this phenomenon? I also believe that the decrease in accuracy is due to a localization issue, but I do not understand why the AP value would be close to 0.

from regionclip.

Thanks for your interest in our work. According to your descriptions, the low performance is caused by localization errors. You could try training an RPN on your own dataset. This should provide a reasonable performance between 0 and GT boxes.

Hello! I have tried to train an RPN on my own dataset. However, even if the RPN model shows that it could predict the catogories on the AP around 60, when I try to use the RPN model weight in RegionCLIP (I just simplily replace the model weight, I guess it may have some mistakes but I didn't find), the AP is not closed to 0 but relatively low (around 5 on some categories and also be closed to 0 on the others). And the GT boxes performs quite good. Could you please give me some advice concerning using my own RPN model?

from regionclip.

@whhong5 According to your descriptions, the RPN you loaded from RegionCLIP seems a bit off. If your own RPN is using exactly the same module as RegionCLIP's RPN, please check the weight loading. If not, you might need to additionally check the module definition & preprocess & feedforward functions.

from regionclip.