Can you provide a download link for the pth file of the SAM model? about groundinglmm HOT 3 CLOSED

Hi @MenSanYan,

Thank you for your interest in our work. The SAM weights can be downloaded from their official GiTHub repository using the following link,

https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth

from groundinglmm.

Hi @mmaaz60 ! As I can read in the paper and after checking the code you fine-tuned those weights, correct? Could you provide the fine-tuned SAM weights? Or are these loaded automatically when running GLaMMForCausalLM.from_pretrained(...)? That would mean that we should actually pass --vision_pretrained None if we actually want to use the fine-tuned weights, right? Thank you!

from groundinglmm.

Hi @ekazakos,

The SAM encoder - referred to as the grounding encoder in the paper is frozen.

During pretraining: we freeze the Grounding encoder here.

When we finetune (when you use from_pretrained) set gad False here. As you can see in this line, we do not set grad True for the grounding_encoder.

Hope this helps.

from groundinglmm.