GLaMM-FullScope model generates only a single mask about groundinglmm HOT 2 CLOSED

Hi @preddy5,

Thank you for your interest in our work.

Regarding your first question, it seems there's been a mix-up with the segmentation token index. In the model checkpoints we've provided, the correct seg_token_idx is actually 32004, not 29871. You can verify this by checking the value of args.seg_token_idx in train.py. Based on your generated_output_ids, 32004 does indeed appear three times, which aligns with the expected behavior for generating multiple masks. The issue might lie within the mask decoder's processing. Could you double-check how the masks are being decoded and ensure that it correctly interprets each occurrence of 32004? This should resolve the issue with mask generation.

For your second question, the discrepancy you've noticed is indeed expected due to the way the tokenizer handles special tokens within different contexts. When you tokenize "[SEG]" directly, the tokenizer recognizes it as a special token and assigns its specific ID, 32004 in this case. However, when "a [SEG]" is tokenized, the "[SEG]" is not recognized as a standalone special token due to the preceding text, leading to a different tokenization outcome. This behavior is by design, to allow the tokenizer to differentiate between special tokens and regular text.

I hope this clarifies your queries. Apologies for the delayed response, and I'll make sure to be more prompt in the future.

from groundinglmm.

Hey @hanoonaR
Thank you for the response.
I model works like a charm generating expected results after changing the seg_token_idx to 32004.
Thank you again for the clarification. I appreciate you helping with my query.

Regards,
Pradyumna.

from groundinglmm.