caoyichao / unihoi Goto Github PK

Could you tell me how do you set the memory_key_padding_mask and the pos_embed of HO Prompt-guided Decoder?
I would be grateful if you can explain that.

Hi,
We have recently considered using a two-stage method for Zero-Shot HOI Detection, but we have some confusion about the detailed setup.
You use the following formula for Zero-Shot HOI Detection:

My understanding is that the text features (600) do similarity calculations with the final interaction representation to get the interaction categories.

1. unseen combination (UC) has seen all verbs and objects. The two-stage method seems to handle this case directly. Alternatively, the "similarity computation" is still used.

2. Unseen Objects (UO) Have not seen a few objects.
   We found that in GEN-VLKT (hico_text_label.py), there are only interaction categories related to unseen objects. So, are unseen objects used as GTs in training for the object detection task? Does this part still use "similarity computation"?

3. Unseen actions (UV) have not seen few verbs. Does this part still use "similarity computation"?

Do you use the features output by Qformer as a classifier or the features output by opt 2.7B as a classifier?

Hi,
Thanks for the awesome work, I've been following UniHOI for a bit now. I am wondering if you have any updates or estimated timeline regarding when the source code might be released? Looking forward to your response.