ys-zong / vl-icl Goto Github PK

can I sft MLLM with your train split?

Good job, your work inspire me a lot.

Here is some little question about dataset.

1. I don't see the train test split from dataset link.
2. I don't know how to orgnize in contex sample from dataset. There seems is only sample one by one, not few shot.

Looking forward to get your responce!

Here is my prompt

Induce the concept from the in-context examples. Answer the question with a single word or phase.
We name this is a slation
We name this is a dax
Based on the in-context examples, we name this is a

gt: blicket

model: School bus

But if i change the last part of question to "Can you name this is a blicket or a dax?"
The Model will say "blicket".

I want to know is this prompt ok for VL-ICL benchmark?

Thanks for the awesome paper and repo!

I was trying out LLaVA and noticed that the model predictions were always empty strings, and was able to narrow it down to this line

It seems LLaVA already outputs only the tokens it generated, and not the whole context tokens + generated tokens.

The fix is quite easy, just decode the whole generated sentence, without truncating it first with the input_token_len.

Is my thinking correct or am I missing something?

Could you please provide the requirements.txt for llava?

Thanks!