Question about the seg-token mask computation about groundinglmm HOT 1 CLOSED

Hi @saunair,

Thank you for reaching out and for your interest in our work.

Regarding your question about the line mask = input_ids[:, 1:] == self.seg_token_idx: this implementation choice relates to the training of LLM, ie. Vicuna - causal (autoregressive) model in our case, where each token is predicted based on the preceding tokens.

During training, the labels are typically shifted to the right (example) so that the model learns to predict the next token in the sequence. This shifting is reflected in how the model aligns its predictions with the input sequence. As a result, the output hidden states are offset compared to the input Ids by one position.

Therefore, when creating a mask, we essentially have to align the mask with these output hidden states. I hope this clarifies your question. Feel free to reach out with any further questions, thank you.

from groundinglmm.