Question on reproducing the evaluation/demo performance from pretrained models about groundinglmm HOT 2 CLOSED

Hi @hetong007,

Thank you for your interest in our work. The issue appears to be related to the environment. Please note that the code has been tested with torch==1.13.1+cu117 and transformers@git+https://github.com/huggingface/transformers.git@cae78c46.

I attempted running the same examples using the above environment configuration and GLaMM-FullScope model and the results are attached are as following,

To resolve the issue and reproduce our results, I recommend the following steps:

1. Follow the instructions at https://github.com/mbzuai-oryx/groundingLMM/blob/main/docs/install.md to recreate the environment. Ensure that you have the correct versions of PyTorch and Transformers installed.
2. Alternatively, you can download the conda environment directly from this Google Drive link and use it for running the demo and evaluations.

Please let me know if this resolves the issue. Thank you.

from groundinglmm.

Thanks @mmaaz60 , having reinstalled the correct version of transformers i can reproduce the results with latest CUDA and pytorch.

from groundinglmm.