RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())] about video-chatgpt HOT 5 CLOSED

Hi @rjccv

Thank you for your interest in our work. The error seems to be generated because of either the corrupted checkpoints or a different transformer version. Can you double check if the error persists after installing the transformers using the following command?

pip install transformers@git+https://github.com/huggingface/transformers.git@cae78c46

Further, please provide the environment and hardware details you are using to comment further. Thanks

from video-chatgpt.

Thank you for the quick response. When running this line, a warning in the git log mentions that the branch or tag cae78c46 could not be found:

This is the version that appears as installed when I check run conda list transformers

In LLaVA-7B-Lightening-v1-1/config.json, the "transfromers_version" variable matches the output of my conda environment, so I don't believe that is the cause.

{
"_name_or_path": "liuhaotian/LLaVA-Lightning-7B-delta-v1-1",
"architectures": [
"LlavaLlamaForCausalLM"
],
...
"transformers_version": "4.28.0.dev0",
"tune_mm_mlp_adapter": false,
"use_cache": false,
"use_mm_proj": true,
"vocab_size": 32003
}

I did notice though in LLaVA-7B-Lightening-v1-1/tokenizer_config.json that there is a variable "special_tokens_map_file" that points to a checkpoint folder not present in my directory. Could this be causing the issue? If I need this installed, where can I download it from?

{
...
  "special_tokens_map_file": "./checkpoints/vicuna-7b-v1-1/special_tokens_map.json",
  "tokenizer_class": "LlamaTokenizer",
  "unk_token": {
    "__type": "AddedToken",
    "content": "<unk>",
    "lstrip": false,
    "normalized": true,
    "rstrip": false,
    "single_word": false
  }
}

Alternatively, if this is not the issue and it is corrupted checkpoints, is there another method for downloading them? As mentioned in the link I referenced, huggingface never sends out an email for the Llama weights, so that doesn't seem like an option.

My conda environment is exactly as instructed in the readme and I am attempting to run this on one RTX 3080 Ti GPU.

from video-chatgpt.

Hi @rjccv
 
Can you try using special tokens maps from https://huggingface.co/lmsys/vicuna-7b-v1.1? Also please try loading the base vicuna model and see if the issue appears there as well. Thanks

from video-chatgpt.

Okay I have changed the path of "special_tokens_maps" to point to the local special_tokens_maps.json file since the two are the same. And I am able to load the base model without any errors when I replace the script as below:

    # # Load tokenizer
    # tokenizer = AutoTokenizer.from_pretrained(model_name)

    # # Load model
    # model = VideoChatGPTLlamaForCausalLM.from_pretrained(model_name, low_cpu_mem_usage=True, torch_dtype=torch.float16,
    #                                                      use_cache=True)

    tokenizer = AutoTokenizer.from_pretrained("lmsys/vicuna-7b-v1.1")
    model = AutoModelForCausalLM.from_pretrained("lmsys/vicuna-7b-v1.1", low_cpu_mem_usage=True, torch_dtype=torch.float16, 
                                                            use_cache=True)

from video-chatgpt.

I was able to resolve this. The bin files were indeed corrupted, but because I didn't have git-lfs installed globally. No error appeared after git lfs install because of the sequential executing command.

git lfs install
git clone https://huggingface.co/mmaaz60/LLaVA-7B-Lightening-v1-1

After installing this and properly downloading the weights, your model worked as expected. Anyway, thank you for all your help. I'm looking forward to playing around with your model.

from video-chatgpt.