T:\Projects\Chat-UniVi\main_demo_7B.py:17: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  assert model_path is not ""
False
===================================BUG REPORT===================================

C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes\cuda_setup\main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
warn(msg)
The following directories listed in your path were found to be non-existent: {WindowsPath('C'), WindowsPath('/Users/firok/.conda/envs/chatunivi/lib')}

C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes\cuda_setup\main.py:166: UserWarning: C:\Users\firok.conda\envs\chatunivi did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...

warn(msg)

CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...

The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}

DEBUG: Possible options found for libcudart.so: set()

CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 7.5.

CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md

CUDA SETUP: Loading binary C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.so...

argument of type 'WindowsPath' is not iterable

CUDA SETUP: Problem: The main issue seems to be that the main CUDA runtime library was not detected.

CUDA SETUP: Solution 1: To solve the issue the libcudart.so location needs to be added to the LD_LIBRARY_PATH variable

CUDA SETUP: Solution 1a): Find the cuda runtime library via: find / -name libcudart.so 2>/dev/null

CUDA SETUP: Solution 1b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_1a

CUDA SETUP: Solution 1c): For a permanent solution add the export from 1b into your .bashrc file, located at ~/.bashrc

CUDA SETUP: Solution 2: If no library was found in step 1a) you need to install CUDA.

CUDA SETUP: Solution 2a): Download CUDA install script: wget https://github.com/TimDettmers/bitsandbytes/blob/main/cuda_install.sh

CUDA SETUP: Solution 2b): Install desired CUDA version to desired location. The syntax is bash cuda_install.sh CUDA_VERSION PATH_TO_INSTALL_INTO.

CUDA SETUP: Solution 2b): For example, "bash cuda_install.sh 113 ~/local/" will download CUDA 11.3 and install into the folder ~/local

Traceback (most recent call last):

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\transformers\utils\import_utils.py", line 1099, in get_module

return importlib.import_module("." + module_name, self.name)

File "C:\Users\firok.conda\envs\chatunivi\lib\importlib_init.py", line 126, in import_module

return _bootstrap.gcd_import(name[level:], package, level)

File "", line 1050, in gcd_import

File "", line 1027, in find_and_load

File "", line 1006, in find_and_load_unlocked

File "", line 688, in load_unlocked

File "", line 883, in exec_module

File "", line 241, in call_with_frames_removed

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\transformers\models\llama\modeling_llama.py", line 32, in 

from ...modeling_utils import PreTrainedModel

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\transformers\modeling_utils.py", line 38, in 

from .deepspeed import deepspeed_config, is_deepspeed_zero3_enabled

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\transformers\deepspeed.py", line 37, in 

from accelerate.utils.deepspeed import HfDeepSpeedConfig as DeepSpeedConfig

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\accelerate_init.py", line 3, in 

from .accelerator import Accelerator

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\accelerate\accelerator.py", line 35, in 

from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\accelerate\checkpointing.py", line 24, in 

from .utils import (

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\accelerate\utils_init.py", line 131, in 

from .bnb import has_4bit_bnb_layers, load_and_quantize_model

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\accelerate\utils\bnb.py", line 42, in 

import bitsandbytes as bnb

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes_init.py", line 6, in 

from . import cuda_setup, utils, research

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes\research_init.py", line 1, in 

from . import nn

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes\research\nn_init.py", line 1, in 

from .modules import LinearFP8Mixed, LinearFP8Global

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes\research\nn\modules.py", line 8, in 

from bitsandbytes.optim import GlobalOptimManager

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes\optim_init.py", line 6, in 

from bitsandbytes.cextension import COMPILED_WITH_CUDA

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\bitsandbytes\cextension.py", line 20, in 

raise RuntimeError('''

RuntimeError:

CUDA Setup failed despite GPU being available. Please run the following command to get more information:
    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

The above exception was the direct cause of the following exception:
Traceback (most recent call last):

File "T:\Projects\Chat-UniVi\main_demo_7B.py", line 4, in 

from ChatUniVi.conversation import conv_templates, Conversation

File "T:\Projects\Chat-UniVi\ChatUniVi_init_.py", line 1, in 

from .model import ChatUniViLlamaForCausalLM

File "T:\Projects\Chat-UniVi\ChatUniVi\model_init_.py", line 1, in 

from .language_model.llama import ChatUniViLlamaForCausalLM, ChatUniViConfig

File "T:\Projects\Chat-UniVi\ChatUniVi\model\language_model\llama.py", line 5, in 

from transformers import AutoConfig, AutoModelForCausalLM, 

File "", line 1075, in _handle_fromlist

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\transformers\utils\import_utils.py", line 1090, in getattr

value = getattr(module, name)

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\transformers\utils\import_utils.py", line 1089, in getattr

module = self._get_module(self._class_to_module[name])

File "C:\Users\firok.conda\envs\chatunivi\lib\site-packages\transformers\utils\import_utils.py", line 1101, in _get_module

raise RuntimeError(

RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
    CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues