Comments (6)
The binary that is used is determined at runtime. This means in your case there are two modes of failures:
- the CUDA driver is not detected (
libcuda.so
) - the runtime library is not detected (
libcudart.so
)
Both libraries need to be detected in order to find the right library for the GPU/CUDA version that you are trying to execute against.
Could you please provide use with the output of python -m bitsandbytes
for the GPU job that you are trying to run?
from bitsandbytes.
I think I'm in a similiar boat trying to get this to engage with the python version of dreambooth-sd
from bitsandbytes.
hi, same issue, exact same use-case too haha
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link
================================================================================
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/nvidia/lib64')}
"WARNING: The following directories listed in your path were found to "
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:99: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain libcudart.so as expected! Searching further paths...
f'{candidate_env_vars["LD_LIBRARY_PATH"]} did not contain '
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('module'), PosixPath('//matplotlib_inline.backend_inline')}
"WARNING: The following directories listed in your path were found to "
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractive SHELL=/bin/bash')}
"WARNING: The following directories listed in your path were found to "
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')}
"WARNING: The following directories listed in your path were found to "
WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
CUDA SETUP: Loading binary /opt/conda/lib/python3.7/site-packages/bitsandbytes/libbitsandbytes_cpu.so...
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cextension.py:49: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
"The installed version of bitsandbytes was compiled without GPU support. "
I know Im using a cuda-enabled machine
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.85.02 Driver Version: 510.85.02 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:01:00.0 Off | N/A |
| 30% 31C P8 20W / 350W | 1MiB / 24576MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
from bitsandbytes.
I believe this is fixed in the latest version. It prints instructions on how to debug the situation and alternatively prints out compilation instructions which should fix the issue.
from bitsandbytes.
I had the same issue, (nvidia-smi had proper output, but was still getting this error)
- I've downloaded cuda drivers from nvidia's website, and followed their instructions:
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-12-0_12.0.1-1_amd64.deb
sudo dpkg -i cuda-12-0_12.0.1-1_amd64.deb
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-drivers_525.85.12-1_amd64.deb
wget https://developer.download.nvidia.com/compute/cuda/repos/$distro/$arch/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt-key del 7fa2af80
sudo apt-get install cuda
- it failed and messed up my debian package graph, (I've done a bunch of below, it will be different on other systems but just to document)
sudo apt --fix-broken install
sudo systemctl stop nvidia-powerd
sudo apt-get remove nvidia-kernel-common-515
sudo dpkg -r nvidia-compute-utils-525
sudo dpkg -i --force-overwrite /var/cache/apt/archives/nvidia-compute-utils-525_525.85.12-0ubuntu1_amd64.deb
sudo dpkg --configure -a
sudo apt-get install -f
- Once everything is sorted out, rebooted the system:
sudo reboot
Now it works.
from bitsandbytes.
Bitsandbytes was not supported windows before, but my method can support windows.(yuhuang)
1 open folder J:\StableDiffusion\sdwebui,Click the address bar of the folder and enter CMD
or WIN+R, CMD 。enter,cd /d J:\StableDiffusion\sdwebui
2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes
3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows
4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl
Replace your SD venv directory file(python.exe Folder) here(J:\StableDiffusion\sdwebui\py310)
from bitsandbytes.
Related Issues (20)
- problem with loading my finetuned Llama2 model - type object 'Params4bit' has no attribute 'from_prequantized' HOT 3
- undefined symbol: cdequantize_blockwise_fp32 HOT 1
- AnimateDiff SDXL won't run
- AttributeError: 'NoneType' object has no attribute 'cquantize_blockwise_fp16_nf4' HOT 5
- Could not load bitsandbytes native library: 'NoneType' object has no attribute 'split' HOT 1
- ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`
- RuntimeError: Something when wrong when trying to find file. Maybe you do not have a linux system? HOT 1
- Error in Windows HOT 2
- please provide python whel package in nvidia jetson agx orin (aarch64 + cuda) HOT 1
- Exact version match required between the system and PyTorch CUDA libraries for the compilation to succeed HOT 1
- Error invalid device ordinal at line 359 in file /opt/bitsandbytes/csrc/pythonInterface.c
- Request for AdamW8bit support on CPU (would help TorchTune) HOT 5
- 8bit CAME optimizer
- "Only Tensors of floating point and complex dtype can require gradients", on FSDP, Accelerate, quatization HOT 1
- Is it possible to enable fused op F.gemv_4bit in F.gemv_4bit backward? HOT 1
- OSError: libcusparse.so.11: cannot open shared object file: No such file or directory CUDA Setup failed despite CUDA being available.
- out kwarg in matmul_4bit() is not working HOT 2
- load_in_8bit hangs on ROCm HOT 3
- RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_mm) HOT 2
- NotImplementedError: igemmlt not available (probably built with NO_CUBLASLT)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bitsandbytes.