llama.cpp run examples/chat-13B.bat with llama-13B model and cuda crashed. about llama.cpp HOT 5 CLOSED

Can you try changing the GGML_ASSERT in ggml_cuda:2392 to CUDA_CHECK(err);? That should give more information about the error.

from llama.cpp.

having the exact same issue only with hip instead of cuda
changed that in the file and got this

ggml_cuda_compute_forward: RMS_NORM failed
CUDA error: invalid device function
  current device: 0, in function ggml_cuda_compute_forward at C:/Users/gaming/src/llama.cpp/ggml-cuda.cu:2392
  err

from llama.cpp.

"invalid device function" suggests that the code was not compiled for the device being used, or with a version of the CUDA toolkit not compatible with the installed driver version.

from llama.cpp.

Can you try changing the GGML_ASSERT in ggml_cuda:2392 to CUDA_CHECK(err);? That should give more information about the error.

Thanks. Found the problem:

ml_cuda_compute_forward: RMS_NORM failed
CUDA error: the provided PTX was compiled with an unsupported toolchain.
  current device: 0, in function ggml_cuda_compute_forward at F:\LLM\llama.cpp\ggml-cuda.cu:2392
  err
GGML_ASSERT: F:\LLM\llama.cpp\ggml-cuda.cu:60: !"CUDA error"

from llama.cpp.

I downgrade the cuda version. It works now. But while answering, cuda seems does not works?

from llama.cpp.