fail to apply on llama-13b about svd-llm HOT 2 CLOSED

Hi, thank you for your feedback! In this case, it's better to use double() precision rather than the float() precision to run cholesky decomposition like the following way:
scaling_diag_matrix = torch.linalg.cholesky(raw_scaling_diag_matrix.double())

I have updated the code to fix this potential problem. You can compress LLaMA-13b using the following command:
python SVDLLM.py --model jeffwan/llama-13b-hf --step 1 --save_path "./"

If you still met the same problem when running the new code, please reopen this issue.

from svd-llm.

I still got the same problem.
Traceback (most recent call last):
File "/home/xxx/SVD-LLM/SVDLLM_new.py", line 193, in whitening
scaling_matrix_inv = torch.linalg.inv(scaling_diag_matrix)
torch._C._LinAlgError: linalg.inv: The diagonal element 6940 is zero, the inversion could not be completed because the input matrix is singular. "
My python environment is built on requirements.txt. And I run the code on 2 3090 GPUs

from svd-llm.