LLaMA3-8B won't inference about llama-factory HOT 6 CLOSED

Please first check if the original model can infer properly before training. Then check the log and GPU status when do inference process by the way.
My System is "win11, python 3.10, RTX 4090 , pytorch 2.1.2, Llama Factory 0.6.3", which is OK for both Llama2 and Llama3.

Bless!

from llama-factory.

Llama-3 model can generate texts in our Linux environment. I think it is likely an issue with your hardware and environment.

from llama-factory.

Llama-3 model can generate texts in our Linux environment. I think it is likely an issue with your hardware and environment.

``

from llama-factory.

I'm using the Smaller Llama3 model, I'm not trying to train on the larger 70b model. I'm on a 3090 rtx gpu, on windows, is windows ok? I'm not sure I'd be able to get this running on Linux... maybe I need to though.

I has WSL for windows to install stuff but I'm so new to it I have no idea how I'd get started

from llama-factory.

Please first check if the original model can infer properly before training. Then check the log and GPU status when do inference process by the way. My System is "win11, python 3.10, RTX 4090 , pytorch 2.1.2, Llama Factory 0.6.3", which is OK for both Llama2 and Llama3.

Bless!

windows is ok. please do more environment check and try the official inference demo of llama3 which is in https://huggingface.co/blog/llama3.

bless.

from llama-factory.

Please first check if the original model can infer properly before training. Then check the log and GPU status when do inference process by the way. My System is "win11, python 3.10, RTX 4090 , pytorch 2.1.2, Llama Factory 0.6.3", which is OK for both Llama2 and Llama3.

Updating pytorch from 2.0.2 to 2.1.1 resolved my problem.

from llama-factory.