expected scalar type Half but found Float about alpaca-cot HOT 5 CLOSED

WORLD_SIZE=4 CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node=4 --master_port=3192 uniform_finetune.py
--data belle1m
--model_type bloom
--model_name_or_path bigscience/bloomz-7b1-mt
--lora_target_modules query_key_value
--per_gpu_train_batch_size 4
--learning_rate 3e-4
--epochs 1

from alpaca-cot.

what should I do

from alpaca-cot.

This may be caused by environmental issues, and we will quickly identify the cause and provide solutions.

from alpaca-cot.

We were unable to reproduce this bug. The code works correctly in A100 using the following command:

CUDA_VISIBLE_DEVICES=1,2,3,4  python3 -m torch.distributed.launch --nproc_per_node 4 --nnodes=1  \
    uniform_finetune.py   --model_type bloom --model_name_or_path bigscience/bloomz-7b1-mt \
    --data alpaca --lora_target_modules query_key_value \
    --per_gpu_train_batch_size 4 --learning_rate 3e-4 --epochs 1  

or

CUDA_VISIBLE_DEVICES=0 python3  uniform_finetune.py \
--data alpaca \
--model_type bloom \
--model_name_or_path bigscience/bloomz-7b1-mt \
--lora_target_modules query_key_value \
--per_gpu_train_batch_size 4 \
--learning_rate 3e-4 \
--epochs 1

from alpaca-cot.

It seems that this may be caused by a low-level version of Python. Try using version 3.9 and above.

from alpaca-cot.