Which GPU did you use? about vit-pytorch HOT 5 OPEN

Distributed training on a single node can be executed as follows.

python3 -m torch.distributed.launch --nproc_per_node=NUM_OF_GPU train.py --train_batch_size BATCH_SIZE_PER_GPU --name cifar10-100_500 --dataset cifar10 --model_type ViT-B_16 --pretrained_dir checkpoint/ViT-B_16.npz 

from vit-pytorch.

@jeonsworld Hello, how can I use multi-GPUs?

There are DataParallel and Distributed ways to use multi-gpu in pytorch.
The current code supports distributed learning and uses the following command.

python3 -m torch.distributed.launch --nproc_per_node=NUM_OF_GPU train.py --train_batch_size BATCH_SIZE_PER_GPU --name cifar10-100_500 --dataset cifar10 --model_type ViT-B_16 --pretrained_dir checkpoint/ViT-B_16.npz 

from vit-pytorch.

I am also using this implementation, is it impossible to call distributed training?

from vit-pytorch.

Transfer learning was performed using v100. Check the relative time of the tensorboard for the learning time.

from vit-pytorch.

@jeonsworld Hello, how can I use multi-GPUs?

from vit-pytorch.