Wonder why didn't implement multi-GPU training? about network-slimming HOT 6 CLOSED

Hi! Thanks for your interest in our code.

The reason why we didn't implement multi-gpu training is because all our experiments fitted on one GPU. I think multi-gpu can be used. How did you implement Multi-GPU?

I suggest using:
model = torch.nn.DataParallel(model).cuda()

from network-slimming.

Thank your for so quick answer!
Yes, I implement just as your suggestion.But, I found the training seem to stop at "output = model(data)" and never continue. At the same time, GPUs are always at 100% utilization rate.
This the my current suitation, and it will be wonderful if you can say something enlighten me.
非常谢谢！

from network-slimming.

Can you share with me what is the size of your GPU memory and which network are you running (VGG, ResNet or DenseNet)?

I just tried adding model = torch.nn.DataParallel(model).cuda() and it worked perfectly. I can run the code on two GPUs.

from network-slimming.

| 2 TITAN X (Pascal) Off | 00000000:08:00.0 Off | N/A |
| 27% 50C P2 78W / 250W | 597MiB / 12189MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 3 TITAN X (Pascal) Off | 00000000:09:00.0 Off | N/A |
| 32% 58C P2 83W / 250W | 597MiB / 12189MiB | 100% Default |

Hello, I run DenseNet and there are 5GPUs, above are the No,2&3 info.

I didn't modify anything except:
#if args.cuda:
#model.cuda()
model = torch.nn.DataParallel(model).cuda()

Whether the pytorch is the issuse, my version is:

torch.version
'1.0.1.post2'

from network-slimming.

Our code is developed based on Pytorch 0.3.1. There is no guarantee of our code on Pytorch 1.0.1. Maybe you should try changing to Pytorch 0.3.1.

from network-slimming.

Thank you so much.
I fixed this issue by using other GPU server and install Pytorch 0.3.1.

from network-slimming.