How model parallelize across GPUs? about nondeepnetworks HOT 5 OPEN

You can use PyTorch Lightning instead. It automatically parallelizes the model training across GPUs and also supports TPU with just a single argument.

from nondeepnetworks.

We use the nccl backend with PyTorch to parallelize the stream while inference (testing). For training, we use usual distributed setup.

from nondeepnetworks.

So we need at least three GPUs to inference three streams in ParNet?

from nondeepnetworks.

Yes if you want to do the multi-GPU inference. Otherwise, you can also do single gpu inference but it will be slower.

from nondeepnetworks.

@imankgoyal

foe the edge devcie, using mulit-gpu for inference is expensive, what is your opinion?

from nondeepnetworks.