Running on Multiple GPU? about aidungeon HOT 5 OPEN

That's a good point. Definitely something we should support.

from aidungeon.

Should as in currently do support? Or as in we should get to supporting it?

from aidungeon.

Should as in currently do support? Or as in we should get to supporting it?

I would assume he means that it’ll be supported later. There is no multi-GPU support at the moment.

from aidungeon.

I don't know if GPT-2 can be split across GPUs. Probably the best bet is to get quantization working or train a smaller model that uses less VRAM.

from aidungeon.

Multi GPU cannot split a model like that without a LOT of work, instead multiGPU would be used to handle multiple inputs at once (not something that this project would benefit from).

Essentially using model-parallelism is much more difficult to implement than data-parallelism and neither would work terribly well with GPT-2 without some significant changes.

from aidungeon.