[BUG] Cannot run deepspeed in containers about deepspeed HOT 6 OPEN

@zxti - that's odd, this shouldn't be the case. Could you share a sample docker container from NGC that you're using so I can test and try to repro on our side?

from deepspeed.

@loadams I'll check which docker container I was specifically using.

But for now - you can also repro in Colab, let me know if this helps: https://colab.research.google.com/drive/1vpmay34Wfc31ilOHSB4G6WIGsmw8H0-0?usp=sharing

from deepspeed.

It was nvcr.io/nvidia/pytorch:23.12-py3

from deepspeed.

I encountered the same issue when running the CIFAR example inside the Docker image, and the solution to the problem was to run the image with the --privileged mode.

$ docker run -it --rm --privileged ...

from deepspeed.

@zxti - does that help resolve the issue for you?

from deepspeed.

Hi @loadams, unfortunately no, since this is what I mentioned in the initial post - you can work around the issue if you have privileged access to the host, but in very many environments (such as RunPod, Colab, etc.) you don't have this.

from deepspeed.