Cannot start container: no space left on device (cpuset cgroup) about runc HOT 17 CLOSED

I have this issue as well. The workaround is to run the following command before starting a Docker container:

# echo 0 >  /sys/fs/cgroup/cpuset/system.slice/cpuset.mems

from runc.

I think docker-archive/libcontainer#642 alread fixed it, you can try again after moby/moby#14693 is merged.

from runc.

Oh, I didn't see that PR. Pretty sure it fixes the problem. I'll try again when libcontainer is updated and reopen if I still have problems.

Thanks!

from runc.

In the process of testing a backport of docker-archive/libcontainer#642 into docker 1.7.1 it seems the problem is not fully solved for me.

If I reproduce the issue with a version of docker that includes the fix, docker will produce the same error. If I run docker again, the issue is then resolved until the next restart of the machine.

I've found that both 1.7.1 (with backported fix) and 1.8.0-rc1 will behave identically in this regard.

from runc.

http://permalink.gmane.org/gmane.linux.kernel.cgroups/13990

Applied to cgroup/for-4.2-fixes w/ stable cc'd.

https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git/commit/?h=for-4.2-fixes&id=24ee3cf89bef04e8bc23788aca4e029a3f0f06d9

Merged in mainline: torvalds/linux@24ee3cf

from runc.

A boot2docker upgrade fixed this for me by upgrading me to v1.8.1

from runc.

@pbx0 Is this problem still existed? Can you show the steps how you reproduce the issue?

from runc.

@alban 's patch did fix the kernel bug in 3.17+, @pbx0 if you are using kernel newer than 3.17, probably should merge that kernel fix and try again.

from runc.

@hqhq I can confirm alban's kernel patch fixed it for me. Forgot to come back here and say so.

from runc.

I'll close this one since you reported back the kernel update fixed it for you. Thanks!

from runc.

echo 1 > /sys/fs/cgroup/docker/cgroup.clone_children did the trick on Debian wheezy with linux kernel v3.16 with docker v1.8.3

from runc.

Unfortunately, problem is still present (Debian wheezy, kernel 3.16.0-0.bpo.4-amd64, Docker 1.9.1)

from runc.

@truebit in my case, Wheezy, kernel 3.16.0-0.bpo.4-amd64, Docker 1.9.1, just like @romuald's your fix doesn't do the trick. You're running Docker 1.8.3 though.

$ cat /sys/fs/cgroup/docker/cgroup.clone_children
1

I have stop/start docker and I am trying to run a (progrium/consul)[https://hub.docker.com/r/progrium/consul/] container fails with the following:

95003f34b7dba95e6dd63a96774ee3505027bb34529dfc7a22999b3d93c57779
Error response from daemon: Cannot start container 95003f34b7dba95e6dd63a96774ee3505027bb34529dfc7a22999b3d93c57779: [8] System error: write /sys/fs/cgroup/docker/95003f34b7dba95e6dd63a96774ee3505027bb34529dfc7a22999b3d93c57779/cgroup.procs: no space left on device

@romuald have you found any fix for this? Thanks :).

from runc.

There's something wrong with the node in question. On another Wheezy/Docker 1.9.1 works. My node needs a restart.

from runc.

I see this issue of cannot start containers on "3.19.0-25-generic".
Do we know if the patch is present in this kernel version? or it is only on 4.2 and mainline?

from runc.

I still get this issue on ubuntu trusty 14.04/docker 17.07

from runc.

Is this still an issue? I see a lot of commits above that mention fixing this issue

from runc.