Question: How to deploy on CoreOS? about volplugin HOT 14 CLOSED

It should be possible, but we're not testing it currently. A few other people have attempted this and this is being (loosely) tracked here: #66.

See #102 for specific problems related to CoreOS right now. I am going to raise the issue with the CoreOS folks and see if I can get some better answers as to why the rbd tool is not available in the host environment, which we depend on for most of our ceph-related operations.

I'm currently vacationing and will be available to answer questions, but I can't really do much development ATM. I return at the end of January.

I'll review this once I get back and see if I can come up with a good set of parameters for you to try out.

from volplugin.

Thanks for the fast reply, and have a nice vacation!

from volplugin.

@erikh I think a good way to do this would be to use the rkt fly stage1. Then you can include rbd, volplugin, etc inside of a container and run it as a container.

from volplugin.

cc @steveej @jonboulle

from volplugin.

@philips OK, I will try that when I get back!

from volplugin.

Thanks!

from volplugin.

Please ping us on rkt-dev or the GH repo to let us know how you go.

from volplugin.

@jonboulle will do -- still here for another week, and I plan to get to this as soon as time permits when I get back -- top priority.

from volplugin.

I did some research on this topic and I ended up creating this project https://github.com/h0tbird/docker-volume-rbd (yp-engineering/rbd-docker-plugin fork). I did some attempts with rkt but at that time fly was not available so I used systemd-nspawn. Whenever rkt fly hits alpha I will try it.

from volplugin.

@h0tbird Thanks a lot for your comment. I looked at the project, and saw that you're using the h0tbird/ceph:v9.2.0-2 image. Would it also be possible to use official ceph/daemon image?

from volplugin.

@tobilg Yes h0tbird/ceph is just a wrapper around ceph/daemon with a patched /usr/sbin/ceph-disk I had the need to fix 2 bugs and to lock my container version to v9.2.0-2.

from volplugin.

I forked h0tbird/docker-volume-rbd and updated the driver imports for volume to docker/go-plugins-helpers/volume as calavera/dkvolume is outdated.

I also added an install.sh script which will compile the driver and put it in /opt/bin, and created a systemd unit for the plugin.

Works nicely for me in our CoreOS cluster. You can have a look at tobilg/docker-volume-rbd if you like. This is a quick hack and I'm a beginner, so please bare with me for my code :-)

from volplugin.

@tobilg @h0tbird I believe I have an answer for you now (I got back from China yesterday), hopefully you will find it useful.

docker run --pid host --net host -v /var/lib/ceph:/var/lib/ceph -v /etc/ceph:/etc/ceph -v /var/run/ceph:/r/run/ceph -v /sys:/sys --privileged -v /dev:/dev -v /run/docker/plugins:/run/docker/plugins contiv/volplugin volplugin --debug

Will run volplugin --debug. It is backed by @h0tbird's ceph/rbd image (and therefore requires the permissions it needs to map volumes).

Note there is more to setting up volplugin than that such as etcd, volmaster, and volsupervisor. The vol* binaries plus the CLI tool (volcli) are included in the image, so you can do it all in containers if you want now. :)

Please read the docs we have over at http://docs.contiv.io so you can get familiar with the tool arguments. The build is today's master.

If you guys have time to try this out, I would appreciate it!

from volplugin.

#66 covers this, I see no point in leaving this open

from volplugin.