docker volume plugin about moosefs HOT 9 CLOSED

Finally I was able to answer your questions @antoinetran about scenarios

Scenarios

In following scenarios we are using directory /mnt/moosefs for sake of simplicity. You can mount your MooseFS storage in any directory you want to.

In all scenarios MooseFS Docker Cluster is used: https://github.com/moosefs/moosefs-docker-cluster

MooseFS should be mounted to create a volume

Directory /mnt/moosefs exist but is not mounted and we create the container

Preconditions

• Empty directory exists: /mnt/moosefs
• MooseFS is not mounted on host

Empty directory exists: /mnt/moosefs

Command:

ls -l /mnt/moosefs

outputs:

total 0

The result of

tree /mnt/

is

/mnt/
└── moosefs
1 directory, 0 files

MooseFS is not mounted on host

Command  mount | grep moosefs  gives no results, so there are no mounts available

Expected result

Error, volume should not be created

Actual result

If we will try to create a volume we will get following error:

docker volume create -d moosefs --name mymoosefs -o mountpoint=/mnt/moosefs
Error response from daemon: create mymoosefs: VolumeDriver.Create: Cannot create volume /mnt/moosefs as it's not a valid MooseFS mount

Can't create volume when mount directory doesn't exist

Directory /mnt/moosefs does not exist and we create the container

Preconditions

• Directory /mnt/moosefs doesn't exist

Directory doesn't exist: /mnt/moosefs

Command:

ls -l /mnt

outputs:

total 0

The result of

tree /mnt

is

0 directory, 0 files

Expected result

Error, volume should not be created

Actual result

If we will try to create a volume we will get following error:

docker volume create -d moosefs --name mymoosefs -o mountpoint=/mnt/moosefs
Error response from daemon: create mymoosefs: VolumeDriver.Create: Cannot create volume /mnt/moosefs as it's not a valid MooseFS mount

MooseFS Client killed when volume is mounted

MooseFS is mounted in /mnt/moosefs, we have a volume mapped, but we then have a MooseFs client crash

Preconditions

• Directory /mnt/moosefs exist
• MooseFS is mounted in /mnt/moosefs
• Docker Volume is attached to /mnt/moosefs
• Run container using volume
• MooseFS Client is killed

Directory exists: /mnt/moosefs

Command:

ls -l /mnt/moosefs

outputs:

total 0

MooseFS is mounted in /mnt/moosefs

Command 

mount | grep moosefs

gives following result:

mfsmaster:9421 on /mnt/moosefs type fuse.mfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)

The result of

tree /mnt/

is

/mnt/
└── moosefs
    └── welcome_to_moosefs.txt
1 directory, 1 file

Docker Volume is attached to /mnt/moosefs

Volume is mounted using following command:

docker volume create -d moosefs --name mymoosefs -o mountpoint=/mnt/moosefs

Response:

mymoosefs

List volumes:

docker volume ls
DRIVER              VOLUME NAME
moosefs             mymoosefs

Run container using volume

Run example nginx container which will mount 

docker run -ti -v mymoosefs:/usr/share/nginx/html -p 10080:80 nginx:latest bash -c "service nginx start;bash"

MooseFS Client is killed

sudo killall -9  mfsmount

Expected result

Actual result

If we will try to list files in the container: (volume is mounted in /usr/share/nginx/html)

ls -l /usr/share/nginx/html
ls: cannot access '/usr/share/nginx/html': Transport endpoint is not connected

Attempt to use subdirectory as volume

MooseFS is mounted in /mnt/moosefs, we create a container mapped to /mnt/moosefs/subdir1 that does not exist

Preconditions

• Directory /mnt/moosefs exists
• MooseFS is mounted in /mnt/moosefs
• /mnt/moosefs/subdir1 doesn't exist

Directory exists: /mnt/moosefs

Command:

ls -l /mnt/moosefs

outputs:

total 0

MooseFS is mounted in /mnt/moosefs

Command 

mount | grep moosefs

gives following result:

mfsmaster:9421 on /mnt/moosefs type fuse.mfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)

The result of

tree /mnt/

is

/mnt/
└── moosefs
    └── welcome_to_moosefs.txt
1 directory, 1 file

/mnt/moosefs/subdir1 doesn't exist

List files of /mnt/moosefs/subdir1 to check if it exist:

ls -l /mnt/moosefs/subdir1
ls: cannot access '/mnt/moosefs/subdir1': No such file or directory

Expected result

Error, volume should not be created

Actual result

If we will try to create a volume we will get following error:

docker volume create -d moosefs --name mymoosefs -o mountpoint=/mnt/moosefs/subdir1
Error response from daemon: create mymoosefs: VolumeDriver.Create: Cannot create volume /mnt/moosefs/subdir1 as it's not a valid MooseFS mount

No MooseFS Client on host machine

MooseFS Client is installed and mounted in a container, and no MooseFS package is installed in host, and we create the container

MooseFS Client is required on host machine for now, we plan to provide Docker Volume Plugin which will not need installing additional packages on host machine. The plugin will be a client for MooseFS

from moosefs.

@tivvit, @muhh - please check this out! :)

from moosefs.

@OXide94 that is really nice, but I was talking about something else.

I was talking about Docker volume plugin, which means you can mount MooseFs in Docker image (like this https://infinit.sh/documentation/docker/volume-plugin)

I am willing to help

from moosefs.

@karolmajek FYI

from moosefs.

@tivvit, oh of course you're right, I've just somehow omitted "volume" while reading your post :)

from moosefs.

Hi @tivvit,
please see this: https://github.com/moosefs/docker-volume-moosefs.
Tests and comments are appreciated!

Thanks,
Best regards,
Peter / MooseFS Team

from moosefs.

Wow! Great job, I will try that and let you know

from moosefs.

Thank you for the job! I will try to test it in a few days. But after reading the README, there are some behaviors I do not understand yet, in the scenario:

1. that /mnt/moosefs exist but is not mounted and we create the container
2. that /mnt/moosefs does not exist and we create the container
3. that /mnt/moosefs is mounted, we have a container mapped to this, but we then have a MooseFs client crash
4. that /mnt/moosefs is mounted, we create a container mapped to /mnt/moosefs/subdir1 that does not exist?
5. that we have a MooseFs client in a container in our project, and no Mfs rpm installed in host, and we create the container

In anycase I will also try to answer this in my future tests.

from moosefs.

@karolmajek, can you please answer @antoinetran questions?
@antoinetran please share with us your tests results and if you would have further questions, we will be happy to help!

Thanks,
Peter / MooseFS Team

from moosefs.