Some fixes to make it work about rbdsr HOT 5 CLOSED

Thank you very much for your tests. I tested it on XenServer 6.6 with Ceph Infernalis only. Tests with XenServer 7 and Jewel was only in plans, so thank you again for your help.

As for XenServer 7 kernel, I had the similar problems with XenServer 6.6. The default RBD kernel module didn't support ceph with enabled cache tiering and RBD mapping didn't work. I managed to fix it by rebuilding libceph.ko and rbd.ko modules. I took sources of these modules from kernel-3.10.0-229 of CentOS 7.0-1503, put them into kernel-3.10.83 source tree of XSenServer 6.6 and rebuilt the modules. Using these rebuilt modules I could map RBDs with enabled cache tiering. I guess this method can help in your case too.
Anyway, I don't think that this method is good. I'm not expert in the linux kernel and not sure that these rebuilt modules won't cause any other problems. It would be much better if XenServer developers could upgrade the kernel.

from rbdsr.

Hi,
You can make the warning

 health HEALTH_WARN
        crush map has legacy tunables (require bobtail, min is firefly)

go away without making any changes to CRUSH by adding the following option to your ceph.conf [mon] section:

mon warn on legacy crush tunables = false

You can read about it here http://docs.ceph.com/docs/master/rados/operations/crush-map/#tunables

Also you can find there compatibility matrix between Linux Kernel versions and CRUSH tunables profiles

from rbdsr.

Yes, I found this option too.
I also found you don't have to disable all features, you can go up to firefly features and only disable one:

sudo ceph osd crush tunables firefly
sudo ceph osd getcrushmap -o /tmp/crush
sudo crushtool -i /tmp/crush --set-chooseleaf_vary_r 0 -o /tmp/crush.new
sudo ceph osd setcrushmap -i /tmp/crush.new

Other features can be disabled on the fly:
sudo rbd feature disable RBD_XenStorage-985b47a3-5c0a-40b1-9149-da96c917f7bc/VHD-8b00e4e6-7865-47cd-aafd-20f7f883433b exclusive-lock object-map fast-diff deep-flatten

You can look which features are enabled on an RBD VHD by calling:
sudo rbd info RBD_XenStorage-985b47a3-5c0a-40b1-9149-da96c917f7bc/VHD-8b00e4e6-7865-47cd-aafd-20f7f883433b
The only enabled feature should be "layering"

I also found a wrong line in my patch. The line
return self._snapshot(self, sr_uuid, vdi_uuid).get_params()
must be corrected to
return self._snapshot(sr_uuid, vdi_uuid).get_params()

from rbdsr.

I have also tried this on Xenserver 7. Thanks so much Roman for writing this. I had the same issues as mentioned in this thread and fixed some of them but i did't manage to fix the clone issue so thanks for the patch for that scpcom.

I also have a suggestion to a change of the code to be able to work around the issue with the kernel to being old. Unfortunately i think it will always be behind in a product like this. There is an rbd client that uses a nbd to map the device to the kernel and the rbd code lies in userspace. I think the code would be about the same because the only difference when mapping is that we will need the nbd-rbd package and use the nbd-rbd map command instead of rbd map, and the device will be something like /dev/nbd0. I think all other commands will be the same.
I think i'm capable to write the code myself and i'm willing to do it and provide a patch.
The only thing that is required is that the nbd module is in the kernel and I haven't got access to my Xenserver machine to check that right now, but I will check that later.
If that is present do you want me to write the code for it?
I guess the best way would be to make this a configurable option so mabye the default would be tne kernel rbd and this another optional way. That would maybe a bit harder for me since i'm not that familiar with this kind of code, but i will try if you want me to.

from rbdsr.

HI,
Issues #1 and #2 have been fixed.
In 'fuse' mode it should support all Jewel's features.

from rbdsr.