VM not starting, get Hotplug.device.timeout about rbdsr HOT 9 CLOSED

Hi André,
You need to install qemu-dp from the extras_testing repository.
With the default installation of xcp-ng 7.5 qemu-dp does not support rbd.
https://github.com/xcp-ng/xcp/wiki/Ceph-on-XCP-ng-7.5-or-later

Also, you can read this
https://xcp-ng.org/forum/topic/4/ceph-on-xcp-ng

from rbdsr.

Hi!
Thanks for your quick answer. I was unclear with my installation instructions. I enabled the repos like this:

[xcp-ng-base]
name=XCP-ng Base Repository
baseurl=https://updates.xcp-ng.org/7/7.5/base/x86_64/
enabled=1
gpgcheck=0

[xcp-ng-updates]
name=XCP-ng Updates Repository
baseurl=https://updates.xcp-ng.org/7/7.5/updates/x86_64/
enabled=1
gpgcheck=0

[xcp-ng-extras]
name=XCP-ng Extras Repository
baseurl=https://updates.xcp-ng.org/7/7.5/extras/x86_64/
enabled=0
gpgcheck=0

[xcp-ng-updates_testing]
name=XCP-ng Updates Testing Repository
baseurl=https://updates.xcp-ng.org/7/7.5/updates_testing/x86_64/
enabled=0
gpgcheck=0

[xcp-ng-extras_testing]
name=XCP-ng Extras Testing Repository
baseurl=https://updates.xcp-ng.org/7/7.5/extras_testing/x86_64/
enabled=1
gpgcheck=0

A yum update installed the qemu updates. I have this on disk

[root@blade12 ~]# rpm -ql qemu-dp
/usr/lib64/qemu-dp/bin
/usr/lib64/qemu-dp/bin/qemu-dp
/usr/lib64/qemu-dp/bin/qemu-img
/usr/lib64/qemu-dp/bin/qemu-io
/usr/lib64/qemu-dp/bin/qemu-nbd
[root@blade12 ~]# rpm -qa qemu-dp
qemu-dp-2.10.2-1.2.0.extras.x86_64

So I hope I'm not missing something.

Kind regards,
André

from rbdsr.

Hi!
It seems qemu-dp is properly openining connections to ceph mons and osds.
tcp 0 0 xcp-ng-cilatfwb:56718 node1.intern.marca:6800 ESTABLISHED 27282/qemu-dp
tcp 0 0 xcp-ng-cilatfwb:47710 node2.intern.marca:6800 ESTABLISHED 27282/qemu-dp
tcp 0 0 xcp-ng-cilatfwb:38272 admin1.intern.marc:6800 ESTABLISHED 27282/qemu-dp
tcp 0 0 xcp-ng-cilatfwb:49768 admin1.intern:smc-https ESTABLISHED 27282/qemu-dp
unix 2 [ ] DGRAM 984694 27282/qemu-dp
unix 3 [ ] STREAM CONNECTED 988594 27282/qemu-dp

But I still cannot understand why it hangs. I see this error in an strace:
27282 bind(34, {sa_family=AF_LOCAL, sun_path="/var/run/qemu-nbd.3475cd94-8e0f-411f-a46b-611afc46a56c"}, 110) = 0
27282 listen(34, 1) = 0
27282 getpeername(34, 0x55951ad0dba0, [128]) = -1 ENOTCONN (Transport endpoint is not connected)
27282 getsockname(34, {sa_family=AF_LOCAL, sun_path="/var/run/qemu-nbd.3475cd94-8e0f-411f-a46b-611afc46a56c"}, [57]) = 0
.....
27282 write(2, "xen be: qdisk-768: ", 19) = 19
27282 write(2, "watching frontend path (/local/domain/2/device/vbd/768) failed\n", 63) = 63
27282 sendmsg(20, {msg_name(0)=NULL, msg_iov(1)=[{"{"return": {}}\r\n", 16}], msg_controllen=0, msg_flags=0}, 0) = 16

Perhaps you could sen me logs of a working v3 installation so I can compare it?

from rbdsr.

Hi,
Could you please check which version of glibc installed?

from rbdsr.

Hi!
Sorry for the delay, these are installed:
glibc-common-2.17-106.el7_2.4.x86_64
glibc-2.17-106.el7_2.4.x86_64

I've checked the yum.log and these are the packages which were installed after base installation:
Sep 19 23:28:43 Installed: centos-release-storage-common-2-2.el7.centos.noarch
Sep 19 23:28:43 Installed: centos-release-ceph-luminous-1.1-2.el7.centos.noarch
Sep 19 23:29:27 Updated: libnl3-3.2.28-4.el7.x86_64
Sep 19 23:29:27 Installed: userspace-rcu-0.10.0-3.el7.x86_64
Sep 19 23:29:27 Installed: lttng-ust-2.10.0-1.el7.x86_64
Sep 19 23:29:28 Installed: rdma-core-15-6.el7.x86_64
Sep 19 23:29:28 Installed: libibverbs-15-6.el7.x86_64
Sep 19 23:29:29 Installed: 2:librados2-12.2.5-0.el7.x86_64
Sep 19 23:29:30 Installed: 2:librbd1-12.2.5-0.el7.x86_64
Sep 19 23:29:30 Installed: 2:python-rados-12.2.5-0.el7.x86_64
Sep 19 23:29:30 Installed: 2:python-rbd-12.2.5-0.el7.x86_64
Sep 19 23:29:30 Installed: 2:rbd-nbd-12.2.5-0.el7.x86_64
Sep 19 23:52:16 Updated: vhd-tool-0.20.0-4.3.xcp.el7.centos.x86_64
Sep 19 23:52:17 Updated: xen-hypervisor-4.7.5-5.5.1.xcp.x86_64
Sep 19 23:52:17 Updated: xen-libs-4.7.5-5.5.1.xcp.x86_64
Sep 19 23:52:17 Updated: xen-dom0-libs-4.7.5-5.5.1.xcp.x86_64
Sep 19 23:52:17 Updated: xen-tools-4.7.5-5.5.1.xcp.x86_64
Sep 19 23:52:18 Updated: xen-dom0-tools-4.7.5-5.5.1.xcp.x86_64
Sep 19 23:52:18 Updated: 2:qemu-dp-2.10.2-1.2.0.extras.x86_64
Sep 19 23:52:21 Updated: xapi-tests-1.90.6-1.x86_64
Sep 19 23:52:31 Updated: xapi-core-1.90.6-1.x86_64
Sep 19 23:52:39 Updated: QConvergeConsoleCLI-Citrix-2.0.00-24.3.xcp.x86_64
Sep 19 23:52:47 Updated: linux-firmware-20170622-3.2.noarch
Sep 19 23:52:48 Updated: xcp-ng-center-7.5.0.8-3.noarch
Sep 19 23:52:49 Updated: xapi-xe-1.90.6-1.x86_64
Sep 19 23:52:49 Updated: 2:microcode_ctl-2.1-26.xs1.x86_64
Sep 19 23:52:57 Updated: kernel-4.4.52-4.0.7.1.x86_64

Kind regards,
André

from rbdsr.

Hi!

Thank you very much about the glibc hint. I just installed the newer version from centos. It is the same version, so API should also be the same.

from rbdsr.

Hi,
Did you update glibc to 2.17-222?
Does qemu-dp work now?

from rbdsr.

Hi Roman,

yes I upgraded it to 222.el7:
Name : glibc
Arch : x86_64
Version : 2.17
Release : 222.el7
Size : 14 M
Repo : installed
From repo : base

And it work perfect. Still testing performance and availability.
BTW:
Perhaps you could update your readme.md so that it is clear howto setup a shared repo. If I understand it correctly, the storage repo is currently bound to the single host, not to a potential the cluster.

from rbdsr.

To setup shared repo you can use shared=true in xe sr-create or xe sr-introduce command
There is a typo in current readme.md. I will fix it.

from rbdsr.