anyvisionltd / automation-infra Goto Github PK

I have to manually modify .local/hardware.yaml to set the username to root, as it just uses the current username.

This happened to golan .. and i suspect that it can happen when we do poweron/poweroff concurrently ..

Jan 28 18:13:59 server5 HYPERVISOR[7111]: 2020-01-28 18:13:59,535 7 139640853866304 DEBUG root Requested vm info for vm server5-vm-1 handle_vm_status /root/lab/vms/rest.py:76
Jan 28 18:13:59 server5 HYPERVISOR[7111]: libvirt: QEMU Driver error : Requested operation is not valid: domain is not running
Jan 28 18:13:59 server5 HYPERVISOR[7111]: 2020-01-28 18:13:59,536 7 139640696047360 ERROR root Libvirt operation failed _libvirt_connection /root/lab/vms/libvirt_wrapper.py:19
Jan 28 18:13:59 server5 HYPERVISOR[7111]: Traceback (most recent call last):
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/root/lab/vms/libvirt_wrapper.py", line 17, in _libvirt_connection
Jan 28 18:13:59 server5 HYPERVISOR[7111]: yield connection
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/root/lab/vms/libvirt_wrapper.py", line 67, in dhcp_lease_info
Jan 28 18:13:59 server5 HYPERVISOR[7111]: nets = vm.interfaceAddresses(libvirt.VIR_DOMAIN_INTERFACE_ADDRESSES_SRC_LEASE, 0)
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/usr/lib/python3/dist-packages/libvirt.py", line 1376, in interfaceAddresses
Jan 28 18:13:59 server5 HYPERVISOR[7111]: if ret is None: raise libvirtError ('virDomainInterfaceAddresses() failed', dom=self)
Jan 28 18:13:59 server5 HYPERVISOR[7111]: libvirt.libvirtError: Requested operation is not valid: domain is not running
Jan 28 18:13:59 server5 HYPERVISOR[7111]: 2020-01-28 18:13:59,536 7 139640853866304 ERROR aiohttp.server Error handling request log_exception /usr/local/lib/python3.6/dist-packages/aiohttp/web_protocol.py:355
Jan 28 18:13:59 server5 HYPERVISOR[7111]: Traceback (most recent call last):
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/usr/local/lib/python3.6/dist-packages/aiohttp/web_protocol.py", line 418, in start
Jan 28 18:13:59 server5 HYPERVISOR[7111]: resp = await task
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/usr/local/lib/python3.6/dist-packages/aiohttp/web_app.py", line 458, in _handle
Jan 28 18:13:59 server5 HYPERVISOR[7111]: resp = await handler(request)
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/root/lab/vms/rest.py", line 80, in handle_vm_status
Jan 28 18:13:59 server5 HYPERVISOR[7111]: info = await self.allocator.vm_manager.network_info(vm)
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/root/lab/vms/vm_manager.py", line 82, in network_info
Jan 28 18:13:59 server5 HYPERVISOR[7111]: lambda: self.libvirt_api.dhcp_lease_info(vm["name"]))
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Jan 28 18:13:59 server5 HYPERVISOR[7111]: result = self.fn(*self.args, **self.kwargs)
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/root/lab/vms/vm_manager.py", line 82, in
Jan 28 18:13:59 server5 HYPERVISOR[7111]: lambda: self.libvirt_api.dhcp_lease_info(vm["name"]))
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/root/lab/vms/libvirt_wrapper.py", line 67, in dhcp_lease_info
Jan 28 18:13:59 server5 HYPERVISOR[7111]: nets = vm.interfaceAddresses(libvirt.VIR_DOMAIN_INTERFACE_ADDRESSES_SRC_LEASE, 0)
Jan 28 18:13:59 server5 HYPERVISOR[7111]: File "/usr/lib/python3/dist-packages/libvirt.py", line 1376, in interfaceAddresses
Jan 28 18:13:59 server5 HYPERVISOR[7111]: if ret is None: raise libvirtError ('virDomainInterfaceAddresses() failed', dom=self)
Jan 28 18:13:59 server5 HYPERVISOR[7111]: libvirt.libvirtError: Requested operation is not valid: domain is not running

https://anyvision.atlassian.net/browse/TMTNNFR-75

since currently VM is a pet either than cattle we need to provide a way to restore state of provisioned VM when server reboots .. or vm hypervistor crashes