Future sprint: test bz: [feat] P9 guest - pci-passthr about tests HOT 10 CLOSED

Below are the list of PCI passthrough testcases which are planned to be run as part of this feature validation:

1. Do pci passthrough of supported PCI card to guest and verify configuration for PCI device inside the guest
2. Reboot the guest and check the passthrough
3. Suspend resume guest with FC/NIC passthrough
4. reboot/restart the host check the passthrough
5. Verify network performance for NIC passthrough ( netperf/netpipe)
6. Hot unplug/plug the pci device on guest while it is passthroughed
7. pci passthough multiple pci device in the guest
8. Test Dumping - virsh dump
9. After passthrough, verify taking internal snapshot via command \u201cvirsh save\u201d or \u201cvirsh snapshot command\u201d
10. NIC bonding after NIC passthrough
11. Do the passthrough in conjunction with memory hotplug
12. Do the passthrough in conjunction with different numa configuration
13. Do the passthrough in conjunction with Memory Hugepages
14. Do the passthrough in conjunction with vCPU pinning
15. EEH testing on PCI Passthroughed adapter. Make sure the adapter is recovered successfully after EEH injection.
16. After detaching FC/NIC adapter from guest, make sure adapter attaches back to host successfully. Run sanity tests link configuring NIC cards, discovering FC disks and mounting them on host etc

Review comments are welcome.

from tests.

We have not got any review comments.. will EEH, hotplug be supported ? i am interested in knowing which all testcases are supported for this sprint 1

from tests.

1, 2 - yes
3 - not sure. If it is simple suspeding/resuming a guest - then yes, if it involves savevm/restorevm to/from disk - then it uses migration mechanism and that just won't work. Easy way to check that is to look to "ps" or "top" - if qemu-system-ppc64 is still there after suspending, then test it.
4 - please explain, it is not clear what a host reboot has to do with passthrough
5, 6, 7, 8 - yes
9 - same concern as for 3
10 - please explain. Pass through 2 NICs and make a bond? If so, you can do that, it is just not clear how it is different from 5, really
11, 12, 13, 14 - yes
15 - yes, just remember it only works if a single PE (a.k.a. IOMMU group) sits on a virtual PHB, 2 PEs will disable EEH; EEH also won't work if a passthru device was hotplugged (I suggest trying but I am pretty sure it won't work).
16 - yes

Sorry for a delay with response.

from tests.

3, 9 - @aik concern is valid, and I have been thinking whether we don't want to document this somehow? I mean, I don't know which exact commands in libvirt/virsh will map to suspend/resume or savevm/restorevm. I suppose we have both options, as I think they would map to suspend and hibernate in a real machine respectively. Could you help here, @ShivaprasadGBhat? And, from a user perspective, I don't think it is clear that savevm/restorevm involves QEMU's migration mechanism, which doesn't support PCI pass through.
15 - Although @aik concern is valid and correct, we should have multiple PHB support now, which makes it possible to use multiple PCI passed through devices with EEH enabled in a VM (probably worth documenting this as well). Now, @aik, is there any way to enable EEH in a hot plugged device? Should we create a feature for this and work on it in the long term?

@jesltc, do we have any place where we can document things for HostOS and KVM on Power in general? If not, I'll add this in the Linux on Power developer portal.

from tests.

EEH testing on PCI Passthroughed adapter

It may actually work when we get PHB hotplug. At the moment when we hotplug devices, they are just plugged as a single device while the proper way is to have a PCI slot (which is more or less a P2P PCI bridge) and plug a device to it. The difference is how the guest handles it and it only enables EEH when a PCI slot is involved and it does not with the existing kick to /sys/bus/pci/rescan from rtasd.

PHB hotplug will hopefully provide necessary infrastructure for proper EEH enablement. @mdroth or @gkurz should know better.

ps: to clarify - PHB kinda acts like a single PCI slot, this is why it should trigger proper eeh_dev structure allocation in the guest. Simple PCI rescan does not create eeh_dev.

from tests.

I may actually work when we get PHB hotplug. At the moment when we hotplug devices, they are just plugged as a single device while the proper way is to have a PCI slot (which is more or less a P2P PCI bridge) and plug a device to it. The difference is how the guest handles it and it only enables EEH when a PCI slot is involved and it does not with the existing kick to /sys/bus/pci/rescan from rtasd.

from tests.

what is that last wrong message?

from tests.

------- Comment From [email protected] 2017-09-22 00:57:51 EDT-------
(In reply to comment #5)
>
> 1, 2 - yes
> 3 - not sure. If it is simple suspeding/resuming a guest - then yes, if it
> involves savevm/restorevm to/from disk - then it uses migration mechanism
> and that just won't work. Easy way to check that is to look to "ps" or "top"
> - if qemu-system-ppc64 is still there after suspending, then test it.
> 4 - please explain, it is not clear what a host reboot has to do with
> passthrough
> 5, 6, 7, 8 - yes
> 9 - same concern as for 3
> 10 - please explain. Pass through 2 NICs and make a bond? If so, you can do
> that, it is just not clear how it is different from 5, really
> 11, 12, 13, 14 - yes
> 15 - yes, just remember it only works if a single PE (a.k.a. IOMMU group)
> sits on a virtual PHB, 2 PEs will disable EEH; EEH also won't work if a
> passthru device was hotplugged (I suggest trying but I am pretty sure it
> won't work).
> 16 - yes
>
> Sorry for a delay with response.

Thanks Alexey for the review comments.
4 --> we opt to test auto boot for guest in this case with pci passthrough. So this tests that guest can auto boot with pcipassthrough on host reboot..

3 & 9 --> ok point noted.

10 --> it's more of sanity for bond test case for pci devices in passthrough mode.. not much different from 5 apart from bonding..
15 --> ok.. point noted.

from tests.

Now, @aik, is there any way to enable EEH in a hot plugged device? Should we create a feature for this and work on it in the long term?

There is no plan to allow EEH for multiple devices on the same PHB but when we get PHB hotplug working - then a VFIO device hotplugged into this new PHB should have working EEH.

from tests.

------- Comment From [email protected] 2018-06-15 18:00:42 EDT-------
When PHB hotplug is implemented, please create a new bug.

from tests.