Trying IOMMU with KVM causes hard freeze

Hardware
1

Hardware: Librem 15v3
Coreboot: 4.7-18-g7ac4919b8a-4.7-Purism-4
OS: openSUSE Tumbleweed; current kernel: 4.17.3-1

I want to run a virtual machine with OpenWrt under libvirt+qemu+kvm, with wireless card attached to it using IOMMU.

This was working under QubesOS, which is using XEN (ofc not with OpenWrt, but with their net-vm, but it doesn’t matter here). Now I wanted to recreate something similar under openSUSE with KVM.

The problem is that as soon as I turn on the VM the host system goes into hard freeze. Even SysRq keys are not working. In dmesg --follow there are no significant messages before freeze.

What I’ve done so far:

  • Enabled iommu with kernel command-line arguments: iommu=on intel_iommu=on,igfx_off (igfx_off is required, because without it after booting you can see some random colors pattern on screen and the system is frozen). Using iommu=pt makes no difference.
  • Added vfio-pci module in /etc/modules-load.d/ and options vfio-pci ids=168c:0034 in /etc/modprobe.d/. This causes the wireless card to be detached from host ath9k driver at boot. It’s not required, as libvirt can do this at runtime. Even I can do it manually as described here (there is attaching to pci-stub, but it works the same for vfio-pci). This seems not to be the cause of freeze.
2

I’ve just tested this with fresh PureOS 8 and it freezes exactly the same way. Therefore I removed “Other OSes” category.

3

Hello? Could someone at least try to reproduce and confirm the problem?

4

Did you check out Is anyone else experiencing freezing issues with Librem 15 v3??

5

Thanks for the hint.

Unfortunately adding intel_pstate=disable didn’t change anything.

By the way: Now I’m running kernel 4.17.13 and with iommu=soft vfio-pci couldn’t grab the device. In dmesg there was vfio-pci: probe of 01:00.0 failed with "error -22". With iommu=on vfio-pci was grabbing the device properly. (to make things clear: with iommu=soft I couldn’t even attach the device to VM. with iommu=on I could attach it and then it caused hard freeze, as usual)

6

So I tried this again and I have good news!
Currently I have Coreboot 4.11-Purism-1 and kernel 5.5.7. Everything works smoothly. Kernel command-line argument required is intel_iommu=on (gfx issue is resolved). No fiddling with vfio-pci is required (it’s optional if you wish). Just attach the device to VM and it Just Works.
(I don’t know why OpenWrt is not picking up this card so easily, but that’s not relevant here)
(Edit: and here’s why OpenWrt doesn’t support my wifi card: https://forum.openwrt.org/t/solved-atheros-ar9462-on-x86-as-an-ap-can-not-get-it-to-work-at-all/32122/6)

1 Like