Libvirt and vfio

[Steve Phillips]

Hi there,

I'm trying to run a VM and pass through an nvidia video card to a virtual
machine using the vfio device driver in order to get hardware acceleration
working within my VM (a shitty java application I'm using within the VM
requires this in order to work).

I've gotten to the point where I've associated the PCI device with the
kernel vfio driver but it seems that the version of qemu/libvirt that comes
with RHEL 6 doesn't support this driver type.

Has anyone gotten video passthrough under RHEL 6 working under libvirt/qemu
?

My system I'm currently running

Linux avernus 3.14.1-1.el6.elrepo.x86_64 #1 SMP Mon Apr 14 19:29:19 EDT
2014 x86_64 x86_64 x86_64 GNU/Linux

(I needed to update the kernel to gain vfio support)

# lsmod | grep vfio
vfio_iommu_type1 17867 0
vfio_pci 36964 0
vfio 21010 2 vfio_iommu_type1,vfio_pci

The linux boot line to enable iommu support and blacklist the pci-ids

intel_iommu=on pci_acs_override=downstream pci-stub.ids=10de:0a65,10de:0be3

The PCI device

# ls /sys/bus/pci/devices/0000\:06\:00.0/iommu_group/devices/
0000:06:00.0 0000:06:00.1

and output from an lspci -k showing the modules attached to each pci device.

06:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210]
(rev a2)
Subsystem: eVga.com. Corp. Device 1311
Kernel driver in use: vfio-pci
Kernel modules: nouveau, nvidiafb
06:00.1 Audio device: NVIDIA Corporation High Definition Audio Controller
(rev a1)
Subsystem: eVga.com. Corp. Device 1311
Kernel driver in use: vfio-pci
Kernel modules: snd-hda-intel

However, the qemu in use doesn't seem to have vfio driver support?

name "pci-bridge", bus PCI
name "virtio-scsi-pci", bus PCI, alias "virtio-scsi"
name "virtio-balloon-pci", bus PCI
name "virtio-serial-pci", bus PCI, alias "virtio-serial"
name "virtio-net-pci", bus PCI
name "virtio-blk-pci", bus PCI, alias "virtio-blk"
name "i82562", bus PCI
name "i82559er", bus PCI
name "i82559c", bus PCI
name "i82559b", bus PCI
name "i82559a", bus PCI
name "i82558b", bus PCI
name "i82558a", bus PCI
name "i82557c", bus PCI
name "i82557b", bus PCI
name "i82557a", bus PCI
name "i82551", bus PCI
name "i82550", bus PCI
name "pcnet", bus PCI
name "rtl8139", bus PCI
name "e1000", bus PCI, desc "Intel Gigabit Ethernet"
name "smbus-eeprom", bus I2C
name "usb-hub", bus USB
name "usb-host", bus USB
name "usb-kbd", bus USB
name "usb-mouse", bus USB
name "usb-tablet", bus USB
name "usb-storage", bus USB
name "scsi-disk", bus SCSI, desc "virtual SCSI disk or CD-ROM (legacy)"
name "scsi-block", bus SCSI, desc "SCSI block device passthrough"
name "scsi-cd", bus SCSI, desc "virtual SCSI CD-ROM"
name "scsi-hd", bus SCSI, desc "virtual SCSI disk"
name "scsi-generic", bus SCSI, desc "pass through generic scsi device
(/dev/sg*)"
name "usb-ccid", bus USB, desc "CCID Rev 1.1 smartcard reader"
name "usb-ccid", bus USB, desc "CCID Rev 1.1 smartcard reader"
name "ccid-card-passthru", bus ccid-bus, desc "passthrough smartcard"
name "usb-redir", bus USB
name "virtserialport", bus virtio-serial-bus
name "virtconsole", bus virtio-serial-bus
name "ich9-usb-ehci1", bus PCI
name "usb-ehci", bus PCI
name "i6300esb", bus PCI
name "ne2k_pci", bus PCI
name "ide-drive", bus IDE
name "isa-ide", bus ISA
name "ES1370", bus PCI, desc "ENSONIQ AudioPCI ES1370"
name "AC97", bus PCI, desc "Intel 82801AA AC97 Audio"
name "intel-hda", bus PCI, desc "Intel HD Audio Controller"
name "hda-micro", bus HDA, desc "HDA Audio Codec, duplex (speaker,
microphone)"
name "hda-duplex", bus HDA, desc "HDA Audio Codec, duplex (line-out,
line-in)"
name "hda-output", bus HDA, desc "HDA Audio Codec, output-only (line-out)"
name "VGA", bus PCI
name "SUNW,fdtwo", bus System
name "sysbus-fdc", bus System
name "pci-serial", bus PCI
name "isa-serial", bus ISA
name "cirrus-vga", bus PCI, desc "Cirrus CLGD 54xx VGA"
name "isa-parallel", bus ISA
name "sga", bus ISA, desc "Serial Graphics Adapter"
name "ich9-usb-uhci3", bus PCI
name "ich9-usb-uhci2", bus PCI
name "ich9-usb-uhci1", bus PCI
name "piix4-usb-uhci", bus PCI
name "piix3-usb-uhci", bus PCI
name "ib700", bus ISA
name "ne2k_isa", bus ISA
name "testdev", bus ISA
name "isa-debugcon", bus ISA
name "pci-assign", bus PCI, desc "pass through host pci devices to the
guest"
name "qxl", bus PCI, desc "Spice QXL GPU (secondary)"
name "qxl-vga", bus PCI, desc "Spice QXL GPU (primary, vga compatible)"
name "pvpanic", bus ISA

(This was obtained by passing through a qemu command line tag to libvirts
xml domain file

<qemu:commandline> <!-- instead of hostdev ... -->
<qemu:arg value='-device'/>
<qemu:arg value='?'/>
</qemu:commandline>

)

Does anyone know of a way to get either, hardware acceleration working
under a VM using stock RHEL 6 libvirt/qemu-kvm packages or how to get vfio
support working under RHEL 6 (without installing the dev tools and
recompiling everything, hence, breaking package management)

Cheers,