QEMU-KVM: VM Freezing with Nvidia GPU PCI Passthrough
In this article, we will discuss how to set up a virtual machine (VM) using QEMU-KVM with Nvidia GPU PCI passthrough, which allows you to use your Nvidia GPU directly in a VM. This setup is particularly useful for running GPU-accelerated workloads, such as deep learning or gaming, in a VM.
Prerequisites
Before we begin, make sure you have the following:
- A host machine with an Intel integrated GPU and an Nvidia GPU (e.g., Intel integrated graphics and Nvidia 1650 mobile GPU).
- The Xorg server running on the integrated GPU.
- The Nvidia driver loaded on the host machine.
- Libvirt and QEMU-KVM installed on the host machine.
Preparing the Host Machine
First, we need to ensure that the host machine can use the Nvidia GPU. To do this, we will use the prime-select command to switch to the Nvidia GPU:
sudo prime-select nvidia
Next, we need to install the vfio-pci module, which will allow us to passthrough the Nvidia GPU to the VM:
sudo modprobe vfio-pci
We also need to blacklist the Nvidia driver for the VM to use the vfio-pci module:
sudo echo "blacklist nouveau" > /etc/modprobe.d/blacklist-nvidia.conf
sudo echo "options nouveau modeset=0" > /etc/modprobe.d/blacklist-nvidia.conf
sudo update-initramfs -u
Creating the VM
Now that we have prepared the host machine, we can create the VM. We will use the virsh command to create a new VM:
sudo virsh define <path-to-xml-file>
Where path-to-xml-file is the path to the XML file that describes the VM. Here is an example XML file:
<domain type='kvm' id='0'>
<name>vm</name>
<memory unit='KiB'>4194304</memory>
<currentMemory unit='KiB'>4194304</currentMemory>
<vcpu placement='static'>2</vcpu>
<os>
<type arch='x86_64' machine='pc-q35-5.2'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic>
<pae/>
</features>
<cpu mode='host-model' check='partial'>
<model>host</model>
<topology sockets='1' cores='2' threads='1'/>
</cpu>
<clock offset='utc'>
<timer name='hpet' present='yes'/>
<timer name='pit' tickpolicy='delay' present='yes'/>
<timer name='tsc' present='yes'/>
</clock>
<on\_poweroff>destroy</on\_poweroff>
<on\_reboot>restart</on\_reboot>
<on\_crash>destroy</on\_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86\_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/path/to/disk.qcow2'/>
<target dev='vda' bus='virtio'/>
</disk>
<controller type='usb' index='0' model='piix3-uhci' slots='3'/>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1' model='pcie-root-port'/>
<controller type='pci' index='2' model='pcie-root-port'/>
<controller type='pci' index='3' model='pcie-root-port'/>
<controller type='pci' index='4' model='pcie-root-port'/>
<controller type='pci' index='5' model='pcie-root-port'/>
<controller type='pci' index='6' model='pcie-root-port'/>
<controller type='virtio-serial' index='0'/>
<interface type='bridge'>
<mac address='52:54:00:00:00:01'/>
<source bridge='virbr0'/>
</interface>
<serial type='pty'>
<target port='0'>
<model type='vcserial'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
<target type='virtio' name='org.qemu.guest_agent.0'/>
</console>
<input type='tablet' bus='usb'/>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='spice' autoport='yes'/>
<sound model='ich6'/>
<video>
<model type='qxl' vram='16384' heads='1' primary='yes'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<redirdev >
<type='unix' path='/tmp/spice.socket'/>
</redirdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</memballoon>
<rng model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</rng>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</source>
<rom file='/usr/share/qemu/90-nvidia.rom'/>
</hostdev>
</devices>
</domain>
Note the <hostdev> section, which passthrough the Nvidia GPU to the VM.
Starting the VM
Now that we have created the VM, we can start it using the virsh command:
sudo virsh start vm
In this article, we have discussed how to set up a virtual machine (VM) using QEMU-KVM with Nvidia GPU PCI passthrough. We have covered the following key concepts:
- Preparing the host machine by switching to the Nvidia GPU, installing the
vfio-pcimodule, and blacklisting the Nvidia driver. - Creating the VM using the
virshcommand and an XML file that describes the VM. - Starting the VM using the
virshcommand.