When running a Linux desktop it sometimes becomes necessary to run a Windows system. When GPU acceleration is necessary, dual booting often becomes the solution. But dual booting is kind of cumbersome so it would be ideal if we could get GPU acceleration in a Windows 10 VM.
Intel has a solution for this – GVT-g. From the GVT-g project website:
Intel GVT-g is a full GPU virtualization solution with mediated pass-through (VFIO mediated device framework based), starting from 5th generation Intel Core(TM) processors with Intel Graphics processors. GVT-g supports both Xen and KVM (a.k.a XenGT & a.k.a KVMGT). A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability.
In this post I’ll be focusing on getting GVT-g working on KVM. My main system runs Fedora 32 but this guide should also work on recent versions of Ubuntu or other distributions with recent kernel and Qemu versions.
If you follow this tutorial to the end you should have a Windows 10 VM with GPU acceleration in a window on your Linux desktop, all without requiring a second GPU or disabling the GPU output of your Linux system. However, keep in mind that running a 3D load on Windows and Linux at the same time will still cause slowdowns though.
Steps to setup GVT-g
- Setup the host machine to support GVT-g
- Create the virtual GPU
- Install a Windows 10 VM with Virtio hardware
- Re-configure the Windows 10 VM to use the virtualized GPU
First we will need to download several things to make this process go smoothly.
We will also need to make sure that we have virt-manager and virsh installed on your host Linux system:
dnf install virt-manager libvirt-client
apt-get install libvirt-bin virt-manager libvirt-clients gir1.2-spiceclientgtk-3.0
After the installation we need to ensure that libvirt and qemu are new enough on your system.
$ libvirtd -V libvirtd (libvirt) 6.1.0
Libvirt should output at a minimum version 4.6.0. If the version reported here is older than that this tutorial will not work.
$ qemu-system-x86_64 --version QEMU emulator version 4.2.0 (qemu-4.2.0-7.fc32) Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
Qemu should output at minimum version 4.0.0. If the version reported here is older than that this tutorial will not work.
It may also be a good idea to add your own user to the
sudo usermod $(id -un) -a -G kvm,libvirt
This isn’t strictly necessary but it makes using virt-manager easier. Without doing this you will have to type your password every time you want to start a VM.
Configuring the host to support GVT-g
In order to support GVT-g we need to add a parameter to the boot loader, add several modules to be automatically loaded at boot, and create a simple systemd-unit to automatically create the virtualized GPU at boot. We will also need to reboot the system; while you could reboot the system after each step, when following this guide only one reboot should be necessary.
Configuring the bootloader
/etc/default/grub using your favorite text editor and find the line that starts with
GRUB_CMDLINE_LINUX and add
i915.enable_gvt=1 i915.enable_fbc=0 at the end of the line. On my system the file looks like this after the edit:
GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="gfxterm" GRUB_CMDLINE_LINUX="resume=/dev/mapper/vg_isla-swap rd.luks.uuid=luks-123568b4-82fb-4cb0-aeed-7cfd3c2603f2 rd.md.uuid=8d843e1d:c4eda4f4:3622e122:9ef0c927 rd.lvm.lv=vg_isla/root rd.md.uuid=f978b92e:db7efe36:2c7037ed:2a6a6dcf rd.lvm.lv=vg_isla/swap rhgb quiet i915.enable_gvt=1 i915.enable_fbc=0 amdgpu.dc=0" GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=true GRUB_FONT="/boot/grub2/fonts/DejaVuSansMono.pf2"
After making the change we need to regenerate the
On Fedora with UEFI boot:
sudo grub2-mkconfig -o /etc/grub2-efi.cfg
On Fedora with BIOS boot:
sudo grub2-mkconfig -o /etc/grub2.cfg
If you don’t know whether you are BIOS booted or UEFI booted see if the directory
/sys/firmware/efi/efivars/ exists. If it does not, you are BIOS booted.
If your system does not use Grub please refer to your distribution’s documentation on how to change kernel parameters.
Loading the GVT-g modules at boot
Use your favorite text editor to create a file called
/etc/modules-load.d/kvm-gvt-g.conf and place the following content in it:
kvmgt vfio-iommu-type1 vfio-mdev
At this point you have to reboot your host machine.
Checking whether GVT-G is working
We first need to find out what the PCI address of our Intel GPU is. The easiest way to do this is to run the
lspci tool and note the PCI address. On my system the output looks like this:
lspci | grep VGA 00:02.0 VGA compatible controller: Intel Corporation UHD Graphics 630 (Mobile) (rev 02) 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon Pro WX 7100 Mobile]
In this case the Intel GPU’s PCI address is
00:02.0. Keep a note of it as we will need this address several times in the future. We now need to go to the device directory of the GPU in
/sys and check whether GVT-G is enabled.
There should be a directory called
mdev_supported_types in this device entry. If there is not, check
/proc/cmdline and see if the bootloader changes were applied.
Creating the virtual GPU
Creating a virtual GPU works by putting a UUID value into the
create virtual file of a virtual GPU type. We first need to check what GPU sizes your GPU supports (assuming you are still in the GPU’s device directory in
$ ls -l mdev_supported_types/ total 0 drwxr-xr-x. 3 root root 0 May 15 03:32 i915-GVTg_V5_4 drwxr-xr-x. 3 root root 0 May 15 03:32 i915-GVTg_V5_8
This means that my system supports two sizes of virtual GPU
V5_8. The size determines how much video RAM the virtual GPU will have. I have tested the two models in my system and I couldn’t find any performance difference so I’m going to continue assuming it doesn’t matter for this post and create the smallest vGPU. Note the larger the numbers the smaller the vGPUs. In this case
V5_8 being the smallest.
Each vGPU needs a UUID. The easiest way to generate a UUID is by running the
uuidgen tool. For instance:
$ uuidgen cb33ec6d-ad44-4702-b80f-c176f56afea1
Keep a note of your UUID or just use the one here. It doesn’t matter that much if you only need one vGPU.
echo cb33ec6d-ad44-4702-b80f-c176f56afea1 | sudo tee mdev_supported_types/i915-GVTg_V5_8/create
If you get an error like
No space left on device you may not have enough vRAM assigned to your Intel GPU in your firmware. You will need to go into your system’s configuration utility and raise it. You will need to refer to your system’s documentation on how to do this. Firmware configuration utilities vary wildly per vendor but in general you will be able to enter it by pressing
DEL during early boot.
If the command succeeded you should now have a directory named after the UUID in the GPU’s
$ ls ari_enabled class current_link_width driver firmware_node i2c-3 iommu link max_link_width msi_irqs rescan resource2 rom uevent boot_vga config d3cold_allowed driver_override graphics i2c-4 iommu_group local_cpulist mdev_supported_types numa_node reset resource2_wc subsystem vendor broken_parity_status consistent_dma_mask_bits device drm gvt_firmware i2c-5 irq local_cpus modalias power resource resource4 subsystem_device cb33ec6d-ad44-4702-b80f-c176f56afea1 current_link_speed dma_mask_bits enable i2c-2 index label max_link_speed msi_bus remove resource0 revision subsystem_vendor
Persist the vGPU creation at boot
Creating the GVT-G device is not permanent, at the next reboot the vGPU just created will need to be recreated. We can create a systemd unit file to automate this process during boot.
Create a new file called
/etc/systemd/system/setup-gvt.service and place the following content in it, replacing the UUID, PCI address, and vGPU size with the values we found for your system:
[Unit] Description=Setup GVT [Service] Type=oneshot ExecStart=/usr/bin/bash -c 'echo cb33ec6d-ad44-4702-b80f-c176f56afea1 > /sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types/i915-GVTg_V5_8/create' [Install] WantedBy=multi-user.target
Make sure it starts automatically at boot with
sudo systemctl enable setup-gvt
Installing the Windows 10 VM with Virtio hardware
Next I will provide a step-by-step guide with screenshots below. If you’re already familiar with virt-manager and Windows 10 setup these are the highlights:
- Create a Qemu/KVM instance (Not a user session)
- Select the Windows 10 .iso as installation media
- Make sure to select ‘Customize configuration before install’
- Set the CPU type to
- Set the hard drive type to
- Set the network controller type to
- Get to the drive selection screen
- Swap the Windows 10 iso with the virtio windows driver iso.
- Select the NETKVM/w10/amd64 directory and install the network controller driver.
- Select the viostor/w10/amd64 directory and install the virtio SCSI driver.
- Swap the Windows 10 iso back and continue the installation like normal
- Install the Windows guest addons from
Press the 'create a new virtual machine' button in virt-manager
Ensure that QEMU/KVM is selected and not a user session
Select the Windows 10 iso downloaded earlier
Create a harddrive for the VM
Select a name and be sure to select 'Customize configuration'
Ensure the firmware is set to 'BIOS'
Set the cpu type to 'host-passthrough' and set a sane cpu topology.
Set the hard drive type to VirtIO
Set the network hardware type to VirtIO
And click 'begin installation'
Windows setup will start, press next
and press install now
Type your Windows product code
Accept the license
At this point there won't be a hard drive to install to. We will install the drivers from the VirtIO iso we downloaded earlier. Click 'load driver'
This screen should pop up. Go to the virt-manager interface and 'view' select 'details'
Now select the VirtIO driver iso, hit apply, and select 'view' 'console'
Select the cdrom drive
Select the NETKVM/w10/amd64 directory and click OK
Install the Network card driver
At this point we still won't have a hard drive. Load another driver
This time select the viostor/w10/amd64 directory and click OK
Load the driver
Go back to details, and select the windows 10 iso agian, click apply and go back to 'console'
Now we will have a drive, click 'New'
Create the partitions
Select the new partition and click next
Windows install will commence
Follow the whole setup process
There we go
Go to 'spice-space.org'
Get to the download page and download the windows drivers
Run the setup, acknowledge the prompt
Make sure to click 'install'
And we're done
Shut down the VM
At this point we have a relatively standard Windows 10 installation with VirtIO drivers. We should make a backup of the libvirt configuration now as we will be making changes to it.
sudo virsh dumpxml win10 > win10.xml
Adding the vGPU to the Windows 10 VM
For this step we will have to edit the libvirt XML file directly as virt-manager does not support the device types we will need to add. We will be making the following changes:
- Setting the ‘domain’ type to QEMU
- Configuring the Spice server to use GL and disable networking
- Removing the virtualized VGA adapter
- Add the vGPU mediated PCI device
- Enable RAMFB for early boot messages
- Enable DMABUF so we can see accelerated GPU output in our virt-manager window
- Enable a workaround for Mesa bug #2678
First, we will make a backup of the
win10.xml file we created earlier.
cp win10.xml win10.xml.bak in case we make an error editing the instance. For the rest of this section we’re assuming you have the
win10.xml file open in a text editor.
Setting the ‘domain’ type to QEMU
To set the domain type to qemu we need to edit the first line of the file. Change the line:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
Configuring the Spice server to use GL and disable networking
Now to change the Spice client settings. Find the
graphics block, it should look something like this:
<graphics type='spice' autoport='yes'> <listen type='address'/> <image compression='off'/> </graphics>
Change it to look like this:
<graphics type='spice'> <listen type='none'/> <gl enable='yes' rendernode='/dev/dri/by-path/pci-0000:00:02.0-render'/> </graphics>
Note that we need to put the PCI address of our real Intel GPU in the
rendernode of this block.
Removing the virtualized VGA adapter
We now need to disable the
QXL virtual VGA adapter but we can’t just remove it as libvirt seems to require at least one
video block. Find the
video block, it should look something like this:
<video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video>
Change it to look like this:
<video> <model type='none'/> </video>
Add the vGPU mediated PCI device
Now to add the vGPU. We need to add a whole new block. You can place this block directly below the
video block we just edited.
<hostdev mode='subsystem' type='mdev' managed='no' model='vfio-pci' display='on'> <source> <address uuid='cb33ec6d-ad44-4702-b80f-c176f56afea1'/> </source> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </hostdev>
Enable RAMFB for early boot messages
Finally, we need to enable DMABUF and RAMFB. For this we need to add another entirely new section, but this time it needs to go at the very end of the file right before the closing
</domain> line. The following block needs to be put directly above the last line of the file:
<qemu:commandline> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.x-igd-opregion=on'/> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.ramfb=on'/> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.driver=vfio-pci-nohotplug'/> <qemu:env name='INTEL_DEBUG' value='norbc'/> </qemu:commandline>
At this point our XML file is ready and we can load it back into libvirt by running
sudo virsh define win10.xml. At this point it should be possible to boot the VM again.
After making the above edits your
win10.xml file should look something like this:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>win10</name> <uuid>1b7fa3cc-b3ba-464b-8585-1423514c1067</uuid> <metadata> <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0"> <libosinfo:os id="http://microsoft.com/win/10"/> </libosinfo:libosinfo> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <vcpu placement='static'>4</vcpu> <os> <type arch='x86_64' machine='pc-q35-4.2'>hvm</type> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> </hyperv> <vmport state='off'/> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' dies='1' cores='2' threads='2'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> <timer name='hypervclock' present='yes'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <pm> <suspend-to-mem enabled='no'/> <suspend-to-disk enabled='no'/> </pm> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/win10.qcow2'/> <target dev='vda' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/home/hp/Downloads/Win10_1909_English_x64.iso'/> <target dev='sdb' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0x15'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </controller> <interface type='network'> <mac address='52:54:00:d4:70:45'/> <source network='default'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='spice'> <listen type='none'/> <gl enable='yes' rendernode='/dev/dri/by-path/pci-0000:00:02.0-render'/> </graphics> <sound model='ich9'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/> </sound> <video> <model type='none'/> </video> <hostdev mode='subsystem' type='mdev' managed='no' model='vfio-pci' display='on'> <source> <address uuid='cb33ec6d-ad44-4702-b80f-c176f56afea1'/> </source> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </hostdev> <redirdev bus='usb' type='spicevmc'> <address type='usb' bus='0' port='2'/> </redirdev> <redirdev bus='usb' type='spicevmc'> <address type='usb' bus='0' port='3'/> </redirdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.x-igd-opregion=on'/> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.ramfb=on'/> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.driver=vfio-pci-nohotplug'/> <qemu:env name='INTEL_DEBUG' value='norbc'/> </qemu:commandline> </domain>
Configuring the VM
After booting the VM you will notice that the VM is very slow and running in a small window. You can still login to it by hitting enter and typing your credentials. At this point this just let it sit for several minutes. Windows will download the appropriate Intel GPU drivers and install them. The screen will go black for a little while and then the VM should return with a normal speed, but still running in a small window. The results will be something like this:
We will fix the resolution issue in a minute, but first we should install the full-fat Intel GPU drivers. To make the VM a bit easier to use you can go to ‘view’/’scale display’ and select ‘always’ in the virt-manager ui. This should result in something like this:
To install the Intel display drivers we need to do the following steps:
- Open a browser and go to https://downloadcenter.intel.com
- Scroll down and select ‘graphics’
- Download the DCH drivers .exe
- Run the installer and reboot
Go to downloadcenter.intel.com
Scroll down to 'graphics'
Select the DCM drivers
Scroll down and select the .exe installer
Allow the installation
Next, next, next. This is where the installer will pause for a bit as it is installing the actual driver
Select reboot and hit finish
Right now the resolutions that your VM supports is pretty limited. Right-click on the desktop and select ‘display settings’.
We can fix this with a tool called ‘custom resolution utility‘:
- Download ‘custom resolution utility’ from monitortests.com
- Unzip to a directory, for instance C:\CRU
- Run CRU and add the simple resolutions you want
- Run restart64.exe from CRU
- Set the new resolution in ‘display settings’
Search for 'custom resolution utility'
Download the zip file and open it
copy the contents
create a new directory like C:CRU
Paste, and run CRU
CRU needs admin permissions
Press 'Add' in the 'Standard resolutions' box and add the resolutions you want
Press 'OK' once you're done
Restart64 also needs admin permissions
Exit the tool
Going back to 'display settings' should now let you select different resolutions.
Validating the installation
At this point everything should work. To validate this we will run
dixdagin the bottom left of the task bar
- Run dxdiag
- Check to see that DirectX 12 is supported
Validate that you have DirectX12
Congratulations! You’re done! You should now have a Windows 10 VM with GPU acceleration. It should be possible to run games that work on the iGPU, run Godot engine, or other 3D workloads.
If you have any comments or issues please feel free to leave a comment below. You can sign in with your existing Google, Twitter, or Github accounts!