Running Windows 10/11 Guest with GPU Passthrough using laptop running Fedora

Abstract

This is a full guide for people who wanted to set up Windows 10/11 VM with QEMU/KVM hypervisor enhancements for a laptop that is configured with hybrid graphics card like Intel/AMD + NVIDIA. This process will take about 1 to 2 hours, depending on your system's performance and your patience =)

There is another comprehensive guide you can follow here (shoutout to asus-linux team who made supergfxctl which is a very important tool for this guide). It is more up-to-date than mine. I would probably incorporate those information into my guide, but you are welcome to use this one as a reference!

Before we proceed:

This guide is exclusively for Fedora users because this distro is quite different to set up than other distro such as Arch. I would say Arch is easier to setup than Fedora, but sometimes you like to use Fedora than Arch in terms of its usage & features.
This tutorial will be mostly covered for a laptop with NVIDIA Optimus MUXed configuration, when dGPU (dedicated GPU) can connect directly to HDMI/DP output. If in doubt, search "VR-Ready laptops", as those are definitely MUXed.
For MUXless configuration, in which the dGPU shares display alongside with iGPU (integrated GPU), you may need to extract your GPU's vBIOS file to be used inside your VM to avoid getting no output from your screen. Refer here first before proceeding.
How to check in terminal: lspci -k | grep -A 2 -E "(VGA|3D)"; If the result displays 2 VGA = likely MUXed, or 1 VGA and 1 3D = highly likely MUXless.
This tutorial also doesn't cover AMD iGPU + AMD dGPU configuration (or AMD Advantage laptops) as I personally don't have them. The process might be the same, but some steps needs slight alteration. Do this at your own risk.

Tested with Lenovo Ideapad Gaming 3 15ARH05 (AMD Ryzen 5 4600H + NVIDIA Geforce 1650 Ti) with Fedora 36 (previously on ArchLinux), using NVIDIA driver

Setting your expectation right...

The performance difference between this method and dual boot is not substantial but also not negligible. You might experiencing some hiccups, freezes or loss of performance depending on your laptop. Your CPU speed and core count matter a lot.
This method requires re-login, which means you have to turn off your dGPU properly and then log out and log back in, in order to start your VM. This will turn off some people of course (might just dual boot instead?), but this is how you work on the limitation.
Usually the use case for this method is to circumvent video game's anticheat that does not work, or to run some tools or application software that does not work underneath Linux such as Adobe products, AutoCAD, etc. But again, dual boot is also an option.
However, this might be a good experience to learn how virtual machine works when passthrough a hardware such as your GPU is possible to uplift its performance, unlike other VM software. You might be able to containerized an untrusted software that requires your GPU to run, if you are paranoid with it.
Or for privacy and security reason if you hook your VM with VPN all the time, so the website you visit might not be able to detect your host operating system easily. If you got infected by a virus, you might delete your VM and start over again, unlike running your Windows bare metal.
Regardless of your use case, I hope this method is easier for you to follow =)

Updates

Just an update log to keep track of all of the changes in this gist:

- 9/30/2024: Fixed *a lot* grammatical mistakes (sorry im an terrible ESL guy and my bad grandma), updated some information regarding Looking Glass and supergfxctl. Also, virtio driver and enhancement information are no longer tagged as *(optional)*, as they are pretty much needed nowadays.

- 2/29/2024: Recommended another guide from asus-linux team, and fix more grammars.

- 9/11/2023: Updated supergfxctl guide with its latest version, recommended NVCleanInstall as an alternative way to install NVIDIA driver without their software bloat, and fix some of the grammar issues.

- 3/11/2023: You don't need SCREAM audio driver or IVSHMEM driver starting from B6 update, as they are already provided their own drivers. It is easier now to use Looking Glass as your remote client. Secure boot is still need to be disabled however.

Index

Prerequisites
Pre-setup VM
Setting up Windows 10/11 VM
Using supergfxctl to bind vfio to GPU
Post VM setup with GPU passthrough
- Add dGPU to the hardware list in VM
- Test your VM and install NVIDIA driver
VM Enhancements
Miscellaneous optimizations
FAQs & Troubleshooting
References

Prerequisites

CPU requirement: Intel Core or AMD Ryzen, at least 6-core CPU with hyper-threading/SMT (12 threads in total) and has support for virtualization extension (VT-d/AMD-Vi)
GPU requirement: Intel/AMD iGPU+ NVIDIA GPU w/ MUXed (or MUXless) configuration
At least have 12GB of RAM. Even better if you have more.
Depending on your use case (eg: gaming, creative work, etc.), you might need to reserve your virtual disk around the ballpark of 256GB or 512GB. So your system should have at least 1TB of total space to make some headroom for it. Even better if you are using SSD.
Have installed Fedora Workstation. Should work with any desktop environment, and with display server like Xorg/Wayland. Type echo $XDG_SESSION_TYPE to check your current display server.
(Optional) Have installed NVIDIA driver. If you haven't installed it yet, get it here, unless you don't want your host to use your dGPU at all. In this case, this doesn't matter.
Things to display your Windows 10/11 VM. Either:
- Have a separate monitor to display Windows 10/11 VM (highly recommended)
- Use Looking Glass (a good alternative)
- Use any remote display client (NVIDIA Stream, Steam Remote Play, etc., not recommended)
Basic terminal knowledge. This guide requires you to type certain command lines to achieve something that is not possible with only just GUI. (tips: use Ctrl+Shift+C to copy and Ctrl+Shift+V to paste from inside the terminal, and use Tab to autocomplete commands or filenames)

Pre-setup VM

You will need to prepare and install virtualization tools before setting up your virtual machine. If you have already set everything up, skip here.

Enable Intel VT-d/AMD Vi virtualization extension

Most laptops nowadays already have this extension build into their chipset. To enable it, you need to go to your UEFI/BIOS settings and search for the option that something relates to virtualization, which they may have different naming scheme in different settings tab. (example from the picture below)

Usually you can hit function keys (F10, F2, Delete, etc.) when rebooting. If you don't know how to do this, consult with your laptop manufacturer or read their user guides on how to boot to BIOS/UEFI. (of course, you know this when you installed Fedora, right?)

Install virtualization tools

Install virtualization tools package by typing:

sudo dnf update # update your system first
sudo dnf groupinstall "Virtualization"

The virtualization tools such as virt-manager, QEMU, and libvirt will be installed to your system, which is required to run Windows 10/11 VM with GPU passthrough.

Then, start libvirtd service:

sudo systemctl enable libvirtd
sudo systemctl start libvirtd

Enable IOMMU grouping

IOMMU is part of Intel/AMD virtualization technology (VT-d/AMD-Vi) that can be used to passthrough some PCI devices. In this case, your dGPU. IOMMU groups PCI devices to ensure your hardware can passthrough your VM properly. NVIDIA has two PCI devices: the VGA controller (or 3D controller for MUXless), and an audio device. You can check by typing lspci | grep NVIDIA or lspci -s 01:00..

If you have AMD CPU, you don't need to do anything because it is already enabled by your Linux kernel.

However, if you have Intel CPU, it isn't enable by default. Edit your kernel parameter through GRUB2 configuration, by typing:

sudo nano /etc/default/grub # you can use vim/emacs if you want

Then, insert/append this code at the end of the GRUB_CMDLINE_LINUX line (without 3 dots):

... intel_iommu=on # insert before the end of the quotation mark (")

Then, apply your GRUB2 config and then restart your laptop:

# UEFI, for most modern laptop that shipped with Windows 8+
sudo grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg 
# BIOS, if you have older laptop that shipped with Windows 7 or older
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

# After GRUB has done their thing, restart your laptop:
sudo shutdown -r now

To confirm that you have enable IOMMU, copy and paste this block of code straight to your terminal:

#!/bin/bash
shopt -s nullglob
for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

If you see something like this:

IOMMU Group 9:
	01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117M [GeForce GTX 1650 Ti Mobile] [10de:1f95] (rev a1)
	01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10fa] (rev a1)

That means your IOMMU grouping is working. If not, you may need to redo those steps above properly.

If, for some reason the IOMMU does not group those PCI properly - for example, a random PCI device grouped with NVIDIA out of nowhere, you should return the device immediately and blame the laptop manufacturer for shipping a broken motherboard. I doubt this will happened to your laptop, though there is ACS patch that can be used but that mostly applies only for PCs, not laptops.

Setting up Windows 10/11 VM

In this section, you will need to set up and configure Windows 10/11 VM, and then go through the OOBE (Windows Out of Box Experience) setup before doing the passthrough process. You can setup the VM entirely through command line by using QEMU, but I'll be using virt-manager for this tutorial, because it is easy for for first-time users.

*If you know about these steps already, you can skip ahead to the next section

Before launching virt-manager, you need to download the Windows ISO file first from Microsoft website. You can get through here: Windows 10 ISO or Windows 11 ISO. Then, create a new folder in your home directory to store the file. Name the folder something like "VM". This folder will be useful to store some ISO files and also creating virtual hard disk which we will need them later.

You also need to download virtio-win drivers from Fedora if you want to increase your VM image disk and virtual network card performance. You can visit here and select Latest virtio-win ISO and it will download the file. Don't forget to put it into your VM folder.

Initial setup

Launch virt-manager in the application menu. You might need to enter your user and password if prompted for root elevation. Then, create a new VM.
Then, choose Local install media option.

Click Browse and select your Windows ISO file (in your VM folder)

Click the + button at the bottom left, then choose the pool's name (eg: myVM), and then locate the folder you just made in the home directory.

Then, choose the pool that you've created, select the Windows ISO file and click Choose Volume. Virt-manager will detect the type of operating system of the ISO file automatically.

This dialogue box might appear, something to do with file search permission. Just check the box and click Yes.

Temporarily configure the amount of memory and CPU core depending on how much available in your system. You can put at least 4GB of memory, which is a bare minimum for Windows 10/11, with 2 or 4 CPU core. Don't worry, you can configure these settings again later.

Setting up virtual disk

Create a virtual hard disk. By default, it will create the hard disk in the default directory /var/lib/libvirt/images, but it is recommended to point to your own directory instead (the "VM" folder you just created).

Select the "Select or create a custom image" option, then click Manage

Select your pool, then press + at the top center besides "Volume"
Name your virtual disk, and enter the number of the capacity. Make sure the format qcow2 is selected. *You can use raw format too. Some people say it actually increase performance, but you will not be able to save "snapshots" of your VM, which is like a save state for your VM in case you want some backup.

Select the disk that you have just created. Then click Choose Volume. Make sure the path is right in the main window.

Finally, name your virtual machine with anything you like (without whitespaces). There are some configurations that you need to change later, so tick the checkbox Customize configuration before install. Then, click Finish.

More VM configuration

In the Overview tab, you can fill out some basic details for your VM. Make sure to select Q35 as your chipset, and change the firmware to UEFI x86_64: /../OVMF_CODE.fd, or OVMF_CODE.secboot.fd if you are installing Windows 11 to enable secure boot. Click Apply if you're done.

In the CPUs tab, you have to manually set the CPU configuration. Tick the checkbox Manually set CPU topology, and edit your sockets, cores and threads. You should not use all of your CPU cores and threads, and leave only one or two cores for your host system. In this case, I use 5 cores instead of 6 in my CPU. Click Apply if you're done.

If you do not how to specify those properties, you can type lscpu in your terminal, and check under the VendorID. This is the result in my terminal:

Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen 5 4600H with Radeon Graphics
    CPU family:          23
    Model:               96
    Thread(s) per core:  2
    Core(s) per socket:  6
    Socket(s):           1

In the Memory tab, make sure your laptop has enough memory to allocate for your VM. Decrease or increase the amount if necessary. "Enable shared memory" is necessary if you plan to use Looking Glass, which you'll enable it later.

In the SATA Disk 1 tab, this is your virtual disk that you have created just now, and change the "Disk bus" type from SATA to virtio. ~~Leave other options by default~~ In advanced options, you need to change "Cache mode" none and "Discard mode" to unmap . Also, check the SATA CDROM 1 section, and make sure it is read-only under "Advanced options". You do not need to change anything else.

In the NIC tab, change the device model from e1000e to virtio to increase network performance. Make sure the link state is active (~~you can disable this to trick Windows 11 to setup offline account~~ this method does not work in the current Windows 11 Home Edition unfrotunately).

In Video QXL tab, make sure QXL is selected.
Add virtio-win ISO file as a secondary CDROM. Click Add Hardware in the bottom left corner, select Storage in the left list. Select virtio-win ISO file (same directory as your Windows ISO file), then select Device Type to CDROM device. Lastly, enable read-only under advanced options, then click Finish to add the hardware.

(For Windows 11 installation) To emulate TPM, click Add Hardware, then select TPM. The "Type" field should be Emulated. and under the advanced option, select ~~CRB~~ TIS as the model and choose version 2.0 as stated from Windows 11 requirement. Click Finish to add the hardware.

Make sure everything is configured properly. If you are satisfied with the configuration, click Begin Installation at the top left corner. When the screen starts to pop out, you must hit any key when asked. If you have an error while starting the VM, you should try to diagnose and troubleshoot before proceed.(you can google the error codes, or comment down below if you have any problem and cannot proceed after this step).

Windows Installation

The windows installation procedure is similar to a normal Windows installation in a bare metal hardware. You need to use the screen provided by virt-manager to interact with the installation process.

However, if the display is too small for you (for folks who uses GNOME fractional scaling or 4k display), you can use Remote Viewer in the Application menu. Enter spice://localhost:5900 then press Connect.

I personally choose Pro edition because it is easier to set up my local account instead of using Microsoft live account (Windows 11 only), and also enable Hyper-V virtualization so I can spin up another VM inside this VM setup. You can also choose Home edition if you wish.

CAUTION: I DO NOT recommend enter any product key as you are likely to mess up your VM or you might find yourself constantly delete/create this VM and configure it to your liking. There is a way to copy your product key embedded from your laptop (ACPI table) and paste it into this VM, but I won't cover here in this tutorial. You can check here if you are interested.

NOTE: You may find that the disk is missing after you choose Custom: Install Windows only (advanced). In this case you need to do the following:

Click Load driver at the bottom left. Then click Browse.

Under This PC > select the folder under CD Drive named virtio-win > viostor > w10/w11 > amd64. Click OK. (w11 if you are using Windows 11, w10 if you are using Windows 10)

The Red Hat VirtIO SCSI Controller should be selected. Click Next.

Now, the Drive 0 Unallocated Space should appear, thought we are not done yet.
There is another driver that you need to install for your VM networking to work. Just like before, select Load driver > Browse > CD Drive named virtio-win > NetKVM > w10/w11 > amd64 and click OK. The Red Hat VirtIO Ethernet Adapter should appear, and select it to install.

Select Drive 0 Unallocated Space and click Next to begin installation.
After the installation, it will reboot by itself (sometimes even reboot several times). Now you have to go through the OOBE process to initialize your Windows operating system. After everything is done initializing, you can shutdown the VM to begin GPU passthrough process.

PRO TIPS: If you are using Windows 11 Pro Edition, select "Set up for work or school" option, then click Sign-in options > Domain join instead, which it will prompt you to set up local account instead.

If you want to clean and debloat your Windows 10/11, you can refer to this guide here by Chris Titus, which I won't be cover here.

Using supergfxctl to bind vfio to GPU

Supergfxctl is a very useful tool to switch your graphics mode, and at the same time it can also bind vfio driver to your dGPU on demand. However, when switching modes, it requires you to log out and log back in to your account which can be slightly inconvenient, but unfortunately that just how it works. You can also switch mode during boot by edit kernel parameter via GRUB menu.

This tool is incompatible with other graphics mode switching such as Envycontrol. If you have one, refer here to completely uninstall it.

Installing supergfxctl

Since Fedora doesn't have this tool in their repository, you have to build it from source by yourself.

Get development tools by copy & paste this line of code in your terminal:

sudo dnf upgrade && sudo dnf install curl git && sudo dnf groupinstall "Development Tools"

This tool also requires Rust as dependency:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

Make a folder called build(or any name that you want) in your home directory. This is just a good practice to reserve a folder to any tools that you want to build from source. Then, clone supergfxctl's repository.

mkdir ~/build && cd ~/build
git clone https://gitlab.com/asus-linux/supergfxctl.git

Proceed to install supergfxctl to your system:

cd supergfxctl
make && sudo make install

Then enable the service:

sudo systemctl enable supergfxd.service --now

Add GUI mode switch (optional)

If you like using GUI instead of using terminal to switch your graphics mode, you can install them via GNOME extension or KDE widget.

For GNOME users, you might consider Super Graphics Control by krst. You can check it out here.

To install, get Extension Manager from Sofware app if you don't have one.

After installing Extension Manager, click Browse at the top center, then search "supergfx". It will automatically show up the first result, which is Super Graphics Control by krst. Click Install.

The UI button "H" at your toolbar should appear at the top right. If you can't see it, make sure to install AppIndicator and KStatusNotifierItem Support as well. The UI might be look like this (minus the "vfio" thing, you need to enable it in the next step):

For KDE users, you might consider an extension called supergfxctl-plasmoid by Jhyub. You can check it out here.

In your terminal, enable Copr repository by using this line of code (if this is the first time you added a Copr repository, the terminal might ask you proceed your own risk or something like that. just accept the prompt and you will be fine):

sudo dnf copr enable jhyub/supergfxctl-plasmoid

Then, install the KDE extension:

sudo dnf install supergfxctl-plasmoid

Right click in your taskbar and select Show Panel Configuration. Then select + Add widgets, search for the name "supergfxctl". Then, drag the extension to wherever you like. (I prefer it next to my system tray)

The icon "H" should be appeared to where you dragged it. When you click the icon, the UI should appear like this (minus the "vfio" thing, you need to enable it in the next step):

Add vfio mode to supergfxctl

Now, you need to configure the tool to add vfio mode. To do that, you need to edit /etc/supergfxd.conf.

Edit the config as super user:

sudo nano /etc/supergfxd.conf # or vim if you like

Change vfio_enable value to true. If you want persistent vfio mode on reboot, you can also set vfio_save to true (recommended). The final config should look like this:

{
  "mode": "Hybrid",
  "vfio_enable": true,
  "vfio_save": true,
  "compute_save": false,
  "always_reboot": false,
  "no_logind": false,
  "logout_timeout_s": 180,
  "hotplug_type": "None"
}

Save the configuration. You may need to restart the service to make changes:

sudo systemctl restart supergfxd.service

You can check whether your laptop can switch to Vfio mode:

supergfxctl -s

The terminal should returned [Integrated, Hybrid, Vfio]. For X11, you also have two additional modes which are Dedicated and Compute. If you use GNOME or KDE extension, the "vfio" mode will appear.

Switch to vfio mode

Make sure to switch to integrated mode before set it to vfio mode, as per tool's instruction (or switch via GUI):

supergfxctl -m Integrated # Yes, it's the big "I"

This mode switch from hybrid > integrated requires logout. After you login back to your account, make sure you don't have NVIDIA card running in your system by typing in your terminal:

nvidia-smi # if you have xorg-x11-drv-nvidia-cuda installed

Should return this line (this error is normal if the mode disables NVIDIA card, which means it worked):

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Or, check your nvidia card via lspci | grep NVIDIA, should return nothing.
In case switching gpu mode does not work, that means your desktop environment killed the supergfxctl task. Then you might need to use this line of code here and repeat the steps above:

sudo sed -i 's/#KillUserProcesses=no/KillUserProcesses=yes/' /etc/systemd/logind.conf

Now, switch to "vfio" mode, either via GUI or in terminal after you switched to integrated:

supergfxctl -m Vfio # Yes, it's the big "V"

To confirm that the vfio driver has binded your GPU, type lspci -ks 01:00.. The result would be like this:

01:00.0 VGA compatible controller: NVIDIA Corporation TU117M [GeForce GTX 1650 Ti Mobile] (rev a1)
	Subsystem: Lenovo Device 3a44
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau, nvidia_drm, nvidia
01:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)
	Subsystem: Lenovo Device 3a44
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

If the line under Kernel driver in use is vfio-pci, then it is success!

To use your laptop normally, you can switch back to Hybrid after you have done using your VM.

Post VM setup with GPU passthrough

Now you can add your NVIDIA GPU PCI devices into your VM, then install NVIDIA drivers in Windows 10/11. Before doing this though, you need to edit your VM's XML config first.

Add dGPU to the hardware list in VM

Make sure to check your libvirt version because some version requires different code snippet which I'll show you later. As of writing this, my version is 8.1.0 which is a little bit older than the latest version. To check your version, open your terminal and type virsh --version.
You also need to get your dGPU's sub-vendor/device ID by typing lspci -nks 01:00.0 , find it under your Subsystem:.

lspci -nks 01:00.0
# Results below
01:00.0 0300: 10de:1f95 (rev a1)
	Subsystem: 17aa:3a44 # your id here
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau, nvidia_drm, nvidia

For example, my subsystem ID is 17aa:3a44. My vendor ID is 17aa, and my device ID is 3a44

Before doing anything, you need to enable XML editing first. In the main window, at the top left, select Edit > Preferences. In the General tab, check the box Enable XML editing, and click Close.

Select your VM, click the bulb icon at the top left corner and then click Add Hardware at the bottom left corner. In the left side of the list, select PCI Host Device and select both PCI devices named NVIDIA Corporation XXXX. After that, click Finish.(if you have another PCI device named just "NVIDIA Corporation", repeat this step again).

To edit full XML, select Overview in the left side of the list, and click XML tab at the top center. In the editing window, change the content of the domain tag at the top line to this line of code (DO NOT HIT APPLY YET!):

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">

(For libvirt version lower than 8.2.0) Scroll down until at the bottom, and between the line </devices> and </domain>, copy and paste this block of codes and edit the value ID:

  <qemu:commandline>
    <qemu:arg value="-set"/>
    <qemu:arg value="device.hostdev0.x-pci-sub-vendor-id=0x17aa"/>
    <qemu:arg value="-set"/>
    <qemu:arg value="device.hostdev0.x-pci-sub-device-id=0x3a44"/>
  </qemu:commandline>
  <qemu:capabilities>
    <qemu:del capability="device.json"/>
  </qemu:capabilities>

Derived from my example above, the sub-vendor-id should be 0x17aa, and sub-device-id would be 0x3a44. Then, click Apply.

(For libvirt version 8.2.0 and above) Scroll down until at the bottom, and between the line </devices> and </domain>, copy and paste this block of codes and edit the value:

  <qemu:override>
    <qemu:device alias="hostdev0">
      <qemu:frontend>
        <qemu:property name="x-pci-sub-vendor-id" type="unsigned" value="6058"/>
        <qemu:property name="x-pci-sub-device-id" type="unsigned" value="14916"/>
      </qemu:frontend>
    </qemu:device>
  </qemu:override>

To get value from the two property, your subsystem ID should be converted from hexadecimal to decimal value. You can go to online converter for this. For example, my vendor id 17aa converts to 6058, and for my device ID 3a44 converts to 14916. Then, click Apply.

Test your VM and install NVIDIA driver

Connect to your external monitor and start your VM. If you don't have any external monitor, you can temporarily use the screen provided by virt-manager or Remote Viewer. (After installing NVIDIA driver, you should install Looking Glass. I will show you how in the next section.)
Check if you have another Display adapter in Device Manager with error sign on it. That means you have successfully passthrough your GPU. (tips: Right Click on Windows button on taskbar > Device Manager)
After booting and login to your Windows, go to official NVIDIA website and download the NVIDIA driver for your appropriate dGPU (or even better, use NVCleanInstall for less bloat and no telemetry). And then install it just like you would normally do on your Windows laptop.

NOTE: Does your VM freezes when installing your NVIDIA driver? Or getting BSOD with error VIDEO_TDR_FAILURE? Refer here for a temporary solution.

After installing NVIDIA driver, go to Device Manager and under Display adapter. There should not have an error if you hover to your dGPU card. If you have an error such as code 43, you can refer here on how to fix it.

VM Enhancements

If you made it here, congrats! You successfully managed to do GPU passthrough in your laptop. You can check out guide below on how to enhance your VM further by squeezing out more performance and some general tips that can help you to get a better user experience. These steps are entirely optional and you can just start using your VM as it is.

Use Looking-Glass

Looking-Glass is a great tool to have if you do not have any external monitor. Unlike using live capture/remote display client such as NVIDIA Stream, Moonlight or FreeRDP; Looking Glass featured insanely low latency and no compression/artifacts because the video memory is shared from Linux to VM directly. More info here.

Even if you have an external monitor, you can take advantage of Looking-Glass to act as a video capture card that can be used to record your Windows VM screen, via OBS on Linux by using OBS plugin.

Before doing anything, you need a HDMI dummy plug for it to work. Or else it will not be at a full resolution that you've set. There is an alternative solution later on, so just follow along this section first.

Build from source

To install Looking-Glass, you need to build it from source as Fedora repository unfortunately doesn't provide the binary package.

Firstly, download the dependencies for Looking Glass. In your terminal, copy & paste this line of code:

sudo dnf install cmake gcc gcc-c++ libglvnd-devel fontconfig-devel spice-protocol make nettle-devel \
            pkgconf-pkg-config binutils-devel libXi-devel libXinerama-devel libXcursor-devel \
            libXpresent-devel libxkbcommon-x11-devel wayland-devel wayland-protocols-devel \
            libXScrnSaver-devel libXrandr-devel dejavu-sans-mono-fonts dkms kernel-devel kernel-headers

If you want to include audio in Looking Glass, you can use one of the lines below depending on your audio system:

# If you are using Pipewire (default in Fedora 36+)
sudo dnf install pipewire-devel pulseaudio-libs-devel libsamplerate-devel
# IF you are using PulseAudio
sudo dnf install pulseaudio-libs-devel libsamplerate-devel

For up to date dependency packages, check here.

Download the package from the website here. Under Official/Stable version, choose Source.
Make a directory called build(or any name that you want) in your home directory if you don't have it. This is just a good practice to reserve a folder to any tools that you want to build from source.

mkdir ~/build

Extract the downloaded package to your build folder using archive manager, or just use this command line:

# depending on your Looking Glass version and where that file is located
tar -xvf ~/Downloads/looking-glass-B6.tar.gz -C ~/build

In your terminal, go to client/build directory from your extracted files.

cd ~/build/looking-glass-B6/client

If you are using GNOME Wayland, install libdecore-devel and use cmake with additional parameter. If you are not using GNOME Wayland, just use cmake as it is.

# run this if you are GNOME wayland user
sudo dnf install libdecor-devel # do this first
cmake -DENABLE_LIBDECOR=ON ../

# run this if you are non-GNOME wayland user
cmake ../

Then finally run with make to build the Looking Glass client. This will take some time to finish.

make

The looking-glass-client should be appeared if you look inside client/build directory. You can type ls to make sure it's there. Copy the file to /usr/local/bin to easily access the application with terminal:

sudo cp looking-glass-client /usr/local/bin

To confirm the client is working, type looking-glass-client -h. It should show up useful commands that you can do. List of commands can be found here as well if you want to know more about them.

Edit VM configuration

If you have virt-manager still opened, go to your Windows VM settings, and in the Memory tab, check the box Enabled shared memory.
In your XML config, scroll down until at the bottom. Between the line <memballoon/> and </devices>, copy paste this block of codes:

<shmem name='looking-glass'>
  <model type='ivshmem-plain'/>
  <size unit='M'>32</size>
</shmem>

Depending on the size, 32MB is plenty enough to display 1080p. If your monitor supports 2K or 4K resolution, you might need to increase it. See this guide to determine your memory size.

Speaking of <memballoon/> line, you need to change the model type to none for performance improvement:

<memballoon model="none"/>

You need to give permission for the Looking Glass shared memory file found in /dev/shm/looking-glass. Create a new file in your terminal:

sudo nano /etc/tmpfiles.d/10-looking-glass.conf # use vim if you like

And copy and paste this block of codes (change user to your username):

#Type Path               Mode UID  GID Age Argument

f /dev/shm/looking-glass 0660 user kvm -

Try start your VM first before proceeding. If you get permission problem regarding /dev/shm/looking-glass, you must set the permission first for that file (change user to your username):

sudo touch /dev/shm/looking-glass # if the file has created, ignore this line
sudo chown user:kvm /dev/shm/looking-glass
sudo chmod 660 /dev/shm/looking-glass

(Thanks @The24thDS for providing the solution) If you have "DMA mapping failed" error, you might want to add this line of code in your VM configuration between <cpu> tag:

<maxphysaddr mode='passthrough' limit='40'/>

If you have other permission problem, see here to know how to fix it.

Installing Looking Glass in Windows VM

When booting, you need to go to BIOS settings before installing a driver by pressing F2 repeatedly.
Go to Device Manager > Secure Boot Configuration > uncheck Attempt Secure Boot. Press F10 to save configuration. Go back to the main menu by ESC, then press Continue.

Now, go to the Looking Glass website and download the Windows Host Binary from here. Make sure the version is the same as your client.
Extract the zip file that you have downloaded and start installing Looking Glass.
After this step, you might need a dummy HDMI plug for Looking Glass to work. There is an alternative solution if you don't have one, which you can refer here. You can return to this section here if you're done.
Everything should be set. Exit the BIOS settings and restart your VM now. Open your terminal and type:

# you can pass more arguments such as ''-F' for fullscreen, but this is just a test
looking-glass-client

Plug in your dummy HDMI (not relevant if you have installed virtual display driver), and it should appear. You can do fullscreen by pressing ScrollLk + F. You can see all keybindings here.
ScrollLk is a "master" key for those bindings (also capture your mouse to VM directly). To change it, refer the number keys here and type looking-glass-client -m <number>. e.g: Right Ctrl is 97, so you should type looking-glass-client -m 97.

That should be it for setting up Looking Glass. You can check out other enhancements here if you want to futher improve your experience.

UPDATE 9/14/2023: There is a new guide on alternative way on share memory buffer using your GPU directly, which uses Direct DMA Upload (DMABUF) instead of IVSHMEM. The benefit of using DMABUF is that it will lessen the workload of your CPU, which could increase your VM performance by 10% while using Looking Glass, maybe.

Though it is slightly more complicated to install it than using IVSHMEM. If you are interested, you can follow this guide here.

Use Remote Desktop client (not recommended)

I do not recommend any network based remote desktop client as their quality is too low, and they are likely to introduce video artifacts compares to Looking Glass. This is a reason that I won't cover this step. Steam Remote Play however does a good job of keeping the latency as low as possible, but you will notice some slight video artifacts when you play a fast moving game. Still, it works if you only use your VM to play games, but not everything else.

Audio passthrough

By default, the virt-manager/QEMU uses HD audio (ICH9) sound driver which can be piped through SPICE server (the screen you use in virt-manager or remote viewer), but the audio quality is terrible and produces high latency. If your monitor supports HDMI audio, you won't need additional steps to do this and just remove the sound driver from virt-manager.

However if you are using Looking Glass B6 version and above, you don't need to do anything, as it already provides the audio passthrough for you. If you have B5 version or below, it is recommended to update your Looking Glass to the latest version.

If, for some reason it does not work in your system, or you don't want to use Looking Glass because you preferred using your external monitor that has no audio output, there are a few options that you can do to achieve this besides Looking Glass or HDMI audio driver:

Use USB sound card (best option, very low latency/overhead)
Use SCREAM audio driver (a good alternative, easy to setup for Windows guest)
Passthrough Pipewire directly (better than pulseaudio, easier to set up for other OS guest)
Passthrough Pulseaudio directly (not a good solution, CPU heavy and poor bitrate)

USB Sound Card

This is the best option if you have a spare sound card. Note that this can be only use for Windows VM and cannot be shared through your host. Most laptops have their own on-board sound card (your built-in headphone jack) and use that for your host.

Plug in your USB Sound Card, then from your VM settings, click Add Hardware > USB Host Device, and choose your sound card. Once added, remove the sound driver (HDA ICH9).

SCREAM audio driver

This is a virtual audio driver in Windows VM which sends through your VM virtual network to passthrough the sound it produces. The latency is very low and produce cleanest sound you can get for virtual audio, which can be piped through your host audio system. The benefit is that the audio from guest can be shared with your host as well with no sound quality loss, unlike using USB Sound card.

Install dependency packages for scream:

sudo dnf install pulseaudio-libs-devel

Make a directory called build in your home directory if you don't have it, or any name that you want. This is just a good practice to reserve a folder to any tools that you want to build from source.

mkdir ~/build

Download Scream git repo:

cd ~/build # go here first
git clone https://github.com/duncanthrax/scream.git

Go to receiver's build directory to start cmake them. This won't take long to build.

cd scream/Receivers/unix/
mkdir build && cd build
cmake ..
make

Copy scream to /usr/local/bin:

sudo cp scream /usr/local/bin
scream -h # check if it's installed in your system

Start your Windows VM if you haven't already. If you didn't install Looking Glass, you might need to disable secure boot first in your VM. (spam F2 while boot > Device Manager > Secure Boot Configuration > uncheck Attempt Secure Boot > F10 to save)
Download the zip file audio driver in the SCREAM's release page.
Extract the zip file, run Install_x86_64.bat. This will install scream audio driver. If an install prompt showed up, click Install.
The scream driver should be installed. You can check it via sound settings in the toolbar at bottom right.
In your linux host, open your terminal and just type:

scream -i virbr0

virbr0 is the default name for virtual network for your VM. If in doubt, type ip link show to know your network interface name.

If something shows up, it means that you have sucessfully passthrough audio from the VM. If you don't hear anything, this might be because of firewall blocking the connection. To remedy this, add a port to the firewall rule:

firewall-cmd --zone=libvirt --add-port=4010/udp
sudo firewall-cmd --runtime-to-permanent # make sure the rule is permanent

Start testing your audio by going to Youtube and play a video, or play a game to see if it works for you.

PipeWire and JACK

Using PipeWire & JACK as a passthrough is also good if you don't want so much hassle installing SCREAM. This is also useful if you plan to spin other VM such as other linux distro or Mac OS using this method. Pipewire and JACK are installed natively in Fedora, so you don't need to do anything besides setting up few things in your VM.

Before anything, make sure QEMU is running as user. Edit qemu.conf:

sudo nano /etc/libvirt/qemu.conf

Find this line, uncomment and change it to your username then save the file. (tips for nano user: Press Ctrl + W and search "user =")

...

user = "bob"

...

Go to VM settings, open your XML window. Scroll down until you find <audio id="1" type="spice"/> line. Replace it with these codes (change win11 to win10 or anything, doesn't matter):

  <audio id="1" type="jack">
    <input clientName="vm-win11" connectPorts="Family 17h/19h HD Audio Controller Analog Stereo"/>
    <output clientName="vm-win11" connectPorts="Family 17h/19h HD Audio Controller Analog Stereo"/>
  </audio>

The Family 17h/19h HD Audio Controller Analog Stereo is my audio port. To acquire your exact port name in your system, install either Carla or Audacity. For me, Audacity is the easiest to identify my audio ports. At the leftmost toolbar, switch from ALSA to JACK Audio Connection Kit. Observe the the name of the input and output:

Then, under </devices>, add these:

  <qemu:commandline>
    <qemu:env name="PIPEWIRE_RUNTIME_DIR" value="/run/user/1000"/>
    <qemu:env name="PIPEWIRE_LATENCY" value="512/48000"/>
  </qemu:commandline>

The /run/user/1000 (1000) is your user ID. If you are not sure what is your user ID, you can check it by typing id in your terminal.
The latency is set at 512/48000 (512 is sample size, 48000 is sample rate) which is enough for most laptop. You probably need to change it depending on your audio card. If you have cracking audio, increase the sample size or change the sample rate according to your pipewire system.
If you don't hear any sound, SELinux might block it from connecting through pipewire. Refer here for the fix.

Use Pulseaudio (not recomended)

I will not cover how to passthrough audio via Pulseaudio because the sound quality is quite poor with constant audio cracking. If you are still insist on doing this for whatever reason, I recommend reading this guide from ArchWiki. (The explanation is quite technical, so proceed at your own risk)

CPU Pinning

CPU pinning improves performance massively because it gives some number of cores in your host CPU to run exclusively on VM. You may not passed every cores as you need one to reserve for your host, or else everything will not be responding and smoothly.

Know your CPU topology

Intel and AMD CPUs have different topology if they support hyper-threading for Intel, or SMT for AMD (1 core each gives 2 threads). Below are examples of 6 cores with 12 threads in total.

In your terminal, type lscpu -e. The output should be like this (my AMD Ryzen 5 4600H):

CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE    MAXMHZ    MINMHZ       MHZ
  0    0      0    0 0:0:0:0          yes 3000.0000 1400.0000 1400.0000
  1    0      0    0 0:0:0:0          yes 3000.0000 1400.0000 1397.3250
  2    0      0    1 1:1:1:0          yes 3000.0000 1400.0000 1396.9139
  3    0      0    1 1:1:1:0          yes 3000.0000 1400.0000 1400.0000
  4    0      0    2 2:2:2:0          yes 3000.0000 1400.0000 1397.3600
  5    0      0    2 2:2:2:0          yes 3000.0000 1400.0000 1396.5480
  6    0      0    3 4:4:4:1          yes 3000.0000 1400.0000 1397.3590
  7    0      0    3 4:4:4:1          yes 3000.0000 1400.0000 1397.2980
  8    0      0    4 5:5:5:1          yes 3000.0000 1400.0000 1394.4351
  9    0      0    4 5:5:5:1          yes 3000.0000 1400.0000 3000.0000
 10    0      0    5 6:6:6:1          yes 3000.0000 1400.0000 1397.4510
 11    0      0    5 6:6:6:1          yes 3000.0000 1400.0000 1397.0450

If you're using Intel, the layout might be like this (Intel Core i7-8700k):

CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    4600.0000 800.0000
1   0    0      1    1:1:1:0       yes    4600.0000 800.0000
2   0    0      2    2:2:2:0       yes    4600.0000 800.0000
3   0    0      3    3:3:3:0       yes    4600.0000 800.0000
4   0    0      4    4:4:4:0       yes    4600.0000 800.0000
5   0    0      5    5:5:5:0       yes    4600.0000 800.0000
6   0    0      0    0:0:0:0       yes    4600.0000 800.0000
7   0    0      1    1:1:1:0       yes    4600.0000 800.0000
8   0    0      2    2:2:2:0       yes    4600.0000 800.0000
9   0    0      3    3:3:3:0       yes    4600.0000 800.0000
10  0    0      4    4:4:4:0       yes    4600.0000 800.0000
11  0    0      5    5:5:5:0       yes    4600.0000 800.0000

*Note: Intel 12th Gen processors and up have different topography than the generations before it, with the presence of p-cores and e-cores. Someone made a post here on how they successfully pinned their CPU.

Note that the link between "ID" of CPU and CORE. If I use the example above, AMD/Intel links their core with threads would be like this:

AMD	Intel	(n)th Core
`0,1`	`0,6`	1
`2,3`	`1,7`	2
`4,5`	`2,8`	3
`6,7`	`3,9`	4
`8,9`	`4,10`	5
`10,11`	`5,11`	6

If I want to isolate them, I would use 2nd-6th cores for VM, and 1st core reserves for host. Remember these setups, as you need to configure them later on:
- To isolate AMD CPU would be: 2-11, leave 0,1
- To isolate Intel CPU would be: 1-5, 7-11, leave 0,6
- To config AMD CPU in XML: 2,3,4,5,6,7,8,9,10,11
- To config Intel CPU in XML: 1,7,2,8,3,9,4,10,5,11
If you want a visualization of how your CPU topology looks like, run lstopo:

sudo dnf install hwloc-gui # install this first
lstopo

Mine looks like this:

Edit your VM configuration

If you have VM settings window open, under CPUs tab, make sure that you define your CPU topology according to how many cores you wanted to pass through.
Go to XML editing window, find the <vcpu/> line. Replace it with one of snippets below:
For AMD Ryzen 5 4600H, passing the last 5 cores to VM:

  <vcpu placement='static'>10</vcpu>
  <iothreads>1</iothreads>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='7'/>
    <vcpupin vcpu='6' cpuset='8'/>
    <vcpupin vcpu='7' cpuset='9'/>
    <vcpupin vcpu='8' cpuset='10'/>
    <vcpupin vcpu='9' cpuset='11'/>
    <emulatorpin cpuset='0-1'/>
    <iothreadpin iothread='1' cpuset='0-1'/>
  </cputune>

For Intel Core i7-8700k, passing the last 5 cores to VM:

  <vcpu placement='static'>10</vcpu>
  <iothreads>1</iothreads>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='7'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='8'/>
    <vcpupin vcpu='4' cpuset='3'/>
    <vcpupin vcpu='5' cpuset='9'/>
    <vcpupin vcpu='6' cpuset='4'/>
    <vcpupin vcpu='7' cpuset='10'/>
    <vcpupin vcpu='8' cpuset='5'/>
    <vcpupin vcpu='9' cpuset='11'/>
    <emulatorpin cpuset='0,6'/>
    <iothreadpin iothread='1' cpuset='0,6'/>
  </cputune>

Configure only cpuset depending on how much core you wanted to pass through this VM. vcpu number increases incrementally if you pass more cores. Arrangement is important!
<iothreads/> and <iothreadpin/> is useful when your storage is using virtio driver. Do not use this when you don't configured it!
Now try starting your VM. If it doesn't complain anything, that means it is working! You can inspect your CPU process by using htop if you have it installed, or use built-in system monitor if you're using GNOME/KDE desktop environment.

Isolate CPU cores

Isolating CPU cores is another major performance improvement to go along with CPU pinning. This is done to make sure that the cores that you want to use for VM will not be used in your host so there are no hiccups in your system.

This can be done either by configure your kernel parameter or via basic libvirt hook, which is an automation tool that can be used when starting/stopping your VM. I recommend choosing latter if you are planning to use your host extensively. If you want total isolation, you can refer here instead.

Create libvirt hook script

Open your terminal, and create a libvirt hooks directory:

sudo mkdir -p /etc/libvirt/hooks

Create a file called qemu:

sudo nano /etc/libvirt/hooks/qemu # or vim if you like

Copy and paste this lines of codes to the file (for AMD use 0-1, Intel use 0,6 depending on how much core you have pass through):

#!/bin/sh

command=$2

# When VM starts, the core from 0 to 1 will be used for host (0,6 if Intel)
if [ "$command" = "started" ]; then
    systemctl set-property --runtime -- system.slice AllowedCPUs=0-1
    systemctl set-property --runtime -- user.slice AllowedCPUs=0-1
    systemctl set-property --runtime -- init.scope AllowedCPUs=0-1
# If VM stops, the host will reclaim all cores
elif [ "$command" = "release" ]; then
    systemctl set-property --runtime -- system.slice AllowedCPUs=0-11
    systemctl set-property --runtime -- user.slice AllowedCPUs=0-11
    systemctl set-property --runtime -- init.scope AllowedCPUs=0-11
fi

Make it executable and restart libvirtd service to apply:

sudo chmod +x /etc/libvirt/hooks/qemu
sudo systemctl restart libvirtd

Try starting your VM and see your htop or system monitor. If the script works, you will see that some of the cores are in 0% utilization at Windows startup. That means it's working.

Miscellaneous optimizations

Some optimizations that may or may not improve your VM performance.

Switch to performance mode for your CPU

Your CPU governor will use their settings by default, which can lower the performance if you wanted to run your VM.

I recommend using auto-cpufreq by @AdnanHodzic to change your CPU mode on the fly. Refer to the instruction on how to install them and create a config for the tool. I recommend set it to performance mode when charged and use it while spinning up your VM, and powersave mode if unplugged to save more battery for your laptop.

Enable SMT for AMD CPUs in VM

This is for AMD CPUs only if you want hyper threading performance uplift. Inside your XML, find the tag <cpu>. Add a TOPOEXT feature like below:

<cpu mode='host-passthrough' check='none'>
  <topology sockets='1' cores='5' threads='2'/>
  <feature policy='require' name='topoext'/>
</cpu>

Total CPU isolation

Sometimes dynamic isolation via libvirt hook does give a bit lower performance than using static isolation via kernel parameter. I recommend use your system GRUB menu (when you boot up your laptop) to add them instead of permanently isolate them which host can only use one core all the time.

To edit your kernel parameter, reboot your laptop and press Right-Shift (or Esc if it didn't work, not sure what is the correct key) repeatedly to force GRUB menu to show up. Select latest linux kernel in the option, then press E.

Under the line linux /boot/..., append this line of code (the number below refer to cores that you wanted to use for virtualization, not host):

... isolcpus=2-11 nohz_full=2-11

This will isolate your cores completely from the start, so that your VM can claim those cores for maximum performance. This changes is only temporary as your next boot will reset those changes.

If you want to make it permanent, refer the GRUB instruction here.

Static/Dynamic hugepage?

I've seen a lot of guides telling you to set up static/dynamic hugepage, however I also seen some people claim that it doesn't make much difference in terms of performance. If you think it will increase your VM performance or wanted to try it out first, check this guide here by asus-linux team.

I need more tips!

This guide doesn't cover everything though. You might be interested here, which has more things to cover, written by the developer of asus-linux and supergfxctl.

You can also look here on how to squeeze your gaming performance in your VM, a great article made by Mathias Hueber.

FAQs & Troubleshooting

Blank screen output when I passthrough my GPU. How?

If there is no output, you might need to extract your vBIOS from your laptop to be used in your VM. (likely will happen if your laptop is configured MUXless)
You might want to follow this guide. You need to find your system's BIOS update from your manufacturer's website (some update files may not provide the vbios file).
If you dual boot Windows 10/11, you can follow this guide here (look at on board GPU section).

Go back to Before we proceed.

My Windows VM freezes after installing NVIDIA driver

You may have issues when booting up your Windows VM if your laptop starts in hybrid mode and then switch to integrated, then vfio mode. The problems such as the remote display not appearing at all (black screen), VM constantly freezes and restarts, or Windows BSOD return error VIDEO_TDR_FAILURE (which is a common occurrence for my laptop unfortunately). My guess is that the dGPU was not properly shut down or the kernel driver just bugged out when switching to integrated mode.

If these problems persist, please restart your laptop after switching to integrated mode, then your Windows VM will be fine and NVIDIA driver should be installed. You can also switch modes from the GRUB menu which you can refer here on how to do it.

The problem doesn't appear when you boot your laptop with integrated or vfio mode already, which then you are safe to boot your VM (which is why vfio_save is recommended).

When you are done using your VM and then switch to hybrid mode, your Linux host can use your dGPU no problem (though the problem might appear again if you switch back to integrated).

If you can identify the problem (and also have a solution), please share in the comment section below. (try this maybe ¯\(ツ)/¯)

Go back to Test your VM and install NVIDIA driver.

How to switch supergfxctl mode using kernel parameter?

To edit your kernel parameter, reboot your laptop and press Right-Shift (or Esc if it didn't work) repeatedly to force GRUB menu to show up. Select latest linux kernel in the option, then press E.

Under the line linux /boot/..., insert/append this (use End button to move your typing cursor at the rightmost position in that line):

... supergfxd.mode=vfio

Of course, you can change vfio to integrated or dedicated if you wish (not case-sensitive).

Go back to Post VM setup with GPU passthrough

Uh, I follow your Envycontrol's instruction before, how to undo?

If you follow my instructions on using Envycontrol to switch VFIO before, you may need to do as follow to remove envycontrol entirely in terminal:

Switch back to hybrid mode: envycontrol -s hybrid then sudo reboot.
After reboot, make sure to type envycontrol -q. It should return hybrid.
Delete these files via terminal:

sudo rm /etc/modprobe.d/nvidia.conf /lib/udev/rules.d/50-remove-nvidia.rules /lib/udev/rules.d/80-nvidia-pm.rules /usr/local/bin/envycontrol

Do sudo reboot. Now envycontrol removed completely from your system. Make sure the command envycontrol returns command not found... .
If you have completed all the steps for my guide before I changed my instruction, you need to install supergfxctl. Click the second hyperlink below to get started. Or if you haven't, ignore this message.

Go back to Before we proceed or Using supergfxctl to bind vfio to GPU

I got error code 43! How to fix? (fake battery fix)

If you have problem installing NVIDIA and you got the error code 43, you might need to add fake battery to your VM to solve the error.

Copy this base64 code and decode it to file here:

U1NEVKEAAAAB9EJPQ0hTAEJYUENTU0RUAQAAAElOVEwYEBkgoA8AFVwuX1NCX1BDSTAGABBMBi5f
U0JfUENJMFuCTwVCQVQwCF9ISUQMQdAMCghfVUlEABQJX1NUQQCkCh8UK19CSUYApBIjDQELcBcL
cBcBC9A5C1gCCywBCjwKPA0ADQANTElPTgANABQSX0JTVACkEgoEAAALcBcL0Dk=

Rename it to SSDT1.dat and put it in your VM folder (make sure the folder is the same as your ISO file).
If you have your VM details window still open, head over to XML edit page. Make sure the first line, <domain> has changed into:

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">

Then, scroll down and between the line </devices> and </domain>, insert this code snippet and click Apply:

  <qemu:commandline>
    <qemu:arg value="-acpitable"/>
    <qemu:arg value="file=/home/bob/VM/SSDT1.dat"/>
  </qemu:commandline>

Try start your VM. If you have problem starting your VM because of permission error, you can refer below to fix the problem.

Go back to Test your VM and install NVIDIA driver

I got permission denied or something when I start my VM.

This is due to SELinux uses "Enforce" mode by default in Fedora. You can set it to permissive, but you should give permission only for QEMU. In your terminal, enter these commands:

sudo -s # login as root
cd ~ # go to /root folder because the command below will create a file
     # that only root can access. you don't want the file in your user directory
ausearch -c 'qemu-system-x86' --raw | audit2allow -M my-qemusystemx86
semodule -X 300 -i my-qemusystemx86.pp
setsebool -P domain_can_mmap_files 1

UPDATE 2/29/2024: If, for some reason one of these lines gave you some errors, then this step is likely outdated. To mitigate this problem, I have an alternative solution.

In your terminal, open the file called /etc/libvirt/qemu.conf:

sudo nano /etc/libvirt/qemu.conf #use vim if you like

Search for the line #security_default_confined = 1 (tips: in nano editor, use Ctrl+W then search security_default then press enter. It should come up instantly), then uncomment it then change from 1 to 0.

security_default_confined = 0

Because of this change, when you start virt-manager, it will no longer asking you for a password. It's less secure, but at least it doesn't complain any permission errors. It should be fine when you are the only person who operates your Windows VM.

Go back to Installing Looking Glass in Windows VM or PipeWire & JACK

I still got error 43 even after battery fix! How???

If you happened to have older graphics card, NVIDIA might forced you to use older version of driver which doesn't have error 43 fix provided by them. In this case, check out this blogpost to solve the problem.

I don't have dummy HDMI plug for Looking Glass...

If you don't have one, you might be interested in this solution by u/ImaginationLatter523. Just in case if the post got deleted or changed for some reason, I'll leave them here:

Start your VM if you haven't, and download custom IddSampleDriver here, and extract it to C:\ (your root C: drive).
Open your command line, either cmd or PowerShell if you like. Run these commands:

cd C:\IddSampleDriver
CertMgr.exe /add IddSampleDriver.cer /s /r localMachine root

Edit C:\IddSampleDriver\option.txt first before installing. You should edit the resolution and refresh rate according to your laptop's screen.
After that, go to Device Manager (tips: right click Windows icon), then click on any device. At the top panel, select Action > Add legacy hardware.
Then click Next > choose the link Install hardware that I manually select from a list (Advanced). Click Next while Show all devices is selected, select Have disk > Browse.
Find C:\IddSampleDriver\IddSampleDriver.inf, select it and click Ok. Then continue clicking Next. Wait for it to install, and then click Next again to exit the setup.

Go back to Installing Looking Glass in Windows VM

I can't get anti-cheat games to work!

Most games equipped with anti-cheat are notoriously for not working under a virtual machine. However, there are several solutions that you can use to get around them, albeit will not work on intrusive or ring 0 anti-cheats (eg: Riot's Vanguard for Valorant).

There is a great documentation on how to (atleast) bypass EAC anti-cheat provided by VRChat developers. See here for the instruction.

For other anti-cheat, you may need some additional steps to hide your virtual machine. For instance, you need to enable "core isolation" under Settings -> Update & Security -> Windows Security -> Device Security. However, this will impact performance on Intel CPUs, and massive performance drop for AMD CPUs (your OS might start lagging terribly, therefore not recommended).

You can do further by passing through a real hard drive and install Windows there, and also use external monitor all the time, avoid using Looking Glass and remove SPICE server or any remote client software.

For advanced users who are very familiar with the inner workings of QEMU/KVM and Linux kernel, you can further spoof your VM by passing more smbios info, patch QEMU source code to change hardware names and also patch RDTSC timing. This is by far the hardest if you don't know how to compile a linux kernel. (Note that you have a high risk of get yourself banned from video games that you want to play)

Full stealth KVM resource can be found here. Only do this if you know what you're doing. Same thing for RDTSC timing patch, which you can follow this instruction on how to do it.

Does this guide works for other distro (Ubuntu, Arch, etc.)?

It should work, but there are some steps that need to be modified on certain distros. Do it at your own risk, and you should do some research & troubleshoot by yourself.
For example, choosing OVMF_code.secboot.fd in virt-manager might not be present in other distro as this is exclusively for Red Hat distros repository (which Fedora also includes). Some distro don't even have OVMF in their repository.
Other example, such as SELinux fix might not be for other distro, but they have their own security tool such as AppArmor which requires different ways to fix permission problems. ~~- I might add a guide for Arch Linux in the future, if I have more time of course.~~ I clearly don't have much time lmao. You are welcome to make your own guide though!

How about other editions of Fedora?

It should work well, but I personally did not test them yet. Any fedora spins should work, if not the same as normal Workstation edition.
Fedora Silverblue/Kinoite might need some slightly different steps, as it is an immutable OS. It might be difficult to do depending on certain steps. They have RPM-ostree and toolbox for package installation, so that adds more difficulty to follow this guide. There is one person who actually successfully passthrough their GPU with their custom Fedora Silverblue, which you can read it here.
Nobara Project should work as well, in fact it some steps aren't required. For instance, it has ~~SELinux sets to "Permissive"~~ (now it uses AppArmor for some reason), applied ACS patch, and supergfxctl installed by default.

firelightning13/[GUIDE] GPU Passthrough for Laptop with Fedora.md