Enable Single GPU PCI Passthrough

Enable Single GPU PCI Passthrough.txt

#############################################
##  +   Enable Single GPU PCI Passthrough   +
#############################################


## System Spec

# Ryzen 3600
# Asus B450-F
# Corsair 2x8GB 3200MHz CL16
# HP EX920 1TB
# Zotac GTX960


##  BIOS Settings

# IOMMU = Enable
# SR-IOV = Enable
# SVM / AMD-V = Enable
# Power supply idle control = Low current idle


##  Take a note of your GPU / PCIe HW IDs

❯ lspci | grep NVIDIA
09:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1)
09:00.1 Audio device: NVIDIA Corporation GM206 High Definition Audio Controller (rev a1)
❯ lspci -n -s 09:00
09:00.0 0300: 10de:1401 (rev a1)
09:00.1 0403: 10de:0fba (rev a1)

# ID 10de:1401 & 10de:0fba would be required later for the setup


##  Update Grub: find the following lines below and edit those

❯ cat /etc/default/grub
# ...
GRUB_TIMEOUT=5
# ...
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt video=efifb:off"
# ...

❯ update-grub
❯ reboot


#   Verify that IOMMU is working

❯ dmesg | grep -e DMAR -e IOMMU
[    1.008249] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    1.009439] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    1.009727] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).


##  Add the following to kernel modules

❯ cat /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd


##  Blacklist drivers from loading on Proxmox

❯ cat /etc/modprobe.d/blacklist.conf
blacklist nvidia
blacklist nvidiafb
blacklist nouveau
blacklist radeon
blacklist amdgpu


##  Disable Passthrough GPU from loading on Proxmox

❯ cat /etc/modprobe.d/vfio.conf
# Make vfio-pci a pre-dependency of the usual video modules
softdep nouveau pre: vfio-pci
#softdep nvidia pre: vfio-pci
#softdep nvidiafb pre: vfio-pci
#softdep amdgpu pre: vfio-pci
#softdep radeon pre: vfio-pci

# Have vfio-pci grab the GTX960 device IDs on boot
options vfio-pci ids=10de:1401,10de:0fba disable_vga=1


## Update initramfs images

❯ update-initramfs -u
❯ reboot now


## Double check to verify that everything is working

❯ dmesg | grep -e DMAR -e IOMMU
[    0.826802] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.827765] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.828118] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

❯  lspci -nnk -d 10de:1401
09:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM206 [GeForce GTX 960] [10de:1401] (rev a1)
        Subsystem: ZOTAC International (MCO) Ltd. GM206 [GeForce GTX 960] [19da:1375]
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau


## If everything went OK then you should be able to passthrough your GPU to a VM


#EOF