GPU-passthrough-setup-on-proxmox.md

https://pve.proxmox.com/wiki/PCI_Passthrough

• check this

'Legacy boot' or CSM: For GPU passthrough it can help to disable this, but keep in mind that PVE has to be installed in UEFI mode, as it will not boot in BIOS mode without this enabled. The reason for disabling this is that it avoids legacy VGA initialization of installed GPUs, making them able to be re-initialized later, as required for passthrough. Most useful when trying to use passthrough in single GPU systems.

Important to follow this

https://forum.proxmox.com/threads/gpu-passthrough-with-rtx-3090-doesnt-work-dmar-errors.120677/

• RTX-3090-GPU passthrough
• #3

Hey! So combining some info I got here as well as another forum, i got the proxmox to passthrough my gtx3090 perfectly(with screen output as well)

I followed this tutorial:
https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

However it did not cover blocking your gpu devices from being completely blocked by the grub bootloader so, I had to take these steps:

You get the info for the manufacturer (vfio-pci-id's) like this

lspci -nn | grep -i nvidia

then take the info and apply it to the grub file:

nano /etc/default/grub

Code:

"quiet intel_iommu=on iommu=pt nomodeset initcall_blacklist=sysfb_init vfio-pci.ids=10de:2204,10de:1aef disable_vga=1"

changed from this when it wasn't working 22/7

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset initcall_blacklist>

In the end it should look similar to this:

Code:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset initcall_blacklist=sysfb_init video=vesafbff,efifbff vfio-pci.ids=10de:2204,10de:1aef disable_vga=1"

then update grub
update-grub

reboot the system:
reboot

shout out for @leesteken for calling out my mistakes, I edited and this config works fine and the other vm's seem a bit more stable.

---

Here are the steps to set up GPU passthrough from Proxmox to an Ubuntu VM. This process involves first configuring your Proxmox PVE host and then setting up your Ubuntu VM.

On your Proxmox PVE host:

1. Enable IOMMU: To use PCI passthrough, you need to enable Intel IOMMU (also known as VT-d). This can be done in your BIOS settings. The exact method to do this depends on your motherboard, but generally, it's under the Advanced/CPU options in the BIOS.

• 2. Enable IOMMU in Proxmox: Once the IOMMU is enabled in BIOS, you need to enable it on your Proxmox host. Open /etc/default/grub with a text editor, look for the line that starts with GRUB_CMDLINE_LINUX_DEFAULT and add intel_iommu=on at the end of the quotes. Then, update GRUB with the command update-grub and reboot your Proxmox host.

• 3. Identify your GPU: You need to identify the PCI addresses of your GPU and its audio device. You can do this by running lspci -n -s $(lspci | grep VGA | cut -d' ' -f 1) in the terminal. You will get an output like 01:00.0 0300: 10de:1b81 (rev a1) and 01:00.1 0403: 10de:10f0 (rev a1). Note these addresses, you'll need them in the next step.

root@pve:~# lspci -n -s $(lspci |grep VGA |cut -d' ' -f 1)
01:00.0 0300: 10de:2204 (rev a1)

4. Blacklist the nouveau driver: Blacklisting will prevent Proxmox from using the GPU, which would interfere with the passthrough. To do this, open /etc/modprobe.d/blacklist.conf with a text editor, add a line blacklist nouveau, save and close.

5. Bind the GPU to the VFIO driver: This makes the GPU available for passthrough. Open /etc/modprobe.d/vfio.conf with a text editor and add the line options vfio-pci ids=10de:1b81,10de:10f0 (replace with your GPU and audio device addresses). Then, open /etc/modules and add the line vfio at the end. Save and close.

6. Rebuild the initramfs: You need to rebuild the initramfs to apply these changes. Run update-initramfs -u in the terminal. Then, reboot your Proxmox host.

Before reboot with changes

# lspci -nnk
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA102 [GeForce RTX 3090] [10de:2204] (rev a1)
        Subsystem: Palit Microsystems Inc. GA102 [GeForce RTX 3090] [1569:2204]
        Kernel driver in use: nouveau
        Kernel modules: nvidiafb, nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation GA102 High Definition Audio Controller [10de:1aef] (rev a1)
        Subsystem: Palit Microsystems Inc. Device [1569:2204]
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel

After reboot with changes - [ ] still the same

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA102 [GeForce RTX 3090] [10de:2204] (rev a1)
        Subsystem: Palit Microsystems Inc. GA102 [GeForce RTX 3090] [1569:2204]
        Kernel modules: nvidiafb, nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation GA102 High Definition Audio Controller [10de:1aef] (rev a1)
        Subsystem: Palit Microsystems Inc. Device [1569:2204]
        Kernel modules: snd_hda_intel

On your Ubuntu VM:

7. Create a new VM: Log in to the Proxmox web interface and create a new VM. Install Ubuntu on it but don't add a graphics card yet. Choose 'SCSI' as the Hard Disk and 'VirtIO SCSI' as the Controller. This will give you the best performance.

8. Configure the VM for GPU passthrough: Once the VM is set up, shut it down. Go to the VM's Hardware tab and delete the current graphics card (likely Vga). Then, add a PCI Device. Choose the PCI addresses of your GPU and its audio device from the dropdown. Make sure 'All Functions' is checked for both.

9. Add a ROM file (Optional): Some GPUs need a ROM file for passthrough to work. If this is the case for your RTX 3090, add it in the 'ROM file' field in the PCI Device settings. You can usually find these files on techpowerup.com.

10. Boot the VM: Power on the VM. If everything is set up correctly, the GPU should be

---

22/7 changes