Proxmox-share-GPU-multiple-LXC.md

Sharing a GPU Between Multiple LXC Containers

1. Identify Required Groups Inside the Container

GPU access in LXC relies on Linux group permissions.

Common groups:

• video → legacy DRM access (e.g. /dev/dri/card*)
• render → render node access

Check group IDs inside the container:

cat /etc/group | egrep 'video|render'

Typical values:

• video: GID 44
• render: GID 104

⚠️ Important
GIDs must match inside the container, not the host.

---

2. Identify GPU Devices on the Host

DRM devices

ls -l /dev/dri

Typical output:

• /dev/dri/cardX → display node
• /dev/dri/renderD128 → render node (preferred)

Multi-GPU systems

ls -l /sys/class/drm/renderD*/device/driver
ls -l /dev/dri/by-path

PCI GPU detection

lspci | grep -E 'VGA|3D'

---

3. Configure DRM Device Mapping (Recommended)

Edit the container config /etc/pve/lxc/<VMID>.conf, add:

dev0: /dev/dri/card1,gid=44,uid=0
dev1: /dev/dri/renderD128,gid=104,uid=0

Explanation:

• card* → optional, needed for display/X11
• renderD* → required for compute & headless workloads
• gid → must match container group IDs
• uid=0 → root inside container

Restart the container and verify:

ls -l /dev/dri groups

---

4. Passing NVIDIA Devices to LXC

Method 1: Direct Device Mapping (Preferred)

This is the simplest and safest approach for modern Proxmox (cgroup v2).

Add to /etc/pve/lxc/<VMID>.conf:

dev0: /dev/dri/card1,gid=44,uid=0
dev1: /dev/dri/renderD128,gid=104,uid=0
dev2: /dev/nvidia0
dev3: /dev/nvidiactl
dev4: /dev/nvidia-modeset
dev5: /dev/nvidia-uvm
dev6: /dev/nvidia-uvm-tools
dev7: /dev/nvidia-caps/nvidia-cap1
dev8: /dev/nvidia-caps/nvidia-cap2

✔ Works with multiple containers
✔ No PCI lock
✔ No major/minor maintenance

---

Method 2: Manual cgroup + Bind Mount (Legacy / Advanced)

Only needed if devX: entries are unavailable or insufficient.

Check NVIDIA device numbers

ls -al /dev/nvidia*

Allow devices via cgroup v2

lxc.cgroup2.devices.allow: c 195:* rwm   # NVIDIA core
lxc.cgroup2.devices.allow: c 226:* rwm   # DRM
lxc.cgroup2.devices.allow: c 510:* rwm   # nvidia-uvm
lxc.cgroup2.devices.allow: c 235:* rwm   # nvidia-caps

⚠️ Major numbers may change between driver versions.

Bind-mount devices

lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-caps dev/nvidia-caps none bind,optional,create=dir

---

5. Installing NVIDIA Driver Inside the Container

💡 The kernel module must remain on the host
Containers only install user-space libraries

---

Step 0: Remove Old Drivers (if any)

See: 3. Remove Existing NVIDIA/CUDA Drivers

---

Step 1: Push Installer from Host

pct push <VMID> NVIDIA-Linux-x86_64-580.95.05.run /tmp/nvidia.run

---

Step 2: Install (No Kernel Modules)

sudo chmod +x /tmp/nvidia.run
sudo /tmp/nvidia.run --no-kernel-module

---

Step 3: Installer Prompts

• Missing X libraries → OK
• 32-bit compatibility → NO
• nvidia-xconfig → NO (headless)

---

Step 4: Reboot & Verify

sudo reboot
sudo nvidia-smi

---

References

• Proxmox-passing-device-in-LXC
• Proxmox Forum: GPU Mapping Error
• Yomi’s Blog on NVIDIA GPU with LXC
• Reddit: Cgroup2 & Device Numbers