Proxmox TheRock nightly ROCm instructions for LXC passthrough and disk mount for single Mi50 (32GB, but 16GB will work too) to run with unsloth-studio

install_in_lxc.md

llama.cpp & unsloth-studio

Install unsloth-studio

HSA_OVERRIDE_GFX_VERSION=9.0.6 curl -fsSL https://unsloth.ai/install.sh | sh

(Optional) Install custom optimized llama.cpp

git clone https://github.com/sixvolts/llamacpp-gfx906-furnace.git
cd llamacpp-gfx906-furnace
cmake -B build -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx906 \
      -DGGML_HIP_GRAPHS=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j4
cp -r ~/llamacpp-gfx906-furnace/build/bin/ ~/.unsloth/llama.cpp/build/bin/

Proxmox_host_lxc.conf

arch: amd64
cores: 6
dev0: /dev/dri/renderD128,gid=993
dev1: /dev/dri/card0,gid=44
dev2: /dev/kfd,gid=993
features: fuse=1,keyctl=1,mknod=1,nesting=1
hostname: aiserver
memory: 8192
mp0: /srv/storage,mp=/mnt/storage,mountoptions=noatime
nameserver: 1.1.1.2
net0: name=eth0,bridge=vmbr0,hwaddr=bc:24:11:dc:37:ff,ip=dhcp,type=veth
onboot: 1
ostype: ubuntu
rootfs: local-lvm:vm-116-disk-0,mountoptions=noatime,size=158G
swap: 6512
tags: ai
timezone: Europe/Warsaw
unprivileged: 1
lxc.cgroup2.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file

Proxmox_host_TheRock_gfx906_mi50.md

On your Proxmox host run to see exactly where your driver is being loaded from:

modinfo amdgpu | grep filename

If it says: /lib/modules/7.x.x-x-pve/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko -> This is the native Proxmox driver. Safe to remove the DKMS packages if installed, if not the skip to point #4.

If it says /updates/dkms/amdgpu.ko, then the DKMS actually loaded, should be rare. Do:

1. Remove the Ubuntu DKMS and ROCm apt packages

apt purge -y amdgpu-dkms rocm* amdgpu-install
apt autoremove -y --purge

2. Clean up the leftover directories and apt sources

rm -rf /opt/rocm* /opt/amdgpu* /etc/apt/sources.list.d/amdgpu.list /etc/apt/sources.list.d/rocm.list

3. Rebuild the boot image just to ensure the broken DKMS module isn't accidentally queued for the next boot

update-initramfs -u

4. Verify

dmesg | grep amdgpu 
ls -l /dev/kfd

5. Download the specific gfx906 nightly build

cd /tmp
wget https://therock-nightly-tarball.s3.amazonaws.com/therock-dist-linux-gfx906-7.14.0a20260612.tar.gz

6. Extract to /opt

mkdir -p /opt/therock
tar -xzf therock-dist-linux-gfx906-7.14.0a20260612.tar.gz -C /opt/therock

7. Add to system path permanently

echo 'export PATH=/opt/therock/bin:$PATH' > /etc/profile.d/therock.sh
echo 'export LD_LIBRARY_PATH=/opt/therock/lib:/opt/therock/llvm/lib:$LD_LIBRARY_PATH' >> /etc/profile.d/therock.sh
source /etc/profile.d/therock.sh

8. Verify

rocm-smi --showproductname

(Optional) Lazy way to allow access to host volume mount models passed to LXC.

chmod 755 /srv/storage/ai/models/*
chown -R 100000:100000 /srv/storage/ai/models/*

eg downloaded by HuggingFaceModelDownloader bash <(curl -sSL https://g.bodaay.io/hfd) install

unslothstudio_in_lxc.sh

#!/bin/bash
export HF_HOME=/mnt/storage/ai/models/
export HF_HUB_CACHE=/mnt/storage/ai/models/
export UNSLOTH_CACHE_DIR=/mnt/storage/ai/models/
export HSA_OVERRIDE_GFX_VERSION=9.0.6
unsloth studio -H 192.168.1.100 -p 8888