Skip to content

Instantly share code, notes, and snippets.

@xmesaj2

xmesaj2/Proxmox_host_TheRock_gfx906_mi50.md

Last active September 11, 2026 09:43
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save xmesaj2/4f8835637bef90d4453821cf8553b8cb to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save xmesaj2/4f8835637bef90d4453821cf8553b8cb to your computer and use it in GitHub Desktop.
Download ZIP
Proxmox TheRock nightly ROCm instructions for LXC passthrough and disk mount for single Mi50 (32GB, but 16GB will work too) to run with unsloth-studio
Raw
install_in_lxc.md

llama.cpp & unsloth-studio

Install unsloth-studio

HSA_OVERRIDE_GFX_VERSION=9.0.6 curl -fsSL https://unsloth.ai/install.sh | sh

(Optional) Install custom optimized llama.cpp

# git clone https://github.com/sixvolts/llamacpp-gfx906-furnace.git
# cd llamacpp-gfx906-furnace
# cmake -B build -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx906 \
#      -DGGML_HIP_GRAPHS=ON -DCMAKE_BUILD_TYPE=Release

git clone https://github.com/mxxm-t/mx-llama.cpp.git
cd mx-llama.cpp
cmake -B build \
  -DGGML_HIP=ON \
  -DGGML_HIP_GRAPHS=ON \
  -DGGML_HIP_RCCL=ON \
  -DLLAMA_OPENSSL=ON \
  -DAMDGPU_TARGETS=gfx906 \
  -DCMAKE_BUILD_TYPE=Release \
  -DHIP_COMPILER=clang \
  -DCMAKE_CXX_FLAGS="-O3 -Wno-unused-command-line-argument"
cmake --build build --config Release -j4

cp -r ~/mx-llama.cpp/build/bin/ ~/.unsloth/llama.cpp/build/bin/
#cp -r ~/llamacpp-gfx906-furnace/build/bin/ ~/.unsloth/llama.cpp/build/bin/
Raw
Proxmox_host_lxc.conf
arch: amd64
cores: 6
dev0: /dev/dri/renderD128,gid=993
dev1: /dev/dri/card0,gid=44
dev2: /dev/kfd,gid=993
features: fuse=1,keyctl=1,mknod=1,nesting=1
hostname: aiserver
memory: 8192
mp0: /srv/storage,mp=/mnt/storage,mountoptions=noatime
nameserver: 1.1.1.2
net0: name=eth0,bridge=vmbr0,hwaddr=bc:24:11:dc:37:ff,ip=dhcp,type=veth
onboot: 1
ostype: ubuntu
rootfs: local-lvm:vm-116-disk-0,mountoptions=noatime,size=158G
swap: 6512
tags: ai
timezone: Europe/Warsaw
unprivileged: 1
lxc.cgroup2.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file
Raw
Proxmox_host_TheRock_gfx906_mi50.md

On your Proxmox host run to see exactly where your driver is being loaded from:

modinfo amdgpu | grep filename

If it says: /lib/modules/7.x.x-x-pve/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko -> This is the native Proxmox driver. Safe to remove the DKMS packages if installed, if not the skip to point #4.

If it says /updates/dkms/amdgpu.ko, then the DKMS actually loaded, should be rare. Do:

  1. Remove the Ubuntu DKMS and ROCm apt packages
apt purge -y amdgpu-dkms rocm* amdgpu-install
apt autoremove -y --purge
  1. Clean up the leftover directories and apt sources
rm -rf /opt/rocm* /opt/amdgpu* /etc/apt/sources.list.d/amdgpu.list /etc/apt/sources.list.d/rocm.list
  1. Rebuild the boot image just to ensure the broken DKMS module isn't accidentally queued for the next boot
update-initramfs -u
  1. Verify
dmesg | grep amdgpu 
ls -l /dev/kfd
  1. Download the specific gfx906 multiarch nightly build eg. therock-dist-linux-multiarch-10.1.0a20260822.tar.gz (this version supports also iGPU integrated with consumer grade Ryzen eg. 5600G)
cd /tmp
wget https://rocm.nightlies.amd.com/tarball-multi-arch/therock-dist-linux-multiarch-10.1.0a20260822.tar.gz
  1. Extract to /opt
mkdir -p /opt/therock
tar -xzf therock-dist-linux-multiarch-10.1.0a20260822.tar.gz -C /opt/therock
  1. Add to system path permanently
echo 'export PATH=/opt/therock/bin:$PATH' > /etc/profile.d/therock.sh
echo 'export LD_LIBRARY_PATH=/opt/therock/lib:/opt/therock/llvm/lib:$LD_LIBRARY_PATH' >> /etc/profile.d/therock.sh
source /etc/profile.d/therock.sh
  1. Verify
rocm-smi --showproductname

(Optional) Lazy way to allow access to host volume mount models passed to LXC.

chmod 755 /srv/storage/ai/models/*
chown -R 100000:100000 /srv/storage/ai/models/*

eg downloaded by HuggingFaceModelDownloader bash <(curl -sSL https://g.bodaay.io/hfd) install

Raw
unslothstudio_in_lxc.sh
#!/bin/bash
export HF_HOME=/mnt/storage/ai/models/
export HF_HUB_CACHE=/mnt/storage/ai/models/
export UNSLOTH_CACHE_DIR=/mnt/storage/ai/models/
export HSA_OVERRIDE_GFX_VERSION=9.0.6
export HIP_VISIBLE_DEVICES=0
export GGML_ENABLE_CUSTOM_AR=1 # custom multi-GPU AllReduce
export HSA_FORCE_FINE_GRAIN_PCIE=1 # peer-write AllReduce fast path (AMD over PCIe, validated gfx906)
export GPU_MAX_HW_QUEUES=8 # MoE throughput on -tps
export LLAMA_ENABLE_MTP_OPT=1 # MTP optimizations (with --spec-type draft-mtp
export AMD_DIRECT_DISPATCH=1
export LLAMACPP_ROCM_ARCH=gfx906
export HSA_ENABLE_INTERRUPT=0
export HSA_ENABLE_SDMA=1
export HIP_FORCE_DEV_KERNARG=1
export ROCBLS_USE_HIPBLASLT=0
export HIP_VISIBLE_DEVICES=0,1
unsloth studio -H 192.168.1.100 -p 8888
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment