local inference

index.md

I found the crash cause and installed a clean reference ROCm runtime alongside LM Studio.

The failure is not corrupted model weights. Iconium’s 128 GiB is currently split into:

• 96 GiB fixed GPU VRAM
• Only 30 GiB visible to Linux
• About 15.6 GiB available through GTT

During a 60,554-token request, LM Studio created a 27.0 GiB prompt-cache snapshot. Linux exhausted its 30 GiB partition, reported repeated SVM mapping failed errors, and the OOM killer terminated LM Studio’s worker.

Changes completed:

• Installed Podman, Distrobox, crun, and Tuned.

• Created the clean llama-rocm-7.2.4 toolbox.

• Confirmed ROCm access to the Radeon 8060S.

• Enabled the accelerator-performance profile.

• Staged the recommended kernel parameters:

  • amd_iommu=off
  • amdgpu.gttsize=126976
  • ttm.pages_limit=32505856

• Backed up the former GRUB configuration as /etc/default/grub.pre-strix-halo-20260711.

• Left LM Studio, LM Link, models, and the loaded Gemma model intact.

• Benchmarked Qwen 3.6 27B Q6 through the clean ROCm toolbox:

  • Prompt processing: 353.82 tokens/sec
  • Generation: 9.33 tokens/sec

• Restored Gemma 4 31B in LM Studio afterward.

The toolbox project currently recommends ROCm 7.2.4, Flash Attention, full GPU offload, and no mmap on Strix Halo. It also identifies Fedora 42/43 with kernel 6.18.9 and January 2026 firmware as its stable reference configuration. AMD Strix Halo Toolboxes (https://github.com/kyuz0/amd-strix-halo-toolboxes)

Required BIOS change

At Iconium’s physical console:

1. Reboot and press Del for BIOS.

2. If AMD CBS is hidden, press Ctrl+F1.

3. Navigate to: Advanced → AMD CBS → NBIO Common Options → GFX Configuration

4. Prefer:

   • iGPU Configuration: Auto
   • UMA Version: Auto
   • GPU Host Translation Cache: Auto
   • Or explicitly set UMA Frame Buffer Size to 512M

5. Save with F4.

Beelink documents this same menu path and configuration process. Beelink GTR9 Pro UMA instructions (https://doc.bee-link.com.cn/books/gtr9-pro-biso/page/gtr9-pro-CDf/revisions/400)

After the BIOS change and reboot, Linux should see roughly 127 GiB rather than 30 GiB, while ROCm dynamically accesses as much as 124 GiB through GTT. The GRUB configuration for this is already staged.

OS recommendation

A clean OS installation is strongly recommended, although it is not required merely to make ROCm work.

Iconium runs Ubuntu 25.10, which reached end of life on July 9, 2026—two days ago. Ubuntu release lifecycle (https://wiki.ubuntu.com/Releases) Because the current installation also mixes a mainline 6.19 kernel, old September 2025 firmware, host ROCm packages from different releases, LM Studio workarounds, and older experimental vLLM/Docker environments, I would rebuild it with Fedora 43. That matches the tested Strix Halo stack most closely.

There is also a firmware concern:

• Current BIOS: GTRP107
• Current microcode: 0x0b70001e
• Kernel reports: RDSEED32 is broken
• The onboard Intel Ethernet controllers are absent; Iconium is currently connected through a USB Ethernet adapter.

Beelink has released GTRP112 for the original GTR9 Pro hardware revision, but it should be obtained directly from Beelink using the machine’s serial number—not from an unofficial mirror. It includes newer AGESA and microcode, though some users report a suspend/ACPI regression. For this headless server, suspend can simply remain disabled. Beelink firmware discussion (https://bbs.bee-link.com/d/10832-gtr9-pro-rdseed32-is-broken-message-on-startup/15)

My recommended order is:

1. Change UMA to Auto/512 MiB.
2. Boot once and verify the staged GTT configuration.
3. Obtain and install the correct GTRP112 firmware from Beelink.
4. Back up models/configuration.
5. Clean-install Fedora 43.
6. Recreate the ROCm 7.2.4 and RADV toolboxes.
7. Benchmark Gemma and Qwen across ROCm and RADV, then retain LM Studio only where LM Link is needed.