AMD Ryzen AI 9 HX 370 GPT OSS 20b benchmark

AMD Ryzen AI 9 HX 370 GPT OSS 20b benchmark.md

GPT-OSS on AMD Radeon 890M (llama.cpp)

Local inference for OpenAI gpt-oss-20b via llama.cpp and Vulkan.

Why gpt-oss-20b

Model	Fits this GPU?	Notes
gpt-oss-20b	✅ Yes	MXFP4 GGUF ~11.3 GiB; full GPU offload on 890M
gpt-oss-120b	❌ No	Needs ~80 GB VRAM (or heavy CPU offload + 64 GB+ RAM)

Weights: ggml-org/gpt-oss-20b-GGUF → gpt-oss-20b-mxfp4.gguf (~12 GB on disk)

System

• GPU: AMD Radeon 890M (gfx1150, ~13.5 GB UMA)
• Backend: Vulkan (ROCm untested for gpt-oss; Vulkan is stable here)
• Build: llama.cpp 6e14286, -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release

Quick start

# One-shot prompt
./run-gpt-oss.sh "What is 17+25? One short sentence."

# OpenAI-compatible server → http://127.0.0.1:8080
./run-gpt-oss-server.sh

Manual CLI

cd llama.cpp
./build/bin/llama-cli \
  -m ../models/gpt-oss-20b-mxfp4.gguf \
  -ngl 99 -fa 1 -c 8192 -b 512 -ub 512 \
  --jinja --single-turn --simple-io \
  -p "Your prompt here" -n 256

If you run out of VRAM

Reduce context or offload MoE layers to CPU:

CTX=4096 ./run-gpt-oss.sh "prompt"

# or (llama-server / llama-cli)
--n-cpu-moe 8   # try 8, 12, 16 until it loads

Performance (Radeon 890M, Vulkan)

Measured 2026-06-14 on this machine.

Test	Throughput
Token generation (tg128, llama-bench)	27.1 t/s
Interactive generation (8192 ctx)	23–26 t/s
Prompt processing (short prompt)	128–162 t/s

Settings: -ngl 99 -fa 1 -c 8192 -b 512 -ub 512, model fully on GPU.

Community reference (same GPU, more RAM): ~27 t/s generation on 890M Vulkan (llama.cpp #15396).

Rebuild

cd llama.cpp
git pull
cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=ON
cmake --build build -j"$(nproc)"

Download model

python3 -m venv .venv && . .venv/bin/activate
pip install huggingface_hub
huggingface-cli download ggml-org/gpt-oss-20b-GGUF gpt-oss-20b-mxfp4.gguf --local-dir models

Benchmark

cd llama.cpp
./build/bin/llama-bench \
  -m ../models/gpt-oss-20b-mxfp4.gguf \
  -ngl 99 -t 1 -fa 1 -p 0 -n 128 -b 2048 -ub 512

OpenCode CLI

OpenCode is configured to use the local llama-server via an OpenAI-compatible adapter.

Install (once)

curl -fsSL https://opencode.ai/install | bash
cd /home/devops/projects/tts/.opencode && npm install

Config files:

• opencode.json (project)
• ~/.config/opencode/opencode.json (user)

Run

Terminal 1 — model server (16384 ctx, 1 slot for agent prompts):

CTX=16384 NP=1 ./run-gpt-oss-server.sh

Terminal 2 — OpenCode (starts servers automatically if needed):

./run-opencode.sh "Explain what run-gpt-oss-server.sh does."

Interactive TUI:

./run-opencode.sh

Inside OpenCode: /models → select llama.cpp/gpt-oss-20b-mxfp4.gguf.

Notes

• OpenCode agent system prompts are ~6k tokens; use ctx ≥ 16384 or you'll get Context size has been exceeded.
• Requires @ai-sdk/openai-compatible in ~/.opencode and .opencode/node_modules.
• Model ID must match llama-server: gpt-oss-20b-mxfp4.gguf.