Skip to content

Instantly share code, notes, and snippets.

@ryan4yin
Last active August 29, 2026 05:32
Show Gist options
  • Select an option

  • Save ryan4yin/48617bbddacc7067f10799770b7cc33f to your computer and use it in GitHub Desktop.

Select an option

Save ryan4yin/48617bbddacc7067f10799770b7cc33f to your computer and use it in GitHub Desktop.
Best llama.cpp config for Qwen3.8-Flash-Next (RTX 4090 24GB)
version: '3.8'
# Qwen3.8-Flash-Next llama-server config
# Start: docker compose -f docker-compose.qwen3.8-flash-next.yml up -d
# Stop: docker compose -f docker-compose.qwen3.8-flash-next.yml down
#
# Model download (ModelScope CLI, ~78GB, 3 shards + mmproj):
# modelscope download --model unsloth/Qwen3.8-Flash-Next-GGUF \
# --include "*UD-IQ3_XXS*" --include "mmproj-F16.gguf" \
# --local-dir models/unsloth/Qwen3.8-Flash-Next-GGUF
#
# Notes:
# - 125B MoE, UD-IQ3_XXS ~78GB: fits 96GB RAM + 24GB VRAM with mmap
# - mainline llama.cpp has no kvarn6, KV uses q8_0
# - mmproj-F16 enabled by default (vision); costs ~900MB VRAM (max ctx 170K -> 150K)
# - reasoning_effort defaults to xhigh; lowered to medium to save tokens
# - MTP: model ships output_hc_* MTP head, unsupported by llama.cpp so far (draft-mtp OOM)
services:
llama-flash-next:
image: ghcr.io/ggml-org/llama.cpp:full-cuda13 # mainline, supports Qwen3.8-Flash-Next (Qwen4 arch)
container_name: llama-flash-next
ports:
- "63082:8001"
volumes:
- ./models:/models
command:
- --server # full-cuda13 entrypoint is a tool dispatcher, must specify explicitly
- --model
- /models/unsloth/Qwen3.8-Flash-Next-GGUF/UD-IQ3_XXS/Qwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf
- --mmproj
- /models/unsloth/Qwen3.8-Flash-Next-GGUF/mmproj-F16.gguf
- --alias
- Qwen3.8-Flash-Next
- --ctx-size
- "120000" # final: smaller KV alloc -> more weight offload -> decode +8% (22.2 vs 20.4 @90K); 110K input 19.5; sessions >120K get truncated
# offload strategy (ref: HF discussion #3):
# same IQ3_XXS as williamliao: --fit on beats manual -ncmoe (manual drops to 8-11 t/s)
# ZoneOverreach (3090 24G): -ncmoe 41 --fit off --no-repack -b 2048 -ub 128 -t 8 -tb 24
- --fit
- "on"
- --fit-target
- "256" # final: +0.3 t/s vs 512, no OOM; do not go below 256 (large ubatch needs headroom)
- -t
- "20" # final: leave 4 of 24 cores for the OS (all 24 is slower: 568/27.6 vs 573/30.3) - CPU-specific
- -tb
- "24" # batch threads: prefill uses all 24 cores (decode still uses -t 20) - CPU-specific
# CPU: Intel Core Ultra 7 270K Plus (24C/24T, no HT); retune -t/-tb on other CPUs
- -b
- "6144" # final sweet spot: prefill near 8192 (982/870), decode near 4096 (29.5/21.9)
- -ub
- "6144" # keep -b == -ub, otherwise 8192 splits into a slow 6144+2048 tail batch
- --reasoning-budget
- "4000"
- --chat-template-kwargs
- '{"reasoning_effort": "medium"}' # flash-next defaults to xhigh, lowered to medium
- --reasoning-preserve
- --reasoning-budget-message
- "... reasoning budget exceeded, need to answer.\n"
# Thinking mode (unsloth recommended): temp=1.0, top_p=0.95, top_k=20, min_p=0.0, presence=0.0
- --temp
- "1.0"
- --top-p
- "0.95"
- --top-k
- "20"
- --min-p
- "0.00"
- --presence-penalty
- "0.0"
- --flash-attn
- "on"
- --cache-type-k
- q8_0 # final: q4 KV frees VRAM but dequant overhead cancels the gain (28.9 vs 30.1 @32K), keep high precision
- --cache-type-v
- q8_0
- --image-max-tokens
- "4000"
- --image-min-tokens
- "1024" # Qwen-VL requires at least 1024 image tokens
- --cont-batching
- --host
- 0.0.0.0
- --port
- "8001"
- --metrics
- --log-verbosity
- "3" # keep print_timing (prefill/decode split)
- -np
- "1"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
shm_size: '1gb'
restart: unless-stopped

Qwen3.8-Flash-Next — llama.cpp on RTX 4090 24GB

Platform: Intel Core Ultra 7 270K Plus (24C/24T), 96GB DDR5, RTX 4090 24GB. Model: unsloth/Qwen3.8-Flash-Next-GGUF, UD-IQ3_XXS (~78GB) + mmproj-F16 (~900MB, vision enabled). llama.cpp mainline (ghcr.io/ggml-org/llama.cpp:full-cuda13). Final config as of 2026-08-29.

Docker Compose (ready to run): see attached docker-compose.qwen3.8-flash-next.yml.

Final flags

llama-server \
  --model Qwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf \
  --mmproj mmproj-F16.gguf \
  --ctx-size 120000 \
  --fit on --fit-target 256 \
  -t 20 -tb 24 \
  -b 6144 -ub 6144 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 \
  --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 \
  --reasoning-budget 4000 \
  --reasoning-budget-message "... reasoning budget exceeded, need to answer.\n" \
  --chat-template-kwargs '{"reasoning_effort": "medium"}' \
  --reasoning-preserve \
  --image-max-tokens 4000 --image-min-tokens 1024 \
  --cont-batching --metrics -np 1

Why each non-default flag

Flag Reason (measured on this box)
--ctx-size 120000 smaller KV alloc → --fit offloads more weights → decode +8% (22.2 vs 20.4 t/s @ 90K). 110K input still 19.5 t/s. Cost: sessions >120K get truncated (client-side compaction handles it)
--fit on --fit-target 256 auto offload beats manual -ncmoe (84K input: 19.5 vs 16.2 t/s); 512→256 frees room for more weights (+0.3 t/s, no OOM at -ub 6144). Don't go below 256
-t 20 -tb 24 prefill: 24T=836, 20T=848 t/s (leave 4 cores for system); decode-batch uses full 24
-b/-ub 6144 prefill sweet spot: 2048=762, 4096=850, 6144=864, 8192=828 t/s. Keep -b == -ub or 8192 becomes a 6144+2048 tail batch and slows down
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 unsloth-recommended sampling for thinking mode
--flash-attn on --cache-type-k/v q8_0 saves ~1.5GB VRAM vs F16 KV. q4_0 KV was tried: dequant overhead cancels the gain (28.9 vs 30.1 t/s @32K), so q8_0 stays
--reasoning-budget 4000 + reasoning_effort: medium default xhigh wastes tokens; high effort degrades quality on this model (self-doubt loops)
--reasoning-preserve pass reasoning history back to model across turns
--mmproj mmproj-F16.gguf vision enabled by default; costs ~20-30K ctx vs text-only
-np 1 single slot → max per-slot context

Performance (fresh corpus per point, /tokenize-calibrated, single slot, no KV hit)

Input Prefill t/s Decode t/s Note
6K ~1360 ~30 warm
90K ~905 ~22 typical agent/coding session
110K ~863 ~19.5 92% of ctx, near-full

Decode tracks absolute KV depth: 6K→30, 90K→22, 110K→~19.5 t/s. ctx allocation itself doesn't change speed — the gain from 120K comes purely from freed VRAM going to weight offload.

Gotchas

  • Request over ctx → HTTP 400 with n_prompt_tokens/n_ctx in the response. At ctx 120K plan client-side compaction before the limit.
  • --ctx-size 131072 OOMs with mmproj (auto-reduced to 130560); max stable with mmproj is ~150K, text-only ~170K.
  • Mainline llama.cpp has no KVarN KV quant; q8_0 + flash-attn is the practical best for this model.
  • Model ships an MTP head but llama.cpp doesn't support it yet (--spec-type draft-mtp OOMs). Watch for upstream updates — could be a big decode win.
  • Docker image note: full-cuda13 entrypoint is a tool dispatcher, pass --server explicitly.
  • 96GB RAM is enough to hold the ~78GB weights fully resident with swap unused; with less RAM expect cold-start penalties on large prefills.
@slarrain

Copy link
Copy Markdown

How would you get this work with a 4090 but 64GB of RAM?
Also, is it so much better over Qwen3.8-27b ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment