llm-bench sweep · aggregate decode tok/s at concurrency 1 · Apple M5 Max, 128 GB unified memory, 40 GPU cores · 30s per cell, greedy, ignore_eos, max_tokens 8192
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.
engine: MTPLX 2.11.3 (brew install youssofal/mtplx/mtplx; native process, no container)
Spec-Bench, 6-category mean · NVFP4, TP=2 both cards · 2× RTX PRO 6000 · median of 3 samples/point
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.
Spec-Bench, 6-category mean · NVFP4, TP=1 single card · RTX PRO 6000 Max-Q · median of 3 samples/point
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.
A byte-level graft of two public NVFP4 checkpoints that produces a combination neither one ships: an FP8 PLE table (small host offload) together with NVFP4 MTP draft experts (cheap speculation). It costs 5.4 GB of disk on top of the parents, needs no retraining and no requantization, and serves on both vLLM and SGLang.
Built and validated 2026-08-27 on 2x RTX PRO 6000 Blackwell Max-Q (sm_120, 96 GB each, 220 GB host RAM).
Spec-Bench, 6-category mean · 2× RTX PRO 6000 Max-Q · median of 3 samples/point · text 0731 + Vision-Exp checkpoints
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.
NVFP4 (vLLM, SGLang) · GGUF UD-Q4 / Q6 / Q8_K_XL (llama.cpp) · avg of the 6 Spec-Bench categories · greedy · conc=1 · 3 samples/point · 1× RTX PRO 6000 Max-Q
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.
Dense Qwen3.6-27B, NVFP4 on vLLM + SGLang · averaged over the 6 Spec-Bench categories · greedy · conc=1 · 3 samples/point · 1× RTX PRO 6000 Max-Q
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.
poolside Laguna S 2.1 (MoE, NVFP4) · greedy · conc=1 · 3 restart-samples/point · 2× RTX PRO 6000 Max-Q (TP=2)
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.
DeepSeek-V4-Flash-0731, 2x RTX PRO 6000 TP=2 · DSpark fixed-K5, conc=1 · Q8 GGUF knot for scale · Spec-Bench 6-cat avg · 3 samples/point
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.
Dense Qwen3.6-27B, UD-Q8_K_XL GGUF on llama.cpp · averaged over the 6 Spec-Bench categories · greedy · conc=1 · 3 samples/point · 1x RTX PRO 6000 Max-Q
Full serve and benchmark commands for every series on this chart, extracted from the canonical benchmark records (the source of truth). One block per engine/variant; the swept draft-depth parameter is shown as N with its range.