dobriak/qwen36-mtp-benchmark.md

Created June 4, 2026 00:08

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/dobriak/d613f2490276685e9e285d25e5157981.js"></script>
Save dobriak/d613f2490276685e9e285d25e5157981 to your computer and use it in GitHub Desktop.

Download ZIP

Benchmarking Qwen3.6-MTP

Raw

qwen36-mtp-benchmark.md

Benchmarking Qwen3.6-27B MTP Performance

RTX 5090, Ryzen 9 9950X3D, 128GB DDR5
Debian 13 6.12.85-1, CUDA 13.2.78, llama.cpp b9200

benchy Tests

1. Control - No MTP bits in gguf

mmproj loaded

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
unsloth/Qwen3.6-27B	pp2048	2642.44 ± 58.11		782.48 ± 17.10	775.67 ± 17.10	782.48 ± 17.10
unsloth/Qwen3.6-27B	tg32	56.75 ± 2.09	58.66 ± 2.16

no mmproj

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
unsloth/Qwen3.6-27B	pp2048	2742.91 ± 206.98		758.36 ± 55.83	751.22 ± 55.83	758.36 ± 55.83
unsloth/Qwen3.6-27B	tg32	55.48 ± 3.51	57.34 ± 3.63

2. MTP bits present but not enabled

mmproj loaded

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
unsloth/Qwen3.6-27B-Q4	pp2048	3007.29 ± 192.09		618.65 ± 62.49	611.74 ± 62.49	618.65 ± 62.49
unsloth/Qwen3.6-27B-Q4	tg32	83.80 ± 7.19	86.40 ± 7.38
unsloth/Qwen3.6-27B-Q5	pp2048	2742.71 ± 161.67		695.08 ± 49.64	686.56 ± 49.64	695.08 ± 49.64
unsloth/Qwen3.6-27B-Q5	tg32	85.52 ± 5.58	88.14 ± 5.83
unsloth/Qwen3.6-27B-Q6	pp2048	2464.15 ± 133.45		732.50 ± 55.54	724.57 ± 55.54	732.50 ± 55.54
unsloth/Qwen3.6-27B-Q6	tg32	66.02 ± 3.63	68.02 ± 3.74

no mmproj

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
unsloth/Qwen3.6-27B-Q4	pp2048	2945.08 ± 292.27		638.99 ± 59.82	630.82 ± 59.82	638.99 ± 59.82
unsloth/Qwen3.6-27B-Q4	tg32	84.88 ± 7.24	87.49 ± 7.43
unsloth/Qwen3.6-27B-Q5	pp2048	2701.85 ± 176.83		687.89 ± 44.52	681.62 ± 44.52	687.89 ± 44.52
unsloth/Qwen3.6-27B-Q5	tg32	85.14 ± 11.00	87.79 ± 11.36
unsloth/Qwen3.6-27B-Q6	pp2048	2548.04 ± 246.37		752.56 ± 50.19	746.01 ± 50.19	752.56 ± 50.19
unsloth/Qwen3.6-27B-Q6	tg32	70.57 ± 5.47	72.58 ± 5.52

3. MTP bits enabled in llama.cpp config

mmproj loaded

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
unsloth/Qwen3.6-27B-Q4-MTP	pp2048	2740.98 ± 91.47		697.70 ± 15.55	689.78 ± 15.55	697.70 ± 15.55
unsloth/Qwen3.6-27B-Q4-MTP	tg32	186.58 ± 39.52	193.47 ± 41.12
unsloth/Qwen3.6-27B-Q5-MTP	pp2048	340.37 ± 9.89		5545.89 ± 292.50	5538.65 ± 292.50	5545.89 ± 292.50
unsloth/Qwen3.6-27B-Q5-MTP	tg32	80.81 ± 10.34	83.35 ± 10.67
unsloth/Qwen3.6-27B-Q6-MTP	pp2048	206.50 ± 1.71		9086.85 ± 411.00	9079.60 ± 411.00	9086.85 ± 411.00
unsloth/Qwen3.6-27B-Q6-MTP	tg32	36.32 ± 1.19	37.42 ± 1.23

no mmproj

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
unsloth/Qwen3.6-27B-Q4-MTP	pp2048	2623.13 ± 109.43		709.12 ± 29.69	701.49 ± 29.69	709.12 ± 29.69
unsloth/Qwen3.6-27B-Q4-MTP	tg32	238.85 ± 30.30	247.89 ± 31.85
unsloth/Qwen3.6-27B-Q5-MTP	pp2048	357.79 ± 12.51		5185.61 ± 40.74	5176.51 ± 40.74	5185.61 ± 40.74
unsloth/Qwen3.6-27B-Q5-MTP	tg32	93.51 ± 19.10	96.41 ± 19.85
unsloth/Qwen3.6-27B-Q6-MTP	pp2048	211.54 ± 8.20		8909.21 ± 525.73	8902.88 ± 525.73	8909.21 ± 525.73
unsloth/Qwen3.6-27B-Q6-MTP	tg32	42.67 ± 8.76	43.97 ± 9.05

mtp-bench.py Results

All tests run with MTP and mmproj enabled.

unsloth/Qwen3.6-27B-Q4-MTP

Task	Predicted	Draft	Accepted	Accept Rate	Tokens/s
code_python	192	168	134	0.798	138.8
code_cpp	192	195	125	0.641	120.9
explain_concept	192	188	127	0.675	124.2
summarize	192	162	136	0.840	143.1
qa_factual	192	181	130	0.718	129.3
translation	192	209	121	0.579	114.1
creative_short	192	214	119	0.556	111.0
stepwise_math	192	173	133	0.769	135.6
long_code_review	192	209	121	0.579	112.6

Aggregate: 9 requests | 1728 predicted | 1699 draft | 1146 accepted | Accept rate: 67.45% | Wall time: 15.59s

unsloth/Qwen3.6-27B-Q5-MTP

Task	Predicted	Draft	Accepted	Accept Rate	Tokens/s
code_python	192	166	135	0.813	55.8
code_cpp	192	201	124	0.617	46.8
explain_concept	192	192	126	0.656	48.6
summarize	192	162	136	0.840	56.8
qa_factual	192	198	125	0.631	47.7
translation	192	196	125	0.638	47.6
creative_short	192	199	124	0.623	47.0
stepwise_math	192	177	132	0.746	53.3
long_code_review	192	204	121	0.593	44.8

Aggregate: 9 requests | 1728 predicted | 1695 draft | 1148 accepted | Accept rate: 67.73% | Wall time: 41.33s

unsloth/Qwen3.6-27B-Q6-MTP

Task	Predicted	Draft	Accepted	Accept Rate	Tokens/s
code_python	192	159	137	0.862	34.9
code_cpp	192	189	126	0.667	28.5
explain_concept	192	197	125	0.634	28.6
summarize	192	168	134	0.798	33.2
qa_factual	192	172	132	0.767	32.1
translation	192	192	126	0.656	29.2
creative_short	192	207	122	0.589	27.3
stepwise_math	192	176	132	0.750	31.7
long_code_review	192	202	123	0.609	27.4

Aggregate: 9 requests | 1728 predicted | 1662 draft | 1157 accepted | Accept rate: 69.61% | Wall time: 67.29s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment