Skip to content

Instantly share code, notes, and snippets.

@cybersiddhu
Last active August 4, 2026 12:24
Show Gist options
  • Select an option

  • Save cybersiddhu/2e365a8557a0e55745d84eb41e6a08c7 to your computer and use it in GitHub Desktop.

Select an option

Save cybersiddhu/2e365a8557a0e55745d84eb41e6a08c7 to your computer and use it in GitHub Desktop.
Artificial Analysis Deep-Dive Leaderboards (AA-Briefcase, AA-Omniscience, GDPval-AA v2) — extracted 2026-08-04

Artificial Analysis Deep-Dive Leaderboards (AA-Briefcase, AA-Omniscience, GDPval-AA v2) — extracted 2026-08-04

Artificial Analysis Deep-Dive Leaderboards (Non-Terminal-Bench)

Extracted from Artificial Analysis page HTML on 2026-08-04. Source: Artificial Analysis.

This file covers evaluations NOT in the Terminal-Bench v2.1 gist:

  • AA-Briefcase — Agentic knowledge work benchmark (91 tasks)
  • AA-Omniscience — Knowledge & hallucination benchmark (~6,000 questions)
  • GDPval-AA v2 — Real-world economically valuable tasks (Elo anchored to human baseline of 1,000)

AA-Briefcase

AA-Briefcase is a frontier agentic evaluation for long-horizon knowledge work, testing agents on realistic business workflows that require deliverables such as spreadsheets, presentations, and memos. 91 tasks, Terminus 2 agent harness.

AA-Briefcase Elo is a combined metric that aggregates rubric pass rate, analytical quality Elo and presentation Elo. Higher is better.

Table 1: AA-Briefcase Elo Leaderboard

Rank Model Creator Elo CI Release Date
1 Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic 1719 ±13 Jul 2026
2 Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 1691 ±13 Jul 2026
3 Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic 1606 ±11 Jul 2026
4 Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic 1574 ±11 Jun 2026
5 Kimi K3 (max) Kimi 1541 ±10 Jul 2026
6 GPT-5.6 Sol (max) OpenAI 1502 ±10 Jul 2026
7 Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic 1470 ±11 Jul 2026
8 Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic 1385 ±9 Jun 2026
9 Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic 1342 ±8 May 2026
10 Grok 4.5 (high) SpaceXAI 1314 ±9 Jul 2026
11 Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 1293 ±9 Jun 2026
12 Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic 1277 ±8 Apr 2026
13 GLM-5.2 (max) Z AI 1252 ±9 Jun 2026
14 Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic 1224 ±10 Jul 2026
15 Claude Sonnet 5 (Adaptive Reasoning, High Effort) Anthropic 1193 ±9 Jun 2026
16 GPT-5.5 (xhigh) OpenAI 1150 ±8 Apr 2026
17 MiniMax-M3 MiniMax 1107 ±8 Jun 2026
18 GPT-5.5 (high) OpenAI 1099 ±8 Apr 2026
19 Claude Opus 4.7 (Non-reasoning, High Effort) Anthropic 1083 ±8 Apr 2026
20 Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic 1075 ±8 Feb 2026
21 Claude Sonnet 5 (Adaptive Reasoning, Medium Effort) Anthropic 1056 ±9 Jun 2026
22 GPT-5.5 (medium) OpenAI 1000 ±0 Apr 2026
23 GLM-5.1 (Reasoning) Z AI 971 ±8 Apr 2026
24 Gemini 3.6 Flash (high) Google 962 ±9 Jul 2026
25 Claude Sonnet 5 (Adaptive Reasoning, Low Effort) Anthropic 930 ±9 Jun 2026
26 DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek 930 ±8 Apr 2026
27 Inkling Small Thinking Machines 917 ±11 Jul 2026
28 Qwen3.7 Max Alibaba 914 ±8 May 2026
29 MiMo-V2.5-Pro Xiaomi 880 ±8 Apr 2026
30 Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA 874 ±9 Jun 2026
31 Gemini 3.5 Flash (high) Google 872 ±8 May 2026
32 Gemini 3.5 Flash (medium) Google 871 ±8 May 2026
33 GPT-5.3 Codex (xhigh) OpenAI 870 ±8 Feb 2026
34 Muse Spark 1.1 (xhigh) Meta 869 ±11 Jul 2026
35 Inkling (xhigh) Thinking Machines 842 ±10 Jul 2026
36 DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek 833 ±8 Apr 2026
37 Kimi K2.6 Kimi 819 ±8 Apr 2026
38 Qwen3.6 27B (Reasoning) Alibaba 810 ±9 Apr 2026
39 Grok 4.3 (high) SpaceXAI 759 ±8 Apr 2026
40 GPT-5.4 mini (xhigh) OpenAI 717 ±9 Mar 2026
41 Muse Spark Meta 642 ±10 Apr 2026
42 Gemini 3.5 Flash-Lite Google 634 ±11 Jul 2026
43 Claude 4.5 Haiku (Reasoning) Anthropic 611 ±10 Oct 2025
44 KAT-Coder-Pro V1 KwaiKAT 599 ±11 Nov 2025
45 Qwen3.5 397B A17B (Reasoning) Alibaba 554 ±10 Feb 2026
46 Mistral Medium 3.5 Mistral 516 ±10 Apr 2026
47 Gemini 3.1 Pro Preview Google 458 ±11 Feb 2026
48 Gemma 4 31B (Reasoning) Google 374 ±11 Apr 2026
49 Command A+ Cohere 369 ±15 May 2026
50 North Mini Code Cohere 238 ±14 Jun 2026
51 Gemini 3.1 Flash-Lite Google 231 ±13 Mar 2026
52 Solar Pro 3 Upstage 138 ±13 Apr 2026
53 K2 Think V2 MBZUAI Institute of Foundation Models 59 ±13 Dec 2025
54 gpt-oss-120b (high) OpenAI 8 ±10 Aug 2025
55 gpt-oss-20b (high) OpenAI 0 ±0 Aug 2025
56 Llama 4 Maverick Meta 0 ±0 Apr 2025
57 NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA 0 ±0 Mar 2026

Table 2: AA-Briefcase Cost per Task (USD)

Mean cost per task from 91 tasks, including representative cache hit rates.

Model Creator Cost/Task
Gemma 4 31B (Reasoning) Google $0.00
North Mini Code Cohere $0.00
Command A+ Cohere $0.00
gpt-oss-20b (high) OpenAI $0.01
Gemini 3.1 Flash-Lite Google $0.03
MiMo-V2.5-Pro Xiaomi $0.07
DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek $0.08
DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek $0.09
Inkling Small Thinking Machines $0.12
Gemini 3.5 Flash-Lite Google $0.12
Solar Pro 3 Upstage $0.15
Llama 4 Maverick Meta $0.19
Grok 4.3 (high) SpaceXAI $0.24
GPT-5.4 mini (xhigh) OpenAI $0.34
Claude 4.5 Haiku (Reasoning) Anthropic $0.34
gpt-oss-120b (high) OpenAI $0.57
Gemini 3.1 Pro Preview Google $0.66
Qwen3.5 397B A17B (Reasoning) Alibaba $0.75
GLM-5.1 (Reasoning) Z AI $0.77
Mistral Medium 3.5 Mistral $0.80
Claude Sonnet 5 (Adaptive Reasoning, Low Effort) Anthropic $0.82
MiniMax-M3 MiniMax $0.94
Qwen3.6 27B (Reasoning) Alibaba $1.05
Grok 4.5 (high) SpaceXAI $1.05
Kimi K2.6 Kimi $1.05
Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA $1.61
Gemini 3.6 Flash (high) Google $1.62
Claude Sonnet 5 (Adaptive Reasoning, Medium Effort) Anthropic $1.73
Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic $1.78
GPT-5.3 Codex (xhigh) OpenAI $1.86
GPT-5.5 (medium) OpenAI $1.92
Qwen3.7 Max Alibaba $2.15
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic $2.41
Muse Spark 1.1 (xhigh) Meta $2.52
GLM-5.2 (max) Z AI $2.63
GPT-5.5 (high) OpenAI $3.30
Claude Sonnet 5 (Adaptive Reasoning, High Effort) Anthropic $3.76
Gemini 3.5 Flash (high) Google $4.05
Claude Opus 4.7 (Non-reasoning, High Effort) Anthropic $4.18
Gemini 3.5 Flash (medium) Google $4.85
GPT-5.5 (xhigh) OpenAI $5.15
Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic $5.25
GPT-5.6 Sol (max) OpenAI $5.32
NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA $6.14
Kimi K3 (max) Kimi $6.73
Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort) Anthropic $7.56
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic $8.26
Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic $10.41
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic $14.26
Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic $14.32
Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic $14.43
Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic $17.79
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic $22.30

Table 3: AA-Briefcase Output Tokens per Task

Model Creator Reasoning Tokens Answer Tokens Total
Llama 4 Maverick Meta 0k 5k 5k
Gemini 3.1 Flash-Lite Google 2k 3k 6k
Muse Spark Meta 4k 9k 13k
Gemma 4 31B (Reasoning) Google 6k 10k 16k
Gemini 3.1 Pro Preview Google 7k 9k 17k
GPT-5.5 (medium) OpenAI 4k 14k 18k
Grok 4.3 (high) SpaceXAI 6k 12k 19k
K2 Think V2 MBZUAI Institute of Foundation Models 11k 8k 19k
Qwen3.5 397B A17B (Reasoning) Alibaba 0k 20k 20k
KAT-Coder-Pro V1 KwaiKAT 0k 20k 20k
Gemini 3.5 Flash-Lite Google 8k 15k 23k
gpt-oss-20b (high) OpenAI 22k 3k 25k
Command A+ Cohere 2k 25k 27k
Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic 4k 24k 28k
GPT-5.5 (high) OpenAI 9k 19k 28k
Claude Sonnet 5 (Adaptive Reasoning, Low Effort) Anthropic 5k 24k 29k
Inkling Small Thinking Machines 6k 23k 29k
Claude 4.5 Haiku (Reasoning) Anthropic 0k 29k 29k
North Mini Code Cohere 7k 24k 31k
Qwen3.7 Max Alibaba 3k 29k 32k
GPT-5.4 mini (xhigh) OpenAI 17k 15k 32k
MiMo-V2.5-Pro Xiaomi 3k 30k 33k
Mistral Medium 3.5 Mistral 2k 32k 34k
Qwen3.6 27B (Reasoning) Alibaba 2k 35k 37k
DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek 4k 34k 38k
GPT-5.5 (xhigh) OpenAI 15k 23k 38k
GPT-5.3 Codex (xhigh) OpenAI 18k 22k 40k
DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek 6k 37k 42k
Kimi K2.6 Kimi 12k 30k 42k
GLM-5.1 (Reasoning) Z AI 9k 34k 43k
Claude Sonnet 5 (Adaptive Reasoning, Medium Effort) Anthropic 13k 34k 47k
Solar Pro 3 Upstage 18k 29k 47k
Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA 4k 43k 48k
Inkling (xhigh) Thinking Machines 12k 41k 52k
Grok 4.5 (high) SpaceXAI 5k 50k 55k
Muse Spark 1.1 (xhigh) Meta 11k 47k 58k
GPT-5.6 Sol (max) OpenAI 23k 35k 58k
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic 13k 51k 64k
Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic 17k 50k 66k
Gemini 3.6 Flash (high) Google 28k 50k 77k
MiniMax-M3 MiniMax 0k 82k 82k
Claude Sonnet 5 (Adaptive Reasoning, High Effort) Anthropic 33k 49k 82k
Claude Opus 4.7 (Non-reasoning, High Effort) Anthropic 0k 84k 84k
gpt-oss-120b (high) OpenAI 28k 70k 98k
Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic 17k 90k 107k
Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic 37k 77k 114k
GLM-5.2 (max) Z AI 75k 40k 115k
Gemini 3.5 Flash (medium) Google 74k 45k 119k
Kimi K3 (max) Kimi 56k 63k 120k
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic 72k 55k 126k
Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 67k 69k 136k
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic 65k 75k 139k
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 51k 93k 145k
Gemini 3.5 Flash (high) Google 96k 50k 146k
Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic 62k 108k 170k
Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic 114k 95k 209k
NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA 336k 268k 604k

Table 4: AA-Briefcase Time per Task (minutes)

Estimated wall-clock time per task: answer and reasoning tokens per task divided by canonical answer output speed, plus mean tool execution time. Lower is better.

Model Creator Min/Task
Gemini 3.1 Flash-Lite Google 0.9
Llama 4 Maverick Meta 1.7
Gemini 3.5 Flash-Lite Google 1.8
gpt-oss-20b (high) OpenAI 1.9
Gemini 3.1 Pro Preview Google 3.3
Qwen3.7 Max Alibaba 3.3
Grok 4.3 (high) SpaceXAI 3.4
GPT-5.4 mini (xhigh) OpenAI 4.0
Inkling Small Thinking Machines 4.5
GPT-5.5 (medium) OpenAI 4.6
Mistral Medium 3.5 Mistral 4.7
Claude 4.5 Haiku (Reasoning) Anthropic 5.0
Command A+ Cohere 5.0
Muse Spark 1.1 (xhigh) Meta 5.0
Qwen3.5 397B A17B (Reasoning) Alibaba 5.1
Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic 6.7
GPT-5.3 Codex (xhigh) OpenAI 6.8
Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA 7.0
Gemini 3.6 Flash (high) Google 7.4
Claude Sonnet 5 (Adaptive Reasoning, Low Effort) Anthropic 7.4
Solar Pro 3 Upstage 7.6
GPT-5.5 (high) OpenAI 7.8
Gemma 4 31B (Reasoning) Google 8.3
GPT-5.5 (xhigh) OpenAI 8.7
gpt-oss-120b (high) OpenAI 9.5
GLM-5.1 (Reasoning) Z AI 10.3
DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek 10.5
Qwen3.6 27B (Reasoning) Alibaba 10.7
Inkling (xhigh) Thinking Machines 10.9
Claude Sonnet 5 (Adaptive Reasoning, Medium Effort) Anthropic 11.8
MiMo-V2.5-Pro Xiaomi 12.3
GLM-5.2 (max) Z AI 14.1
Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic 14.8
Gemini 3.5 Flash (medium) Google 14.8
Grok 4.5 (high) SpaceXAI 15.9
Gemini 3.5 Flash (high) Google 16.1
MiniMax-M3 MiniMax 16.8
GPT-5.6 Sol (max) OpenAI 18.0
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic 18.1
Claude Sonnet 5 (Adaptive Reasoning, High Effort) Anthropic 18.3
Claude Opus 4.7 (Non-reasoning, High Effort) Anthropic 19.7
Kimi K2.6 Kimi 21.2
Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic 22.5
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic 22.8
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic 24.9
Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 25.8
Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic 26.0
North Mini Code Cohere 26.9
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 32.4
Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic 37.1
Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic 37.9
Kimi K3 (max) Kimi 55.9
NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA 69.3
Muse Spark Meta N/A
KAT-Coder-Pro V1 KwaiKAT N/A
K2 Think V2 MBZUAI Institute of Foundation Models N/A
DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek N/A

Table 5: AA-Briefcase Mean Turns per Task

Model Creator Turns
gpt-oss-20b (high) OpenAI 7
Gemini 3.1 Flash-Lite Google 14
K2 Think V2 MBZUAI Institute of Foundation Models 18
Qwen3.7 Max Alibaba 18
MiMo-V2.5-Pro Xiaomi 20
Gemini 3.5 Flash-Lite Google 22
Grok 4.5 (high) SpaceXAI 23
GPT-5.5 (medium) OpenAI 24
Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic 25
GPT-5.4 mini (xhigh) OpenAI 25
DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek 26
DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek 27
Grok 4.3 (high) SpaceXAI 27
GLM-5.1 (Reasoning) Z AI 27
Muse Spark Meta 28
Claude 4.5 Haiku (Reasoning) Anthropic 28
Qwen3.5 397B A17B (Reasoning) Alibaba 28
Command A+ Cohere 29
KAT-Coder-Pro V1 KwaiKAT 30
Claude Sonnet 5 (Adaptive Reasoning, Low Effort) Anthropic 31
Gemini 3.1 Pro Preview Google 32
GPT-5.5 (high) OpenAI 32
Inkling Small Thinking Machines 34
Qwen3.6 27B (Reasoning) Alibaba 34
Mistral Medium 3.5 Mistral 37
Gemini 3.6 Flash (high) Google 38
Claude Opus 4.7 (Non-reasoning, High Effort) Anthropic 38
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic 40
Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA 42
GPT-5.5 (xhigh) OpenAI 42
Solar Pro 3 Upstage 43
Llama 4 Maverick Meta 45
GPT-5.6 Sol (max) OpenAI 50
North Mini Code Cohere 51
Gemma 4 31B (Reasoning) Google 53
Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic 53
Kimi K2.6 Kimi 54
Claude Sonnet 5 (Adaptive Reasoning, Medium Effort) Anthropic 55
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic 55
GLM-5.2 (max) Z AI 56
GPT-5.3 Codex (xhigh) OpenAI 57
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic 67
Muse Spark 1.1 (xhigh) Meta 72
Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic 76
Inkling (xhigh) Thinking Machines 81
Gemini 3.5 Flash (medium) Google 83
Kimi K3 (max) Kimi 83
Gemini 3.5 Flash (high) Google 88
Claude Sonnet 5 (Adaptive Reasoning, High Effort) Anthropic 89
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 91
Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic 92
MiniMax-M3 MiniMax 101
Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic 103
gpt-oss-120b (high) OpenAI 113
Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 130
Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic 183
NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA 414

AA-Omniscience

AA-Omniscience is a knowledge and hallucination benchmark (~6,000 questions) that rewards accuracy, punishes bad guesses, and provides a comprehensive view of which models produce factually reliable outputs across different domains. Scores range from -100 to 100.

Table 6: AA-Omniscience Index Leaderboard

Rank Model Creator Index Accuracy Hallucination Rate
1 Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic 40 61% 55%
2 Gemini 3.1 Pro Preview Google 33 55% 50%
3 Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic 31 54% 50%
4 Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 30 53% 50%
5 Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic 28 53% 52%
6 Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic 27 47% 36%
7 Grok 4.5 (high) SpaceXAI 26 52% 54%
8 Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic 26 46% 36%
9 Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic 26 51% 52%
10 Gemini 3.6 Flash (high) Google 24 50% 54%
11 Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic 23 50% 55%
12 Gemini 3.5 Flash (high) Google 23 52% 61%
13 GPT-5.6 Sol (max) OpenAI 22 59% 89%
14 Gemini 3.5 Flash (medium) Google 22 51% 60%
15 GPT-5.6 Sol (xhigh) OpenAI 21 58% 89%
16 GPT-5.5 (xhigh) OpenAI 20 57% 86%
17 GPT-5.6 Sol (high) OpenAI 20 57% 88%
18 GPT-5.6 Sol (medium) OpenAI 19 57% 87%
19 Kimi K3 (max) Kimi 18 46% 51%
20 GPT-5.6 Sol (low) OpenAI 18 56% 87%
21 GPT-5.5 (high) OpenAI 18 56% 86%
22 Grok 4.3 (high) SpaceXAI 18 35% 25%
23 Muse Spark 1.1 (xhigh) Meta 18 41% 38%
24 GPT-5.5 (medium) OpenAI 17 56% 86%
25 Grok 4.3 (medium) SpaceXAI 17 28% 16%
26 Gemini 3 Pro Preview (high) Google 16 56% 91%
27 Grok 4.20 0309 v2 (Reasoning) SpaceXAI 15 27% 17%
28 Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic 15 38% 37%
29 GPT-5.5 (low) OpenAI 15 54% 86%
30 Claude Opus 4.7 (Non-reasoning, High Effort) Anthropic 14 44% 52%
31 Qwen3.7 Max Alibaba 14 30% 23%
32 Grok 4.3 (low) SpaceXAI 14 26% 16%
33 Claude Opus 4.6 (Adaptive Reasoning, Max Effort) Anthropic 14 46% 61%
34 Grok 4.20 0309 (Reasoning) SpaceXAI 13 29% 22%
35 Claude Opus 4.5 (Reasoning) Anthropic 13 46% 60%
36 Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic 12 40% 46%
37 Gemini 3 Flash Preview (Reasoning) Google 12 54% 92%
38 GPT-5.5 Instant (May 2026) OpenAI 10 45% 64%
39 Qwen3.6 Max Preview Alibaba 10 38% 44%
40 GPT-5.3 Codex (xhigh) OpenAI 10 52% 87%
41 Grok Build 0.1 0616 SpaceXAI 7 51% 90%
42 Gemini 3.5 Flash-Lite Google 7 30% 34%
43 Kimi K2.6 Kimi 6 33% 39%
44 GPT-5.4 (xhigh) OpenAI 6 50% 89%
45 GPT-5.1 (high) OpenAI 6 38% 51%
46 MiMo-V2-Pro Xiaomi 5 27% 30%
47 GPT-5.4 (low) OpenAI 5 47% 81%
48 G9v3-39A5B AI9Stars 4 17% 15%
49 Muse Spark Meta 4 45% 73%
50 GLM-5.2 (max) Z AI 4 25% 28%
51 Grok 4 SpaceXAI 4 41% 64%
52 MiMo-V2.5-Pro Xiaomi 4 23% 25%
53 GPT-5.5 Instant (June 2026) OpenAI 3 44% 73%
54 Claude Opus 4.6 (Non-reasoning, High Effort) Anthropic 3 45% 76%
55 Kimi K3 (low) Kimi 3 44% 74%
56 Qwen3.6 Plus Alibaba 3 26% 32%
57 Qwen3.7 Plus Alibaba 2 22% 25%
58 Gemini 3 Pro Preview (low) Google 2 47% 85%
59 Inkling (xhigh) Thinking Machines 2 40% 63%
60 GLM-5 (Reasoning) Z AI 2 27% 34%
61 GLM-5.1 (Reasoning) Z AI 2 24% 29%
62 MiniMax-M3 MiniMax 1 15% 16%
63 Gemini 3.5 Flash (minimal) Google 1 43% 73%
64 GPT-5.6 Sol (Non-reasoning) OpenAI 1 48% 91%
65 MiniMax-M2.7 MiniMax 1 26% 34%
66 Claude 4.5 Sonnet (Reasoning) Anthropic 0 32% 47%
67 Claude 3.7 Sonnet (Reasoning) Anthropic 0 28% 39%
68 Claude 4 Sonnet (Reasoning) Anthropic 0 22% 29%
69 GPT-5.2 (medium) OpenAI 0 38% 61%
70 GPT-5.6 Terra (max) OpenAI -0 46% 85%
71 MiniCPM5-1B (Non-reasoning) OpenBMB -1 0% 1%
72 Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA -1 22% 29%
73 GPT-5.2 (xhigh) OpenAI -1 44% 80%
74 Claude Sonnet 5 (Non-reasoning, High Effort) Anthropic -1 33% 50%
75 Claude Sonnet 4.6 (Non-reasoning, Low Effort) Anthropic -2 36% 60%
76 GPT-5.2 Codex (xhigh) OpenAI -2 41% 73%
77 Claude Sonnet 4.6 (Non-reasoning, High Effort) Anthropic -3 38% 66%
78 GPT-5.6 Terra (xhigh) OpenAI -3 45% 87%
79 Gemini 3 Flash Preview (Non-reasoning) Google -4 46% 90%
80 Claude Opus 4.5 (Non-reasoning) Anthropic -4 41% 75%
81 Command A+ Cohere -4 9% 14%
82 GPT-5.6 Terra (high) OpenAI -4 44% 87%
83 Claude 4.5 Haiku (Reasoning) Anthropic -4 17% 26%
84 G9v3-3B AI9Stars -5 7% 12%
85 GPT-5.5 (Non-reasoning) OpenAI -5 45% 91%
86 GPT-5.6 Terra (medium) OpenAI -5 44% 88%
87 GPT-5.1 Codex (high) OpenAI -6 39% 74%
88 Grok 3 mini Reasoning (high) SpaceXAI -6 15% 25%
89 GLM-5.2 (Non-reasoning) Z AI -6 20% 34%
90 GPT-5 Codex (high) OpenAI -7 39% 74%
91 GPT-5.6 Terra (low) OpenAI -7 43% 88%
92 Claude 4.5 Haiku (Non-reasoning) Anthropic -8 14% 25%
93 GPT-5 (high) OpenAI -8 41% 82%
94 Kimi K2.5 (Reasoning) Kimi -8 34% 65%
95 Inkling Small Thinking Machines -9 31% 57%
96 Claude 4.5 Sonnet (Non-reasoning) Anthropic -9 28% 51%
97 Claude 3.7 Sonnet (Non-reasoning) Anthropic -9 28% 52%
98 Claude 4 Sonnet (Non-reasoning) Anthropic -9 22% 41%
99 MiMo-V2.5 Xiaomi -9 17% 32%
100 DeepSeek V4 Pro (Reasoning, High Effort) DeepSeek -10 42% 89%
101 Kimi K2.6 (Non-reasoning) Kimi -10 23% 43%
102 DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek -10 43% 94%
103 GPT-5 (low) OpenAI -10 37% 76%
104 GPT-5 (medium) OpenAI -10 39% 80%
105 JT-4.1 Flash 236B A21B China Mobile -10 23% 43%
106 o1 OpenAI -11 35% 69%
107 Kimi K2.7 Code Kimi -11 39% 80%
108 GPT-4o (Nov '24) OpenAI -11 20% 38%
109 GPT-5 mini (medium) OpenAI -11 22% 42%
110 GPT-5.6 Luna (max) OpenAI -11 42% 90%
111 GLM-5 (Non-reasoning) Z AI -11 23% 44%
112 GPT-5.6 Luna (xhigh) OpenAI -12 41% 90%
113 Qwen3.5 Omni Plus Alibaba -12 17% 36%
114 GPT-5.6 Luna (high) OpenAI -12 41% 90%
115 GPT-5.2 (Non-reasoning) OpenAI -14 29% 61%
116 GPT-5.6 Luna (medium) OpenAI -14 40% 89%
117 Kimi K2.5 (Non-reasoning) Kimi -14 23% 48%
118 MiMo-V2-Omni-0327 Xiaomi -14 18% 40%
119 Gemini 2.5 Pro Google -14 39% 87%
120 GLM-5-Turbo Z AI -15 29% 62%
121 GPT-5.6 Luna (low) OpenAI -15 39% 88%
122 o3 OpenAI -15 38% 87%
123 Gemini 3.1 Flash-Lite Google -16 36% 82%
124 DeepSeek V4 Flash 0731 (Reasoning, Max Effort) DeepSeek -16 37% 84%
125 GPT-5.4 (Non-reasoning) OpenAI -16 37% 83%
126 DeepSeek V3.2 Speciale DeepSeek -16 38% 89%
127 GPT-5.1 Codex mini (high) OpenAI -17 23% 51%
128 MiniCPM5-1B (Reasoning) OpenBMB -17 2% 19%
129 GPT-5 mini (high) OpenAI -17 24% 54%
130 Motif 3 (Beta) Motif Technologies -17 21% 49%
131 Llama 3.1 Instruct 405B Meta -17 22% 51%
132 MiMo-V2-Omni Xiaomi -17 19% 44%
133 MiMo-V2-Flash (Feb 2026) Xiaomi -18 20% 48%
134 Hy3 Tencent -18 32% 73%
135 GPT-5.4 mini (xhigh) OpenAI -19 37% 90%
136 GPT-5.4 nano (medium) OpenAI -19 21% 50%
137 GLM 5V Turbo (Reasoning) Z AI -19 29% 68%
138 Qwen3.6 27B (Reasoning) Alibaba -20 19% 48%
139 Gemma 4 E4B (Reasoning) Google -20 9% 31%
140 GPT-5.4 mini (medium) OpenAI -20 36% 89%
141 Kimi K2 Thinking Kimi -20 30% 72%
142 DeepSeek V3.2 (Reasoning) DeepSeek -21 33% 82%
143 GLM-5.1 (Non-reasoning) Z AI -21 26% 63%
144 Qwen3.6 35B A3B (Reasoning) Alibaba -21 19% 50%
145 KAT Coder Pro V2 KwaiKAT -22 22% 56%
146 GPT-4o (Aug '24) OpenAI -22 24% 59%
147 Qwen3 Coder 480B A35B Instruct Alibaba -22 15% 43%
148 DeepSeek V4 Flash (Reasoning, High Effort) DeepSeek -22 36% 90%
149 LongCat 2.0 LongCat -23 30% 75%
150 DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek -23 37% 96%
151 Claude 3.5 Haiku Anthropic -23 13% 42%
152 JT-35B-Flash China Mobile -23 25% 63%
153 GPT-5.6 Terra (Non-reasoning) OpenAI -23 36% 94%
154 Gemma 4 E2B (Reasoning) Google -24 7% 33%
155 DeepSeek V3.1 Terminus (Reasoning) DeepSeek -24 28% 73%
156 GPT-5 nano (medium) OpenAI -25 16% 49%
157 Cogito v2.1 (Reasoning) Deep Cogito -25 30% 79%
158 GPT-5.6 Luna (Non-reasoning) OpenAI -25 28% 73%
159 GLM-4.5 (Reasoning) Z AI -25 25% 67%
160 Magistral Medium 1.2 Mistral -26 21% 59%
161 Agnes 2.5 Pro Alpha Sapiens AI -26 32% 87%
162 Magistral Medium 1 Mistral -26 21% 59%
163 Kimi K2 0905 Kimi -26 25% 70%
164 GLM-4.6V (Reasoning) Z AI -26 16% 50%
165 DeepSeek R1 0528 (May '25) DeepSeek -27 31% 84%
166 Kimi K2 Kimi -28 27% 74%
167 Nex-N2-Pro Nex AGI -28 34% 95%
168 GPT-5 nano (high) OpenAI -28 18% 56%
169 Gemini 2.5 Flash (Reasoning) Google -28 25% 71%
170 DeepSeek V3.1 (Reasoning) DeepSeek -28 29% 80%
171 Grok 4 Fast (Reasoning) SpaceXAI -28 23% 66%
172 Grok 4.1 Fast (Reasoning) SpaceXAI -29 25% 72%
173 GPT-5.4 nano (xhigh) OpenAI -30 25% 74%
174 Qwen3.5 397B A17B (Reasoning) Alibaba -30 31% 89%
175 Mistral Small 4 (Reasoning) Mistral -30 22% 67%
176 DeepSeek V3.2 Exp (Reasoning) DeepSeek -30 28% 81%
177 DeepSeek V4 Pro (Non-reasoning) DeepSeek -30 31% 88%
178 Llama 3.1 Instruct 8B Meta -30 8% 42%
179 Gemma 3 270M Google -31 1% 32%
180 DeepSeek R1 (Jan '25) DeepSeek -31 31% 90%
181 Mistral Medium 3 Mistral -31 18% 61%
182 GLM-4.6 (Non-reasoning) Z AI -32 21% 66%
183 Devstral Medium Mistral -32 19% 63%
184 Grok 4.3 (Non-reasoning) SpaceXAI -32 24% 74%
185 Grok 3 SpaceXAI -32 28% 85%
186 MiniMax-M2.1 MiniMax -33 20% 67%
187 Hermes 4 - Llama-3.1 405B (Non-reasoning) Nous Research -33 26% 80%
188 LFM2.5-8B-A1B Liquid AI -33 9% 47%
189 K2 Think V2 MBZUAI Institute of Foundation Models -34 16% 59%
190 GPT-5 (minimal) OpenAI -34 28% 86%
191 Mistral Large 2 (Nov '24) Mistral -34 20% 68%
192 Qwen3 Max Thinking Alibaba -34 30% 92%
193 Doubao Seed Code ByteDance Seed -34 25% 79%
194 GLM-4.7 (Reasoning) Z AI -35 29% 90%
195 Hy3-preview (Reasoning) Tencent -35 28% 87%
196 Gemini 2.5 Flash Preview (Sep '25) (Reasoning) Google -35 29% 89%
197 GPT-5.1 (Non-reasoning) OpenAI -35 29% 90%
198 Qwen3.5 0.8B (Reasoning) Alibaba -35 1% 37%
199 o4-mini (high) OpenAI -36 25% 80%
200 ERNIE 4.5 300B A47B Baidu -36 19% 67%
201 Grok Code Fast 1 SpaceXAI -36 24% 79%
202 Hermes 4 - Llama-3.1 405B (Reasoning) Nous Research -36 30% 95%
203 Qwen3.5 397B A17B (Non-reasoning) Alibaba -36 24% 80%
204 GPT-4.1 OpenAI -36 24% 80%
205 Nova Premier Amazon -36 19% 68%
206 Mistral Medium 3.5 Mistral -36 25% 82%
207 Hy3-preview (Non-reasoning) Tencent -36 22% 76%
208 KAT-Coder-Pro V1 KwaiKAT -37 17% 67%
209 Step 3.7 Flash StepFun -38 25% 84%
210 DeepSeek V3 0324 DeepSeek -38 22% 77%
211 Ring-2.6-1T InclusionAI -38 25% 84%
212 Qwen3 Max Thinking (Preview) Alibaba -38 28% 91%
213 GLM-4.6V (Non-reasoning) Z AI -38 18% 67%
214 Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning) Google -38 27% 88%
215 MiMo-V2.5-Pro (Non-reasoning) Xiaomi -38 27% 89%
216 GPT-5.4 nano (Non-Reasoning) OpenAI -39 14% 61%
217 Mistral Large 3 Mistral -39 24% 84%
218 Qwen3.5 122B A10B (Reasoning) Alibaba -40 25% 85%
219 MiniMax-M2.5 MiniMax -40 26% 89%
220 Nova Lite Amazon -40 10% 56%
221 Qwen3 Max (Preview) Alibaba -41 24% 86%
222 Llama 3.1 Instruct 70B Meta -41 19% 75%
223 Step 3.5 Flash StepFun -41 24% 85%
224 Llama 3.1 Nemotron Instruct 70B NVIDIA -41 16% 69%
225 DeepSeek V3.1 (Non-reasoning) DeepSeek -41 23% 84%
226 o3-mini (high) OpenAI -41 21% 79%
227 DeepSeek V3 (Dec '24) DeepSeek -41 25% 89%
228 Gemma 4 E4B (Non-reasoning) Google -42 8% 54%
229 GLM-4.6 (Reasoning) Z AI -42 27% 95%
230 Llama 4 Maverick Meta -42 24% 87%
231 DeepSeek V3.1 Terminus (Non-reasoning) DeepSeek -42 23% 85%
232 Nanbeige4.1-3B Nanbeige -42 10% 57%
233 Gemini 2.5 Flash (Non-reasoning) Google -42 27% 93%
234 Qwen3.5 27B (Reasoning) Alibaba -42 21% 80%
235 NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA -42 24% 87%
236 Qwen3 235B A22B 2507 Instruct Alibaba -42 17% 71%
237 Gemini 2.0 Flash (Feb '25) Google -43 24% 87%
238 Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning) Google -43 14% 66%
239 Grok 4.20 0309 (Non-reasoning) SpaceXAI -43 26% 93%
240 NVIDIA Nemotron Nano 9B V2 (Reasoning) NVIDIA -43 11% 61%
241 Qwen3 Max Alibaba -43 24% 89%
242 Magistral Small 1 Mistral -43 18% 75%
243 MiMo-V2-Flash (Reasoning) Xiaomi -43 25% 91%
244 GLM-4.5V (Reasoning) Z AI -44 21% 81%
245 Step 3.5 Flash 2603 StepFun -44 25% 92%
246 Olmo 3.1 32B Think Allen Institute for AI -44 14% 67%
247 ERNIE 5.0 Thinking Preview Baidu -44 22% 85%
248 DeepSeek V4 Flash (Non-reasoning) DeepSeek -44 26% 95%
249 Trinity Large Thinking Arcee AI -44 23% 87%
250 Qwen3 235B A22B (Reasoning) Alibaba -45 18% 77%
251 Grok 4.20 0309 v2 (Non-reasoning) SpaceXAI -45 27% 97%
252 Qwen3 VL 235B A22B (Reasoning) Alibaba -45 21% 83%
253 Gemini 2.5 Flash-Lite (Reasoning) Google -45 18% 75%
254 Gemma 4 31B (Reasoning) Google -45 20% 82%
255 Mistral Medium 3.1 Mistral -45 20% 81%
256 Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) NVIDIA -46 20% 82%
257 Qwen3 235B A22B 2507 (Reasoning) Alibaba -46 23% 89%
258 Llama Nemotron Super 49B v1.5 (Non-reasoning) NVIDIA -46 12% 66%
259 Nova 2.0 Pro Preview (low) Amazon -46 22% 87%
260 Llama Nemotron Super 49B v1.5 (Reasoning) NVIDIA -46 17% 76%
261 GPT-5.4 mini (Non-Reasoning) OpenAI -46 25% 95%
262 Devstral 2 Mistral -46 21% 85%
263 GLM-4.7 (Non-reasoning) Z AI -46 24% 92%
264 Qwen3.5 35B A3B (Reasoning) Alibaba -46 20% 84%
265 DeepSeek R1 Distill Llama 70B DeepSeek -46 19% 82%
266 DeepSeek V3.2 (Non-reasoning) DeepSeek -47 24% 93%
267 Qwen3.5 27B (Non-reasoning) Alibaba -47 16% 74%
268 MiniMax-M2 MiniMax -47 22% 88%
269 DeepSeek V3.2 Exp (Non-reasoning) DeepSeek -47 23% 90%
270 Hermes 4 - Llama-3.1 70B (Non-reasoning) Nous Research -47 18% 80%
271 MiniMax M1 80k MiniMax -47 21% 87%
272 Qwen3.5 2B (Reasoning) Alibaba -47 5% 55%
273 Nova Pro Amazon -48 17% 78%
274 Claude 3 Haiku Anthropic -48 17% 78%
275 Ring-1T InclusionAI -48 20% 84%
276 Nova 2.0 Pro Preview (medium) Amazon -48 22% 90%
277 Gemma 4 26B A4B (Reasoning) Google -48 18% 81%
278 K2-V2 (low) MBZUAI Institute of Foundation Models -48 16% 76%
279 Nova 2.0 Pro Preview (Non-reasoning) Amazon -48 16% 77%
280 Mistral Small 4 (Non-reasoning) Mistral -48 16% 77%
281 Command A Cohere -48 16% 76%
282 MiMo-V2-Flash (Non-reasoning) Xiaomi -48 15% 75%
283 Nova Micro Amazon -49 10% 65%
284 North Mini Code Cohere -49 19% 83%
285 Qwen3 14B (Reasoning) Alibaba -49 15% 75%
286 Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) NVIDIA -50 13% 73%
287 Grok 4 Fast (Non-reasoning) SpaceXAI -50 16% 78%
288 Hermes 4 - Llama-3.1 70B (Reasoning) Nous Research -50 23% 95%
289 K2-V2 (medium) MBZUAI Institute of Foundation Models -50 17% 81%
290 Nova 2.0 Omni (low) Amazon -50 18% 84%
291 gpt-oss-120b (high) OpenAI -50 22% 91%
292 GPT-4.1 mini OpenAI -50 18% 82%
293 gpt-oss-120b (low) OpenAI -50 16% 78%
294 Qwen3 VL 32B (Reasoning) Alibaba -50 17% 80%
295 Mistral Small 3.1 Mistral -50 15% 77%
296 Mistral Small 3.2 Mistral -50 15% 76%
297 Qwen3 32B (Reasoning) Alibaba -51 17% 82%
298 Qwen3 Next 80B A3B (Reasoning) Alibaba -51 17% 82%
299 Qwen3 VL 235B A22B Instruct Alibaba -51 20% 89%
300 Qwen3 Coder 30B A3B Instruct Alibaba -51 16% 79%
301 Seed-OSS-36B-Instruct ByteDance Seed -51 18% 83%
302 Grok 4.1 Fast (Non-reasoning) SpaceXAI -51 17% 82%
303 Ling-2.6-1T InclusionAI -51 21% 92%
304 INTELLECT-3 Prime Intellect -51 19% 87%
305 Nova 2.0 Lite (low) Amazon -51 16% 80%
306 Olmo 3.1 32B Instruct Allen Institute for AI -51 11% 70%
307 Devstral Small (Jul '25) Mistral -51 15% 77%
308 Qwen3 30B A3B (Reasoning) Alibaba -51 16% 80%
309 Gemma 4 31B (Non-reasoning) Google -51 17% 82%
310 EXAONE 4.5 33B LG AI Research -51 16% 81%
311 NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) NVIDIA -52 17% 83%
312 Gemma 4 12B (Reasoning) Google -52 16% 81%
313 Llama 3.3 Instruct 70B Meta -52 18% 85%
314 Qwen3 VL 8B Instruct Alibaba -52 20% 90%
315 LFM2 2.6B Liquid AI -52 5% 60%
316 Qwen2.5 Instruct 72B Alibaba -52 18% 85%
317 Mercury 2 Inception -52 20% 92%
318 Llama 4 Scout Meta -52 15% 78%
319 Qwen3 VL 8B (Reasoning) Alibaba -52 20% 91%
320 Nemotron Cascade 2 30B A3B NVIDIA -52 18% 85%
321 Qwen3.5 9B (Reasoning) Alibaba -53 16% 81%
322 Gemma 4 12B (Non-reasoning) Google -53 12% 73%
323 Qwen3.6 27B (Non-reasoning) Alibaba -53 17% 84%
324 HyperCLOVA X SEED Think (32B) Naver -53 15% 79%
325 Qwen3 235B A22B (Non-reasoning) Alibaba -53 18% 87%
326 Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning) Google -53 18% 87%
327 GLM-4.5V (Non-reasoning) Z AI -54 18% 87%
328 Llama 3 Instruct 70B Meta -54 18% 87%
329 Solar Pro 3 Upstage -54 18% 88%
330 Qwen3.5 122B A10B (Non-reasoning) Alibaba -54 19% 89%
331 Solar Open 100B (Reasoning) Upstage -54 18% 88%
332 Nova 2.0 Lite (high) Amazon -54 19% 90%
333 GPT-5 mini (minimal) OpenAI -54 18% 88%
334 Apertus 70B Instruct Swiss AI Initiative -55 13% 79%
335 Llama 3.2 Instruct 1B Meta -55 7% 67%
336 Tri-21B-think Preview Trillion Labs -55 9% 70%
337 Ling-1T InclusionAI -55 20% 94%
338 Jamba 1.7 Large AI21 Labs -56 20% 94%
339 Nova 2.0 Lite (medium) Amazon -56 18% 90%
340 Apriel-v1.5-15B-Thinker ServiceNow -56 16% 86%
341 Nemotron 3 Nano Omni 30B A3B Reasoning NVIDIA -56 15% 83%
342 GPT-4.1 nano OpenAI -56 13% 80%
343 HyperNova 60B 2605 Multiverse Computing -56 14% 82%
344 Solar Pro 2 (Reasoning) Upstage -57 19% 94%
345 K2-V2 (high) MBZUAI Institute of Foundation Models -57 18% 92%
346 Phi-4 Microsoft -57 13% 81%
347 Devstral Small 2 Mistral -57 15% 85%
348 Qwen3.5 4B (Reasoning) Alibaba -57 13% 80%
349 Qwen3 30B A3B 2507 (Reasoning) Alibaba -57 16% 87%
350 Devstral Small (May '25) Mistral -57 16% 87%
351 Gemini 2.5 Flash-Lite (Non-reasoning) Google -57 15% 85%
352 NVIDIA Nemotron Nano 9B V2 (Non-reasoning) NVIDIA -57 10% 74%
353 Mi:dm K 2.5 Pro Korea Telecom -57 17% 89%
354 Qwen3 VL 30B A3B (Reasoning) Alibaba -57 17% 89%
355 Nova 2.0 Omni (medium) Amazon -58 18% 92%
356 K-EXAONE (Reasoning) LG AI Research -58 16% 89%
357 Llama 3.3 Nemotron Super 49B v1 (Reasoning) NVIDIA -58 17% 91%
358 Olmo 3 32B Think Allen Institute for AI -58 14% 85%
359 Apriel-v1.6-15B-Thinker ServiceNow -59 17% 91%
360 Ring-flash-2.0 InclusionAI -59 16% 90%
361 Step3 VL 10B StepFun -59 13% 82%
362 DiffusionGemma 26B A4B Google -59 17% 91%
363 LFM2 24B A2B Liquid AI -59 6% 70%
364 Nova 2.0 Lite (Non-reasoning) Amazon -59 14% 84%
365 Qwen3 Next 80B A3B Instruct Alibaba -59 18% 93%
366 GLM-4.7-Flash (Reasoning) Z AI -59 16% 89%
367 Sarvam 105B (high) Sarvam -60 18% 94%
368 Qwen3 4B 2507 Instruct Alibaba -60 10% 78%
369 Qwen3.6 35B A3B (Non-reasoning) Alibaba -60 17% 93%
370 gpt-oss-20b (low) OpenAI -60 14% 87%
371 Qwen3 Coder Next Alibaba -61 16% 91%
372 Qwen3 Omni 30B A3B (Reasoning) Alibaba -61 15% 89%
373 EXAONE 4.0 32B (Reasoning) LG AI Research -61 14% 86%
374 Qwen3 4B 2507 (Reasoning) Alibaba -61 13% 84%
375 Granite 4.0 H Small IBM -61 14% 88%
376 Phi-4 Mini Instruct Microsoft -61 8% 76%
377 K-EXAONE (Non-reasoning) LG AI Research -61 12% 84%
378 Motif-2-12.7B-Reasoning Motif Technologies -61 15% 90%
379 Gemma 4 E2B (Non-reasoning) Google -61 8% 75%
380 Solar Pro 2 (Non-reasoning) Upstage -62 16% 91%
381 Qwen3.5 35B A3B (Non-reasoning) Alibaba -62 16% 92%
382 Mi:dm K 2.5 Pro Preview Korea Telecom -62 16% 93%
383 Gemma 4 26B A4B (Non-reasoning) Google -62 16% 92%
384 Falcon-H1R-7B TII UAE -62 14% 89%
385 GLM-4.7-Flash (Non-reasoning) Z AI -62 12% 84%
386 Sarvam M (Reasoning) Sarvam -62 15% 91%
387 EXAONE 4.0 32B (Non-reasoning) LG AI Research -62 10% 81%
388 GLM-4.5-Air Z AI -63 16% 92%
389 Qwen3 VL 32B Instruct Alibaba -63 15% 91%
390 Qwen3 VL 30B A3B Instruct Alibaba -63 16% 93%
391 Llama 3.2 Instruct 11B (Vision) Meta -63 10% 82%
392 Tri-21B-Think Trillion Labs -63 12% 85%
393 Ling-flash-2.0 InclusionAI -63 14% 90%
394 gpt-oss-20b (high) OpenAI -64 16% 94%
395 GPT-5 nano (minimal) OpenAI -64 11% 84%
396 Ministral 3 3B Mistral -64 8% 78%
397 DeepSeek R1 0528 Qwen3 8B DeepSeek -64 12% 86%
398 NVIDIA Nemotron Nano 12B v2 VL (Reasoning) NVIDIA -64 14% 91%
399 Nova 2.0 Omni (Non-reasoning) Amazon -64 13% 89%
400 JT-MINI China Mobile -65 14% 91%
401 Granite 4.1 8B IBM -65 12% 87%
402 Reka Flash 3 Reka AI -65 13% 90%
403 Mistral 7B Instruct Mistral -65 8% 79%
404 Magistral Small 1.2 Mistral -65 13% 91%
405 Qwen3 8B (Reasoning) Alibaba -65 13% 89%
406 Qwen3 30B A3B (Non-reasoning) Alibaba -65 12% 87%
407 Qwen3.5 Omni Flash Alibaba -66 14% 94%
408 Ling 2.6 Flash InclusionAI -66 15% 96%
409 Gemma 3 27B Instruct Google -66 12% 90%
410 Qwen3 30B A3B 2507 Instruct Alibaba -66 15% 95%
411 Ministral 3 14B Mistral -67 12% 90%
412 Qwen3 14B (Non-reasoning) Alibaba -67 13% 92%
413 Granite 4.1 30B IBM -68 14% 95%
414 Ministral 3 8B Mistral -68 11% 89%
415 Molmo2-8B Allen Institute for AI -69 11% 91%
416 Qwen3 Omni 30B A3B Instruct Alibaba -69 14% 98%
417 NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) NVIDIA -69 11% 91%
418 Qwen3 VL 4B (Reasoning) Alibaba -69 12% 93%
419 Llama 3 Instruct 8B Meta -70 10% 88%
420 LongCat Flash Lite LongCat -70 13% 96%
421 Qwen3.5 9B (Non-reasoning) Alibaba -71 14% 99%
422 NVIDIA Nemotron 3 Nano 4B NVIDIA -72 9% 89%
423 LFM2 1.2B Liquid AI -72 4% 79%
424 Sarvam 30B (high) Sarvam -72 13% 97%
425 Granite 4.0 350M IBM -72 3% 78%
426 Celeris-1 Celeris -73 11% 94%
427 NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning) NVIDIA -73 11% 95%
428 Granite 4.0 H 1B IBM -74 5% 83%
429 Olmo 3 7B Think Allen Institute for AI -74 11% 95%
430 Jamba Reasoning 3B AI21 Labs -74 7% 86%
431 LFM2.5-1.2B-Instruct Liquid AI -74 6% 85%
432 Qwen3 8B (Non-reasoning) Alibaba -74 10% 94%
433 Jamba 1.7 Mini AI21 Labs -74 11% 97%
434 Apertus 8B Instruct Swiss AI Initiative -75 10% 95%
435 Qwen3.5 4B (Non-reasoning) Alibaba -75 11% 97%
436 LFM2 8B A1B Liquid AI -75 7% 88%
437 Qwen3 VL 4B Instruct Alibaba -76 10% 96%
438 Granite 3.3 8B (Non-reasoning) IBM -77 9% 93%
439 Gemma 3 12B Instruct Google -77 10% 97%
440 Qwen3 1.7B (Reasoning) Alibaba -77 8% 93%
441 Gemma 3 1B Instruct Google -77 4% 84%
442 Granite 4.1 3B IBM -77 9% 95%
443 Olmo 3 7B Instruct Allen Institute for AI -78 7% 92%
444 Granite 4.0 Micro IBM -78 9% 96%
445 Ling-mini-2.0 InclusionAI -79 9% 96%
446 Gemma 3n E2B Instruct Google -80 7% 92%
447 Gemma 3n E4B Instruct Google -80 8% 96%
448 Qwen3 0.6B (Reasoning) Alibaba -82 6% 92%
449 Granite 4.0 1B IBM -82 6% 94%
450 Exaone 4.0 1.2B (Reasoning) LG AI Research -82 6% 94%
451 Exaone 4.0 1.2B (Non-reasoning) LG AI Research -83 5% 91%
452 Qwen3.5 2B (Non-reasoning) Alibaba -83 7% 97%
453 Gemma 3 4B Instruct Google -83 8% 99%
454 Qwen3 1.7B (Non-reasoning) Alibaba -83 7% 96%
455 LFM2.5-1.2B-Thinking Liquid AI -84 7% 97%
456 LFM2.5-VL-1.6B Liquid AI -84 5% 94%
457 MiniCPM-V 4.6 1.3B OpenBMB -85 6% 97%
458 Tiny Aya Global Cohere -85 6% 96%
459 Qwen3 0.6B (Non-reasoning) Alibaba -86 4% 94%
460 Granite 4.0 H 350M IBM -87 4% 94%
461 Qwen3.5 0.8B (Non-reasoning) Alibaba -89 5% 99%

Table 6 reflects all 461 models with a numeric AA-Omniscience index score as of 2026-08-04.

Table 7: AA-Omniscience Domains

Benchmark covers 6 domains with subdomains:

Domain Subdomains
Business Accounting (200), Business & Management, Corporate & Markets (200), Economics (200), Financial Institutions (150), Investments (150)
Humanities & Social Sciences History (250), Literature (150), Philosophy (150), Politics & International Relations (300), Religion (150)
Science, Engineering & Mathematics Biology (100), Chemistry (100), Engineering (400), Geography & Environments, Mathematics (200), Physics (100), Psychology (50)
Health Biomedical Sciences (200), Medicine (700), Public Health (100)
Law Admin Law (50), Constitutional Law (150), Contract Law (200), Corporations & Business Law (200), Criminal Law (200), Property Law (100), Tort Law (100)
Software Engineering (SWE) C (100), Dart (50), Go (50), HTML (50), Java (100), JavaScript/TypeScript (200), Julia (25), Kotlin (50), PHP (50), Python (200), R (50), Rust (50), Swift (25)

Table 8: AA-Omniscience Cost Breakdown (Total USD)

Model Creator Total Cost
MiMo-V2-Pro Xiaomi $0.00
G9v3-39A5B AI9Stars $0.00
Muse Spark Meta $0.00
Claude 3.7 Sonnet (Reasoning) Anthropic $0.00
MiniCPM5-1B (Non-reasoning) OpenBMB $0.00
Command A+ Cohere $0.00
G9v3-3B AI9Stars $0.00
JT-4.1 Flash 236B A21B China Mobile $0.00
MiMo-V2-Omni-0327 Xiaomi $0.00
GLM-5-Turbo Z AI $0.00
DeepSeek V3.2 Speciale DeepSeek $0.00
MiniCPM5-1B (Reasoning) OpenBMB $0.00
Motif 3 (Beta) Motif Technologies $0.00
MiMo-V2-Omni Xiaomi $0.00
MiMo-V2-Flash (Feb 2026) Xiaomi $0.00
GLM 5V Turbo (Reasoning) Z AI $0.00
Claude 3.5 Haiku Anthropic $0.00
JT-35B-Flash China Mobile $0.00
Gemma 4 E2B (Reasoning) Google $0.00
GLM-4.5 (Reasoning) Z AI $0.00
Magistral Medium 1 Mistral $0.00
Grok 4.1 Fast (Reasoning) SpaceXAI $0.00
Gemma 3 270M Google $0.00
Devstral Medium Mistral $0.00
LFM2.5-8B-A1B Liquid AI $0.00
K2 Think V2 MBZUAI Institute of Foundation Models $0.00
Mistral Large 2 (Nov '24) Mistral $0.00
Qwen3 Max Thinking Alibaba $0.00
Doubao Seed Code ByteDance Seed $0.00
Gemini 2.5 Flash Preview (Sep '25) (Reasoning) Google $0.00
Qwen3.5 0.8B (Reasoning) Alibaba $0.00
Grok Code Fast 1 SpaceXAI $0.00
KAT-Coder-Pro V1 KwaiKAT $0.00
Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning) Google $0.00
Nanbeige4.1-3B Nanbeige $0.00
Gemini 2.0 Flash (Feb '25) Google $0.00
Magistral Small 1 Mistral $0.00
Olmo 3.1 32B Think Allen Institute for AI $0.00
ERNIE 5.0 Thinking Preview Baidu $0.00
Gemma 4 31B (Reasoning) Google $0.00
Devstral 2 Mistral $0.00
Qwen3.5 2B (Reasoning) Alibaba $0.00
Ring-1T InclusionAI $0.00
K2-V2 (low) MBZUAI Institute of Foundation Models $0.00
MiMo-V2-Flash (Non-reasoning) Xiaomi $0.00
North Mini Code Cohere $0.00
Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) NVIDIA $0.00
K2-V2 (medium) MBZUAI Institute of Foundation Models $0.00
Grok 4.1 Fast (Non-reasoning) SpaceXAI $0.00
INTELLECT-3 Prime Intellect $0.00
Olmo 3.1 32B Instruct Allen Institute for AI $0.00
Devstral Small (Jul '25) Mistral $0.00
EXAONE 4.5 33B LG AI Research $0.00
LFM2 2.6B Liquid AI $0.00
Nemotron Cascade 2 30B A3B NVIDIA $0.00
HyperCLOVA X SEED Think (32B) Naver $0.00
Solar Open 100B (Reasoning) Upstage $0.00
Llama 3.2 Instruct 1B Meta $0.00
Tri-21B-think Preview Trillion Labs $0.00
Ling-1T InclusionAI $0.00
Apriel-v1.5-15B-Thinker ServiceNow $0.00
Solar Pro 2 (Reasoning) Upstage $0.00
K2-V2 (high) MBZUAI Institute of Foundation Models $0.00
Devstral Small 2 Mistral $0.00
Devstral Small (May '25) Mistral $0.00
Mi:dm K 2.5 Pro Korea Telecom $0.00
K-EXAONE (Reasoning) LG AI Research $0.00
Llama 3.3 Nemotron Super 49B v1 (Reasoning) NVIDIA $0.00
Olmo 3 32B Think Allen Institute for AI $0.00
Apriel-v1.6-15B-Thinker ServiceNow $0.00
Step3 VL 10B StepFun $0.00
DiffusionGemma 26B A4B Google $0.00
LFM2 24B A2B Liquid AI $0.00
Qwen3 4B 2507 Instruct Alibaba $0.00
EXAONE 4.0 32B (Reasoning) LG AI Research $0.00
Qwen3 4B 2507 (Reasoning) Alibaba $0.00
Phi-4 Mini Instruct Microsoft $0.00
K-EXAONE (Non-reasoning) LG AI Research $0.00
Motif-2-12.7B-Reasoning Motif Technologies $0.00
Gemma 4 E2B (Non-reasoning) Google $0.00
Solar Pro 2 (Non-reasoning) Upstage $0.00
Mi:dm K 2.5 Pro Preview Korea Telecom $0.00
Falcon-H1R-7B TII UAE $0.00
Sarvam M (Reasoning) Sarvam $0.00
EXAONE 4.0 32B (Non-reasoning) LG AI Research $0.00
Tri-21B-Think Trillion Labs $0.00
DeepSeek R1 0528 Qwen3 8B DeepSeek $0.00
JT-MINI China Mobile $0.00
Gemma 3 27B Instruct Google $0.00
Granite 4.1 30B IBM $0.00
Molmo2-8B Allen Institute for AI $0.00
Qwen3 VL 4B (Reasoning) Alibaba $0.00
LongCat Flash Lite LongCat $0.00
Qwen3.5 9B (Non-reasoning) Alibaba $0.00
NVIDIA Nemotron 3 Nano 4B NVIDIA $0.00
LFM2 1.2B Liquid AI $0.00
Granite 4.0 350M IBM $0.00
Granite 4.0 H 1B IBM $0.00
Olmo 3 7B Think Allen Institute for AI $0.00
Jamba Reasoning 3B AI21 Labs $0.00
LFM2.5-1.2B-Instruct Liquid AI $0.00
Jamba 1.7 Mini AI21 Labs $0.00
LFM2 8B A1B Liquid AI $0.00
Qwen3 VL 4B Instruct Alibaba $0.00
Gemma 3 12B Instruct Google $0.00
Qwen3 1.7B (Reasoning) Alibaba $0.00
Gemma 3 1B Instruct Google $0.00
Granite 4.1 3B IBM $0.00
Granite 4.0 Micro IBM $0.00
Ling-mini-2.0 InclusionAI $0.00
Gemma 3n E2B Instruct Google $0.00
Qwen3 0.6B (Reasoning) Alibaba $0.00
Granite 4.0 1B IBM $0.00
Exaone 4.0 1.2B (Reasoning) LG AI Research $0.00
Exaone 4.0 1.2B (Non-reasoning) LG AI Research $0.00
Qwen3.5 2B (Non-reasoning) Alibaba $0.00
Gemma 3 4B Instruct Google $0.00
Qwen3 1.7B (Non-reasoning) Alibaba $0.00
LFM2.5-1.2B-Thinking Liquid AI $0.00
LFM2.5-VL-1.6B Liquid AI $0.00
MiniCPM-V 4.6 1.3B OpenBMB $0.00
Tiny Aya Global Cohere $0.00
Qwen3 0.6B (Non-reasoning) Alibaba $0.00
Granite 4.0 H 350M IBM $0.00
Qwen3.5 0.8B (Non-reasoning) Alibaba $0.00
Jamba 1.7 Large AI21 Labs $0.01
Gemma 4 E4B (Non-reasoning) Google $0.03
Nova Micro Amazon $0.04
Llama 3 Instruct 8B Meta $0.04
Granite 3.3 8B (Non-reasoning) IBM $0.04
Nova Lite Amazon $0.05
Gemma 3n E4B Instruct Google $0.05
NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) NVIDIA $0.06
GPT-5 nano (minimal) OpenAI $0.06
GLM-4.7-Flash (Non-reasoning) Z AI $0.07
Granite 4.1 8B IBM $0.07
Llama 3.1 Instruct 8B Meta $0.07
Granite 4.0 H Small IBM $0.08
Mistral Small 3.1 Mistral $0.10
Mistral Small 3.2 Mistral $0.10
GPT-4.1 nano OpenAI $0.10
Hy3-preview (Non-reasoning) Tencent $0.10
Apertus 8B Instruct Swiss AI Initiative $0.10
DeepSeek V4 Flash (Non-reasoning) DeepSeek $0.11
NVIDIA Nemotron Nano 9B V2 (Non-reasoning) NVIDIA $0.11
Ministral 3 3B Mistral $0.11
Ling 2.6 Flash InclusionAI $0.12
Olmo 3 7B Instruct Allen Institute for AI $0.12
Qwen3.5 Omni Flash Alibaba $0.13
Hermes 4 - Llama-3.1 70B (Non-reasoning) Nous Research $0.13
Mistral Small 4 (Non-reasoning) Mistral $0.14
Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning) Google $0.15
Ministral 3 8B Mistral $0.16
Ling-flash-2.0 InclusionAI $0.17
Gemini 2.5 Flash-Lite (Non-reasoning) Google $0.18
Gemma 4 12B (Non-reasoning) Google $0.19
Phi-4 Microsoft $0.20
Ministral 3 14B Mistral $0.20
Celeris-1 Celeris $0.21
Qwen3 VL 8B Instruct Alibaba $0.21
DeepSeek V3.2 Exp (Non-reasoning) DeepSeek $0.22
Grok 4 Fast (Non-reasoning) SpaceXAI $0.22
DeepSeek V3.2 (Non-reasoning) DeepSeek $0.23
Qwen3 8B (Non-reasoning) Alibaba $0.23
GPT-5.6 Luna (Non-reasoning) OpenAI $0.25
DeepSeek V3.1 Terminus (Non-reasoning) DeepSeek $0.25
Qwen3.5 4B (Non-reasoning) Alibaba $0.26
Gemma 4 E4B (Reasoning) Google $0.26
Mistral 7B Instruct Mistral $0.26
Llama 4 Scout Meta $0.27
DeepSeek V3 0324 DeepSeek $0.27
Claude 3 Haiku Anthropic $0.27
Qwen3 30B A3B (Non-reasoning) Alibaba $0.28
GPT-5 mini (minimal) OpenAI $0.29
Gemma 4 26B A4B (Non-reasoning) Google $0.31
Gemma 4 31B (Non-reasoning) Google $0.31
Qwen3 VL 8B (Reasoning) Alibaba $0.32
DeepSeek V3 (Dec '24) DeepSeek $0.33
Qwen3 Omni 30B A3B Instruct Alibaba $0.33
ERNIE 4.5 300B A47B Baidu $0.34
Llama 3.2 Instruct 11B (Vision) Meta $0.34
DeepSeek V4 Pro (Non-reasoning) DeepSeek $0.35
GLM-4.6V (Non-reasoning) Z AI $0.35
Llama 4 Maverick Meta $0.36
Qwen3 14B (Non-reasoning) Alibaba $0.37
Qwen2.5 Instruct 72B Alibaba $0.37
gpt-oss-20b (low) OpenAI $0.39
GPT-5.4 nano (Non-Reasoning) OpenAI $0.39
Qwen3 30B A3B 2507 Instruct Alibaba $0.39
Gemini 2.5 Flash (Non-reasoning) Google $0.40
Mistral Medium 3.1 Mistral $0.41
KAT Coder Pro V2 KwaiKAT $0.42
Magistral Small 1.2 Mistral $0.42
Mistral Medium 3 Mistral $0.43
GPT-4.1 mini OpenAI $0.44
Llama 3.1 Instruct 70B Meta $0.46
Gemini 3 Flash Preview (Non-reasoning) Google $0.46
Ling-2.6-1T InclusionAI $0.48
MiMo-V2.5-Pro (Non-reasoning) Xiaomi $0.48
NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning) NVIDIA $0.50
Mistral Large 3 Mistral $0.50
DeepSeek V3.1 (Non-reasoning) DeepSeek $0.50
Qwen3.6 35B A3B (Non-reasoning) Alibaba $0.50
Llama Nemotron Super 49B v1.5 (Non-reasoning) NVIDIA $0.53
Qwen3 Next 80B A3B Instruct Alibaba $0.54
Llama 3 Instruct 70B Meta $0.56
GLM-4.5V (Non-reasoning) Z AI $0.57
Qwen3.5 27B (Non-reasoning) Alibaba $0.58
Llama 3.3 Instruct 70B Meta $0.58
Kimi K2 Kimi $0.61
GLM-4.6 (Non-reasoning) Z AI $0.65
Kimi K2 0905 Kimi $0.65
Qwen3 Coder Next Alibaba $0.66
GPT-5.4 mini (Non-Reasoning) OpenAI $0.70
Qwen3.5 122B A10B (Non-reasoning) Alibaba $0.71
Qwen3 VL 235B A22B Instruct Alibaba $0.72
Nova Pro Amazon $0.73
Qwen3.5 35B A3B (Non-reasoning) Alibaba $0.73
Nova 2.0 Omni (Non-reasoning) Amazon $0.75
Qwen3.5 397B A17B (Non-reasoning) Alibaba $0.79
Qwen3 235B A22B 2507 Instruct Alibaba $0.88
GLM-4.7 (Non-reasoning) Z AI $0.93
NVIDIA Nemotron Nano 9B V2 (Reasoning) NVIDIA $1.01
Qwen3 VL 32B Instruct Alibaba $1.02
Qwen3 235B A22B (Non-reasoning) Alibaba $1.05
Qwen3.6 27B (Non-reasoning) Alibaba $1.14
Qwen3 Max (Preview) Alibaba $1.20
Qwen3 Max Alibaba $1.30
GPT-5 (minimal) OpenAI $1.34
GPT-5.1 (Non-reasoning) OpenAI $1.36
Qwen3.5 Omni Plus Alibaba $1.41
gpt-oss-120b (low) OpenAI $1.50
Qwen3 235B A22B (Reasoning) Alibaba $1.57
Hermes 4 - Llama-3.1 405B (Non-reasoning) Nous Research $1.58
Sarvam 30B (high) Sarvam $1.73
Llama 3.1 Nemotron Instruct 70B NVIDIA $1.74
Qwen3 VL 30B A3B Instruct Alibaba $1.79
Gemini 3.5 Flash (minimal) Google $1.88
Grok 4.3 (Non-reasoning) SpaceXAI $1.91
GLM-5 (Non-reasoning) Z AI $1.94
Nova 2.0 Lite (Non-reasoning) Amazon $1.98
GLM-5.1 (Non-reasoning) Z AI $2.03
DeepSeek V4 Flash (Reasoning, High Effort) DeepSeek $2.25
GPT-5.6 Terra (Non-reasoning) OpenAI $2.26
Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning) Google $2.37
GPT-5.4 (Non-reasoning) OpenAI $2.39
GPT-4.1 OpenAI $2.41
Sarvam 105B (high) Sarvam $2.42
Mistral Small 4 (Reasoning) Mistral $2.54
gpt-oss-20b (high) OpenAI $2.61
GPT-5.2 (Non-reasoning) OpenAI $2.77
GLM-5.2 (Non-reasoning) Z AI $2.83
GPT-4o (Aug '24) OpenAI $2.91
Qwen3 Coder 480B A35B Instruct Alibaba $2.94
GPT-5.6 Luna (low) OpenAI $2.95
Grok 4.20 0309 v2 (Non-reasoning) SpaceXAI $2.97
GPT-4o (Nov '24) OpenAI $2.98
Hermes 4 - Llama-3.1 70B (Reasoning) Nous Research $3.10
MiMo-V2.5 Xiaomi $3.16
GPT-5 nano (medium) OpenAI $3.51
Claude 4.5 Haiku (Non-reasoning) Anthropic $3.69
Apertus 70B Instruct Swiss AI Initiative $3.78
GPT-5.4 nano (medium) OpenAI $3.80
Gemma 4 12B (Reasoning) Google $3.93
Grok 4 Fast (Reasoning) SpaceXAI $3.96
DeepSeek V3.2 Exp (Reasoning) DeepSeek $4.08
Grok 4.20 0309 (Non-reasoning) SpaceXAI $4.09
Grok 3 mini Reasoning (high) SpaceXAI $4.11
Llama 3.1 Instruct 405B Meta $4.16
Grok 3 SpaceXAI $4.31
Command A Cohere $4.39
Nova Premier Amazon $4.44
Claude Sonnet 4.6 (Non-reasoning, Low Effort) Anthropic $4.45
Claude 3.7 Sonnet (Non-reasoning) Anthropic $4.46
DeepSeek R1 Distill Llama 70B DeepSeek $4.50
DeepSeek V3.2 (Reasoning) DeepSeek $4.63
Kimi K2.5 (Non-reasoning) Kimi $4.69
Reka Flash 3 Reka AI $4.71
Hy3-preview (Reasoning) Tencent $4.88
GPT-5.6 Luna (medium) OpenAI $4.90
Seed-OSS-36B-Instruct ByteDance Seed $5.32
Ring-flash-2.0 InclusionAI $5.32
Cogito v2.1 (Reasoning) Deep Cogito $5.42
Llama Nemotron Super 49B v1.5 (Reasoning) NVIDIA $5.48
MiMo-V2-Flash (Reasoning) Xiaomi $5.53
GPT-5.5 (Non-reasoning) OpenAI $5.57
GPT-5.6 Sol (Non-reasoning) OpenAI $5.79
Kimi K2.6 (Non-reasoning) Kimi $5.83
DeepSeek V3.1 (Reasoning) DeepSeek $6.01
NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) NVIDIA $6.03
Nova 2.0 Pro Preview (Non-reasoning) Amazon $6.25
Claude Sonnet 5 (Non-reasoning, High Effort) Anthropic $6.53
Nemotron 3 Nano Omni 30B A3B Reasoning NVIDIA $6.95
NVIDIA Nemotron Nano 12B v2 VL (Reasoning) NVIDIA $7.00
gpt-oss-120b (high) OpenAI $7.00
Claude 4.5 Sonnet (Non-reasoning) Anthropic $7.17
GLM-4.7-Flash (Reasoning) Z AI $7.22
Claude 4 Sonnet (Non-reasoning) Anthropic $7.42
Gemma 4 26B A4B (Reasoning) Google $7.44
Solar Pro 3 Upstage $7.87
HyperNova 60B 2605 Multiverse Computing $7.96
Gemini 2.5 Flash-Lite (Reasoning) Google $7.97
Qwen3 Omni 30B A3B (Reasoning) Alibaba $8.10
Claude Sonnet 4.6 (Non-reasoning, High Effort) Anthropic $8.42
GPT-5 mini (medium) OpenAI $8.52
Gemini 3.1 Flash-Lite Google $9.35
GPT-5 nano (high) OpenAI $9.53
GLM-4.6V (Reasoning) Z AI $9.56
Grok 4.3 (low) SpaceXAI $9.93
GLM-4.5-Air Z AI $10.02
Nova 2.0 Omni (low) Amazon $10.40
GLM-4.5V (Reasoning) Z AI $10.81
Step 3.5 Flash StepFun $10.95
NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA $11.06
Claude Opus 4.5 (Non-reasoning) Anthropic $11.15
DeepSeek V4 Pro (Reasoning, High Effort) DeepSeek $11.15
Gemini 3 Pro Preview (low) Google $11.18
Claude Opus 4.7 (Non-reasoning, High Effort) Anthropic $11.28
Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) NVIDIA $11.32
DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek $11.66
Nova 2.0 Lite (low) Amazon $11.68
MiniMax-M2.5 MiniMax $12.44
Qwen3 Coder 30B A3B Instruct Alibaba $12.55
Qwen3 VL 30B A3B (Reasoning) Alibaba $13.13
MiniMax-M2.1 MiniMax $13.99
Qwen3 30B A3B (Reasoning) Alibaba $14.86
GPT-5.6 Luna (high) OpenAI $15.00
Claude Opus 4.6 (Non-reasoning, High Effort) Anthropic $16.24
Step 3.5 Flash 2603 StepFun $16.36
Qwen3 30B A3B 2507 (Reasoning) Alibaba $16.45
Qwen3.5 9B (Reasoning) Alibaba $16.54
Gemini 2.5 Flash (Reasoning) Google $16.65
MiMo-V2.5-Pro Xiaomi $16.94
GPT-5.6 Terra (low) OpenAI $17.07
Grok 4.3 (medium) SpaceXAI $17.37
MiniMax-M2.7 MiniMax $17.39
Qwen3.7 Plus Alibaba $17.41
Gemini 3.5 Flash-Lite Google $18.60
MiniMax-M2 MiniMax $18.88
Qwen3.6 35B A3B (Reasoning) Alibaba $19.32
Qwen3 8B (Reasoning) Alibaba $19.74
Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA $20.21
Qwen3 14B (Reasoning) Alibaba $20.63
MiniMax-M3 MiniMax $20.97
Hy3 Tencent $21.03
Mercury 2 Inception $21.20
Hermes 4 - Llama-3.1 405B (Reasoning) Nous Research $21.31
DeepSeek R1 0528 (May '25) DeepSeek $21.42
Agnes 2.5 Pro Alpha Sapiens AI $21.90
Nova 2.0 Omni (medium) Amazon $22.25
Qwen3.5 4B (Reasoning) Alibaba $22.55
Grok 4.20 0309 v2 (Reasoning) SpaceXAI $23.08
Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic $23.80
DeepSeek V4 Flash 0731 (Reasoning, Max Effort) DeepSeek $25.49
DeepSeek V3.1 Terminus (Reasoning) DeepSeek $25.82
DeepSeek R1 (Jan '25) DeepSeek $26.31
Magistral Medium 1.2 Mistral $26.35
Nova 2.0 Lite (medium) Amazon $26.63
GPT-5.6 Terra (medium) OpenAI $27.98
GPT-5.5 Instant (June 2026) OpenAI $28.32
Ring-2.6-1T InclusionAI $29.11
GLM-4.6 (Reasoning) Z AI $29.37
GPT-5.6 Luna (xhigh) OpenAI $30.10
Kimi K3 (low) Kimi $30.19
MiniMax M1 80k MiniMax $30.80
GPT-5.4 (low) OpenAI $31.18
GPT-5 (low) OpenAI $31.36
Trinity Large Thinking Arcee AI $32.46
Gemini 3 Flash Preview (Reasoning) Google $32.49
GPT-5 mini (high) OpenAI $33.35
DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek $33.42
GPT-5.2 (medium) OpenAI $35.13
Inkling Small Thinking Machines $35.91
Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic $37.02
Grok 4.5 (high) SpaceXAI $38.73
GPT-5.5 Instant (May 2026) OpenAI $41.05
Kimi K2 Thinking Kimi $42.45
GPT-5.4 mini (medium) OpenAI $43.17
Grok 4.3 (high) SpaceXAI $44.80
Nova 2.0 Pro Preview (low) Amazon $45.80
Qwen3.6 Plus Alibaba $46.52
Qwen3 VL 235B A22B (Reasoning) Alibaba $46.65
Step 3.7 Flash StepFun $49.12
Qwen3 32B (Reasoning) Alibaba $49.54
GLM-5 (Reasoning) Z AI $49.98
GLM-5.2 (max) Z AI $50.24
GPT-5.6 Sol (low) OpenAI $52.24
Qwen3 Next 80B A3B (Reasoning) Alibaba $53.05
Qwen3.5 35B A3B (Reasoning) Alibaba $54.52
GPT-5.1 Codex mini (high) OpenAI $54.92
Grok 4.20 0309 (Reasoning) SpaceXAI $55.27
Kimi K2.5 (Reasoning) Kimi $56.34
Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic $56.40
GPT-5.6 Terra (high) OpenAI $58.49
GPT-5.4 nano (xhigh) OpenAI $59.69
Qwen3 Max Thinking (Preview) Alibaba $61.20
GLM-5.1 (Reasoning) Z AI $61.57
Gemini 3.6 Flash (high) Google $62.20
Nex-N2-Pro Nex AGI $65.57
GPT-5.6 Luna (max) OpenAI $66.02
GPT-5.5 (low) OpenAI $66.47
Qwen3.5 27B (Reasoning) Alibaba $66.76
Gemini 3.5 Flash (medium) Google $67.48
Qwen3 235B A22B 2507 (Reasoning) Alibaba $73.06
Claude 4.5 Haiku (Reasoning) Anthropic $74.24
o3-mini (high) OpenAI $80.10
Qwen3.5 122B A10B (Reasoning) Alibaba $81.80
Grok Build 0.1 0616 SpaceXAI $82.76
Qwen3.7 Max Alibaba $82.77
Muse Spark 1.1 (xhigh) Meta $85.57
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic $85.74
Gemini 3.5 Flash (high) Google $86.50
Mistral Medium 3.5 Mistral $86.92
Qwen3.6 Max Preview Alibaba $89.04
GPT-5.6 Sol (medium) OpenAI $91.49
Qwen3.5 397B A17B (Reasoning) Alibaba $92.06
Nova 2.0 Pro Preview (medium) Amazon $92.17
GPT-5.6 Terra (xhigh) OpenAI $97.18
GPT-5 (medium) OpenAI $98.95
o3 OpenAI $100.85
Gemini 2.5 Pro Google $103.61
Gemini 3.1 Pro Preview Google $106.03
Claude 4 Sonnet (Reasoning) Anthropic $115.31
GPT-5 Codex (high) OpenAI $119.78
Inkling (xhigh) Thinking Machines $126.58
Kimi K2.7 Code Kimi $127.39
GPT-5.1 Codex (high) OpenAI $133.98
o4-mini (high) OpenAI $138.89
LongCat 2.0 LongCat $141.31
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic $147.97
GPT-5.1 (high) OpenAI $148.42
Nova 2.0 Lite (high) Amazon $152.21
Kimi K2.6 Kimi $154.20
Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic $155.28
GPT-5.6 Sol (high) OpenAI $161.36
Gemini 3 Pro Preview (high) Google $168.06
Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic $174.17
Claude 4.5 Sonnet (Reasoning) Anthropic $176.46
GLM-4.7 (Reasoning) Z AI $183.36
Claude Opus 4.5 (Reasoning) Anthropic $198.37
Qwen3.6 27B (Reasoning) Alibaba $204.06
Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic $213.19
GPT-5 (high) OpenAI $214.02
Qwen3 VL 32B (Reasoning) Alibaba $218.66
GPT-5.5 (medium) OpenAI $223.82
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic $263.99
Grok 4 SpaceXAI $277.37
GPT-5.6 Sol (xhigh) OpenAI $297.27
Claude Opus 4.6 (Adaptive Reasoning, Max Effort) Anthropic $298.23
GPT-5.4 mini (xhigh) OpenAI $342.67
GPT-5.3 Codex (xhigh) OpenAI $420.78
GPT-5.6 Terra (max) OpenAI $422.02
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic $436.96
GPT-5.5 (high) OpenAI $515.28
o1 OpenAI $577.85
GPT-5.2 (xhigh) OpenAI $607.24
GPT-5.4 (xhigh) OpenAI $654.08
GPT-5.6 Sol (max) OpenAI $767.80
Kimi K3 (max) Kimi $796.95
GPT-5.5 (xhigh) OpenAI $909.41
GPT-5.2 Codex (xhigh) OpenAI $1326.26

GDPval-AA v2

GDPval-AA v2 evaluates AI models on real-world, economically valuable tasks across a wide range of occupations. Elo ratings are anchored to a human baseline of 1,000. Higher is better.

Table 9: GDPval-AA v2 Full Leaderboard

Rank Model Creator Elo CI Release Date
1 Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic 1852 ±24 Jul 2026
2 Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 1819 ±23 Jul 2026
3 Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic 1743 ±17 Jun 2026
4 Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic 1735 ±21 Jul 2026
5 GPT-5.6 Sol (max) OpenAI 1730 ±17 Jul 2026
6 Kimi K3 (max) Kimi 1685 ±21 Jul 2026
7 GPT-5.6 Sol (xhigh) OpenAI 1683 ±17 Jul 2026
8 Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic 1628 ±20 Jul 2026
9 GPT-5.6 Sol (high) OpenAI 1623 ±17 Jul 2026
10 Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic 1600 ±16 Jun 2026
11 Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic 1588 ±15 May 2026
12 GPT-5.6 Luna (max) OpenAI 1578 ±17 Jul 2026
13 GPT-5.6 Terra (max) OpenAI 1577 ±17 Jul 2026
14 GPT-5.6 Terra (xhigh) OpenAI 1571 ±17 Jul 2026
15 DeepSeek V4 Flash 0731 (Reasoning, Max Effort) DeepSeek 1558 ±20 Jul 2026
16 GPT-5.6 Sol (medium) OpenAI 1553 ±16 Jul 2026
17 GPT-5.6 Luna (xhigh) OpenAI 1527 ±17 Jul 2026
18 Grok 4.5 (high) SpaceXAI 1526 ±19 Jul 2026
19 GPT-5.6 Terra (high) OpenAI 1510 ±17 Jul 2026
20 GLM-5.2 (max) Z AI 1508 ±15 Jun 2026
21 Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 1505 ±17 Jun 2026
22 Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic 1490 ±15 Apr 2026
23 GPT-5.5 (xhigh) OpenAI 1490 ±15 Apr 2026
24 GPT-5.6 Luna (high) OpenAI 1467 ±17 Jul 2026
25 GPT-5.5 (high) OpenAI 1466 ±15 Apr 2026
26 Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic 1454 ±19 Jul 2026
27 GPT-5.6 Sol (low) OpenAI 1441 ±16 Jul 2026
28 Gemini 3.6 Flash (high) Google 1423 ±17 Jul 2026
29 GPT-5.6 Terra (medium) OpenAI 1404 ±16 Jul 2026
30 Claude Sonnet 5 (Adaptive Reasoning, High Effort) Anthropic 1400 ±16 Jun 2026
31 GPT-5.4 (xhigh) OpenAI 1393 ±15 Mar 2026
32 GLM-5.2 (Non-reasoning) Z AI 1391 ±20 Jun 2026
33 MiniMax-M3 MiniMax 1389 ±15 Jun 2026
34 GPT-5.6 Sol (Non-reasoning) OpenAI 1378 ±17 Jul 2026
35 Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic 1375 ±15 Feb 2026
36 GPT-5.5 (medium) OpenAI 1373 ±16 Apr 2026
37 Claude Sonnet 5 (Non-reasoning, High Effort) Anthropic 1371 ±17 Jun 2026
38 Muse Spark 1.1 (xhigh) Meta 1371 ±18 Jul 2026
39 Gemini 3.5 Flash (high) Google 1343 ±15 May 2026
40 DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek 1304 ±15 Apr 2026
41 Claude Sonnet 5 (Adaptive Reasoning, Medium Effort) Anthropic 1303 ±16 Jun 2026
42 DeepSeek V4 Pro (Reasoning, High Effort) DeepSeek 1292 ±20 Apr 2026
43 GPT-5.6 Luna (medium) OpenAI 1275 ±17 Jul 2026
44 Qwen3.7 Max Alibaba 1271 ±15 May 2026
45 Kimi K3 (low) Kimi 1269 ±20 Jul 2026
46 Inkling Small Thinking Machines 1268 ±20 Jul 2026
47 MiMo-V2.5-Pro Xiaomi 1264 ±15 Apr 2026
48 JT-4.1 Flash 236B A21B China Mobile 1263 ±19 Jul 2026
49 Motif 3 (Beta) Motif Technologies 1257 ±19 Jul 2026
50 GLM-5.1 (Reasoning) Z AI 1257 ±15 Apr 2026
51 GPT-5.6 Terra (low) OpenAI 1254 ±16 Jul 2026
52 Nex-N2-Pro Nex AGI 1249 ±17 Jun 2026
53 GPT-5.6 Terra (Non-reasoning) OpenAI 1243 ±17 Jul 2026
54 Inkling (xhigh) Thinking Machines 1237 ±19 Jul 2026
55 Claude Sonnet 5 (Adaptive Reasoning, Low Effort) Anthropic 1217 ±16 Jun 2026
56 Hy3 Tencent 1215 ±19 Jul 2026
57 Grok Build 0.1 0616 SpaceXAI 1213 ±16 Jun 2026
58 G9v3-39A5B AI9Stars 1194 ±20 Aug 2026
59 DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek 1190 ±16 Apr 2026
60 Kimi K2.6 Kimi 1189 ±15 Apr 2026
61 Kimi K2.7 Code Kimi 1188 ±16 Jun 2026
62 GPT-5.5 (low) OpenAI 1187 ±16 Apr 2026
63 Agnes 2.5 Pro Alpha Sapiens AI 1172 ±20 Jul 2026
64 GPT-5.4 mini (xhigh) OpenAI 1169 ±15 Mar 2026
65 GLM-4.7 (Reasoning) Z AI 1167 ±17 Dec 2025
66 Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA 1162 ±15 Jun 2026
67 MiniMax-M2.7 MiniMax 1158 ±15 Mar 2026
68 GPT-5.6 Luna (low) OpenAI 1154 ±17 Jul 2026
69 DeepSeek V4 Flash (Reasoning, High Effort) DeepSeek 1151 ±20 Apr 2026
70 MiMo-V2.5 Xiaomi 1148 ±20 Apr 2026
71 Muse Spark Meta 1144 ±15 Apr 2026
72 Qwen3.6 27B (Reasoning) Alibaba 1140 ±15 Apr 2026
73 Qwen3.6 Plus Alibaba 1138 ±15 Apr 2026
74 Gemini 3.5 Flash-Lite Google 1137 ±19 Jul 2026
75 GPT-5.5 (Non-reasoning) OpenAI 1123 ±15 Apr 2026
76 Qwen3.6 27B (Non-reasoning) Alibaba 1111 ±18 Apr 2026
77 GPT-5.4 nano (xhigh) OpenAI 1102 ±15 Mar 2026
78 Grok 4.3 (Non-reasoning) SpaceXAI 1097 ±15 Apr 2026
79 Grok 4.3 (high) SpaceXAI 1085 ±15 Apr 2026
80 GPT-5 (high) OpenAI 1082 ±18 Aug 2025
81 GPT-5.6 Luna (Non-reasoning) OpenAI 1073 ±17 Jul 2026
82 Qwen3.6 35B A3B (Reasoning) Alibaba 1053 ±16 Apr 2026
83 Claude 4.5 Sonnet (Reasoning) Anthropic 1051 ±18 Sep 2025
84 LongCat 2.0 LongCat 1029 ±18 Jun 2026
85 Qwen3.6 35B A3B (Non-reasoning) Alibaba 1020 ±20 Apr 2026
86 Step 3.7 Flash StepFun 1016 ±16 May 2026
87 Kimi K2.5 (Reasoning) Kimi 1003 ±19 Jan 2026
88 GPT-5.1 (high) OpenAI 990 ±18 Nov 2025
89 Qwen3.5 122B A10B (Reasoning) Alibaba 985 ±16 Feb 2026
90 Gemini 3.1 Pro Preview Google 963 ±16 Feb 2026
91 Qwen3.5 397B A17B (Reasoning) Alibaba 961 ±16 Feb 2026
92 Qwen3.7 Plus Alibaba 943 ±16 Jun 2026
93 GPT-5 mini (high) OpenAI 933 ±18 Aug 2025
94 GLM-4.6 (Reasoning) Z AI 932 ±18 Sep 2025
95 Mistral Medium 3.5 Mistral 931 ±16 Apr 2026
96 Ring-2.6-1T InclusionAI 921 ±16 May 2026
97 Claude 4.5 Haiku (Reasoning) Anthropic 911 ±16 Oct 2025
98 KAT-Coder-Pro V1 KwaiKAT 902 ±17 Nov 2025
99 KAT Coder Pro V2 KwaiKAT 902 ±20 Mar 2026
100 Qwen3.5 122B A10B (Non-reasoning) Alibaba 887 ±19 Feb 2026
101 DeepSeek V3.1 Terminus (Reasoning) DeepSeek 880 ±20 Sep 2025
102 Claude 4 Sonnet (Reasoning) Anthropic 871 ±18 May 2025
103 G9v3-3B AI9Stars 863 ±27 Jul 2026
104 DeepSeek V3.2 (Reasoning) DeepSeek 862 ±21 Dec 2025
105 MiMo-V2-Flash (Non-reasoning) Xiaomi 838 ±20 Dec 2025
106 Gemma 4 31B (Reasoning) Google 807 ±16 Apr 2026
107 gpt-oss-120b (high) OpenAI 798 ±16 Aug 2025
108 Qwen3.5 35B A3B (Non-reasoning) Alibaba 792 ±22 Feb 2026
109 GPT-5.4 mini (Non-Reasoning) OpenAI 786 ±17 Mar 2026
110 Gemma 4 26B A4B (Reasoning) Google 767 ±16 Apr 2026
111 Gemma 4 31B (Non-reasoning) Google 747 ±20 Apr 2026
112 Devstral 2 Mistral 741 ±18 Dec 2025
113 Devstral Small 2 Mistral 729 ±17 Dec 2025
114 Qwen3 Coder Next Alibaba 715 ±19 Feb 2026
115 GPT-5.5 Instant (June 2026) OpenAI 714 ±18 Jun 2026
116 Command A+ Cohere 713 ±19 May 2026
117 NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA 697 ±16 Mar 2026
118 Mercury 2 Inception 694 ±20 Feb 2026
119 EXAONE 4.5 33B LG AI Research 680 ±21 Apr 2026
120 Nova 2.0 Pro Preview (medium) Amazon 677 ±17 Nov 2025
121 Gemini 2.5 Pro Google 667 ±18 Jun 2025
122 HyperNova 60B 2605 Multiverse Computing 656 ±20 May 2026
123 Nova 2.0 Pro Preview (low) Amazon 651 ±17 Nov 2025
124 Gemini 3.1 Flash-Lite Google 645 ±17 Mar 2026
125 Gemma 4 12B (Reasoning) Google 643 ±21 Jun 2026
126 Qwen3.5 9B (Reasoning) Alibaba 643 ±19 Mar 2026
127 Mistral Large 3 Mistral 638 ±17 Dec 2025
128 Mistral Medium 3.1 Mistral 608 ±19 Aug 2025
129 Mistral Small 3.1 Mistral 601 ±18 Mar 2025
130 K-EXAONE (Reasoning) LG AI Research 590 ±21 Dec 2025
131 Mistral Small 4 (Reasoning) Mistral 588 ±20 Mar 2026
132 Nova 2.0 Lite (high) Amazon 587 ±18 Oct 2025
133 gpt-oss-20b (high) OpenAI 562 ±17 Aug 2025
134 Trinity Large Thinking Arcee AI 560 ±21 Apr 2026
135 Nova 2.0 Pro Preview (Non-reasoning) Amazon 559 ±17 Nov 2025
136 DiffusionGemma 26B A4B Google 552 ±18 Jun 2026
137 Ling 2.6 Flash InclusionAI 544 ±20 Apr 2026
138 Qwen3 235B A22B 2507 (Reasoning) Alibaba 541 ±20 Jul 2025
139 North Mini Code Cohere 540 ±19 Jun 2026
140 Celeris-1 Celeris 533 ±22 Jul 2026
141 DeepSeek R1 (Jan '25) DeepSeek 527 ±19 Jan 2025
142 GPT-4.1 mini OpenAI 503 ±19 Apr 2025
143 Nemotron Cascade 2 30B A3B NVIDIA 499 ±22 Mar 2026
144 Solar Pro 3 Upstage 496 ±17 Apr 2026
145 NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) NVIDIA 487 ±17 Dec 2025
146 Ministral 3 14B Mistral 483 ±18 Dec 2025
147 o3-mini (high) OpenAI 468 ±20 Jan 2025
148 Nemotron 3 Nano Omni 30B A3B Reasoning NVIDIA 463 ±22 Apr 2026
149 Claude 3.5 Haiku Anthropic 455 ±18 Oct 2024
150 Ministral 3 8B Mistral 451 ±18 Dec 2025
151 Granite 4.1 30B IBM 429 ±17 Apr 2026
152 Magistral Medium 1.2 Mistral 412 ±19 Sep 2025
153 gpt-oss-120b (low) OpenAI 409 ±22 Aug 2025
154 K2 Think V2 MBZUAI Institute of Foundation Models 377 ±19 Dec 2025
155 Qwen3 Next 80B A3B (Reasoning) Alibaba 373 ±21 Sep 2025
156 DeepSeek V3 0324 DeepSeek 323 ±20 Mar 2025
157 Qwen3 30B A3B 2507 (Reasoning) Alibaba 319 ±21 Jul 2025
158 Qwen3 32B (Reasoning) Alibaba 287 ±19 Apr 2025
159 Ministral 3 3B Mistral 284 ±18 Dec 2025
160 Magistral Small 1.2 Mistral 261 ±19 Sep 2025
161 GPT-4o mini OpenAI 237 ±20 Jul 2024
162 GPT-4 OpenAI 237 ±19 Mar 2023
163 Qwen3 14B (Reasoning) Alibaba 233 ±19 Apr 2025
164 DeepSeek V3 (Dec '24) DeepSeek 229 ±20 Dec 2024
165 Gemma 4 E4B (Reasoning) Google 228 ±23 Apr 2026
166 Qwen3.5 2B (Reasoning) Alibaba 216 ±22 Mar 2026
167 Qwen3 8B (Reasoning) Alibaba 213 ±20 Apr 2025
168 NVIDIA Nemotron 3 Nano 4B NVIDIA 201 ±22 Mar 2026
169 Granite 4.1 3B IBM 124 ±21 Apr 2026
170 Llama 4 Scout Meta 109 ±18 Apr 2025
171 Llama 3.3 Instruct 70B Meta 96 ±20 Dec 2024
172 Gemma 4 E2B (Reasoning) Google 85 ±22 Apr 2026
173 Mistral Small 3.2 Mistral 76 ±20 Jun 2025
174 GPT-4.1 nano OpenAI 60 ±20 Apr 2025
175 Llama 4 Maverick Meta 4 ±18 Apr 2025
176 NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) NVIDIA -73 ±18 Dec 2025
177 Qwen3.5 2B (Non-reasoning) Alibaba -81 ±19 Mar 2026
178 Qwen3.5 0.8B (Non-reasoning) Alibaba -83 ±19 Mar 2026
179 MiniCPM-V 4.6 1.3B OpenBMB -86 ±17 May 2026
180 Llama 3.1 Instruct 8B Meta -102 ±17 Jul 2024
181 Qwen3.5 0.8B (Reasoning) Alibaba -103 ±18 Mar 2026
182 Nanbeige4.1-3B Nanbeige -117 ±18 Feb 2026
183 Gemma 3 12B Instruct Google -122 ±18 Mar 2025
184 Gemma 3 27B Instruct Google -122 ±17 Mar 2025
185 Phi-4 Mini Instruct Microsoft -123 ±18 Feb 2024

Table 10: GDPval-AA v2 Cost per Task

Calculated from page evaluation-cost records across 220 GDPval tasks, including source cache pricing.

Model Creator Cost/Task
Gemma 4 31B (Reasoning) Google $0.00
Devstral 2 Mistral $0.00
Devstral Small 2 Mistral $0.00
Phi-4 Mini Instruct Microsoft $0.00
G9v3-3B AI9Stars $0.00
North Mini Code Cohere $0.00
Command A+ Cohere $0.00
Gemma 3 27B Instruct Google $0.00
Gemma 3 12B Instruct Google $0.00
GPT-5.6 Luna (low) OpenAI $0.01
GPT-5.6 Luna (Non-reasoning) OpenAI $0.01
DeepSeek V3 (Dec '24) DeepSeek $0.01
MiMo-V2.5 Xiaomi $0.01
GPT-5.6 Luna (medium) OpenAI $0.01
Gemini 3.1 Flash-Lite Google $0.02
Llama 4 Scout Meta $0.02
DeepSeek V3 0324 DeepSeek $0.02
Mistral Small 4 (Reasoning) Mistral $0.02
GPT-5.4 mini (Non-Reasoning) OpenAI $0.02
Mistral Small 3.1 Mistral $0.03
NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) NVIDIA $0.03
Hy3 Tencent $0.04
MiMo-V2.5-Pro Xiaomi $0.04
GPT-5.6 Luna (high) OpenAI $0.04
DeepSeek V4 Flash (Reasoning, High Effort) DeepSeek $0.04
GPT-5 mini (high) OpenAI $0.05
Gemma 4 31B (Non-reasoning) Google $0.05
DeepSeek V4 Pro (Reasoning, High Effort) DeepSeek $0.05
Gemma 4 E4B (Reasoning) Google $0.05
Mercury 2 Inception $0.05
DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek $0.05
DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek $0.06
HyperNova 60B 2605 Multiverse Computing $0.07
GPT-5.6 Luna (xhigh) OpenAI $0.07
Qwen3 8B (Reasoning) Alibaba $0.07
GPT-5.5 Instant (June 2026) OpenAI $0.07
Ministral 3 8B Mistral $0.07
Nemotron 3 Nano Omni 30B A3B Reasoning NVIDIA $0.07
DeepSeek V4 Flash 0731 (Reasoning, Max Effort) DeepSeek $0.07
Step 3.7 Flash StepFun $0.07
Qwen3 14B (Reasoning) Alibaba $0.08
Gemma 4 26B A4B (Reasoning) Google $0.08
gpt-oss-120b (low) OpenAI $0.08
gpt-oss-20b (high) OpenAI $0.08
Inkling Small Thinking Machines $0.09
GPT-5.6 Terra (low) OpenAI $0.09
Llama 4 Maverick Meta $0.10
Nova 2.0 Lite (high) Amazon $0.10
Grok 4.3 (high) SpaceXAI $0.10
GPT-5.6 Luna (max) OpenAI $0.10
Gemini 3.5 Flash-Lite Google $0.10
gpt-oss-120b (high) OpenAI $0.11
Mistral Large 3 Mistral $0.11
Kimi K2.5 (Reasoning) Kimi $0.11
Nova 2.0 Pro Preview (low) Amazon $0.11
GPT-5.6 Terra (Non-reasoning) OpenAI $0.13
Qwen3.7 Plus Alibaba $0.14
Grok Build 0.1 0616 SpaceXAI $0.14
GPT-4.1 nano OpenAI $0.14
GPT-5.5 (Non-reasoning) OpenAI $0.15
Ministral 3 3B Mistral $0.15
Grok 4.3 (Non-reasoning) SpaceXAI $0.17
Qwen3 30B A3B 2507 (Reasoning) Alibaba $0.17
Nova 2.0 Pro Preview (medium) Amazon $0.18
GPT-4.1 mini OpenAI $0.18
GPT-5.6 Terra (medium) OpenAI $0.18
Nova 2.0 Pro Preview (Non-reasoning) Amazon $0.19
Claude 4.5 Haiku (Reasoning) Anthropic $0.19
GPT-5.5 (low) OpenAI $0.19
GPT-4o mini OpenAI $0.19
Solar Pro 3 Upstage $0.20
Kimi K3 (low) Kimi $0.21
Llama 3.3 Instruct 70B Meta $0.21
LongCat 2.0 LongCat $0.21
Qwen3.5 122B A10B (Non-reasoning) Alibaba $0.22
Mistral Medium 3.1 Mistral $0.22
Qwen3 32B (Reasoning) Alibaba $0.23
GLM-4.6 (Reasoning) Z AI $0.24
Gemini 2.5 Pro Google $0.24
Ministral 3 14B Mistral $0.24
Qwen3.5 35B A3B (Non-reasoning) Alibaba $0.24
Qwen3.5 122B A10B (Reasoning) Alibaba $0.24
GPT-5.4 nano (xhigh) OpenAI $0.24
Gemma 4 12B (Reasoning) Google $0.26
Claude Sonnet 5 (Adaptive Reasoning, Low Effort) Anthropic $0.26
Gemini 3.1 Pro Preview Google $0.27
Qwen3.6 35B A3B (Reasoning) Alibaba $0.28
Llama 3.1 Instruct 8B Meta $0.28
Qwen3 Coder Next Alibaba $0.29
Ring-2.6-1T InclusionAI $0.29
Muse Spark 1.1 (xhigh) Meta $0.29
NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) NVIDIA $0.29
GPT-5.6 Sol (Non-reasoning) OpenAI $0.32
GPT-5.6 Sol (low) OpenAI $0.33
GPT-5.1 (high) OpenAI $0.33
Qwen3 Next 80B A3B (Reasoning) Alibaba $0.34
MiniMax-M3 MiniMax $0.35
GPT-5 (high) OpenAI $0.35
Qwen3 235B A22B 2507 (Reasoning) Alibaba $0.36
NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA $0.37
Qwen3.5 397B A17B (Reasoning) Alibaba $0.37
Trinity Large Thinking Arcee AI $0.39
Kimi K2.7 Code Kimi $0.40
Mistral Medium 3.5 Mistral $0.41
Qwen3.6 27B (Reasoning) Alibaba $0.44
GLM-5.1 (Reasoning) Z AI $0.45
Mistral Small 3.2 Mistral $0.45
Grok 4.5 (high) SpaceXAI $0.47
GPT-5.6 Terra (high) OpenAI $0.49
Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA $0.51
Claude 4.5 Sonnet (Reasoning) Anthropic $0.51
Qwen3.6 Plus Alibaba $0.51
Claude Sonnet 5 (Adaptive Reasoning, Medium Effort) Anthropic $0.53
GLM-4.7 (Reasoning) Z AI $0.53
Kimi K2.6 Kimi $0.55
Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic $0.58
GPT-5.4 mini (xhigh) OpenAI $0.59
Qwen3.5 9B (Reasoning) Alibaba $0.62
Qwen3.6 27B (Non-reasoning) Alibaba $0.62
GPT-5.6 Sol (medium) OpenAI $0.74
GPT-5.6 Terra (xhigh) OpenAI $0.74
GPT-5.5 (medium) OpenAI $0.77
Magistral Small 1.2 Mistral $0.86
DeepSeek R1 (Jan '25) DeepSeek $0.88
Qwen3.6 35B A3B (Non-reasoning) Alibaba $0.97
Claude Sonnet 5 (Non-reasoning, High Effort) Anthropic $1.02
Claude Sonnet 5 (Adaptive Reasoning, High Effort) Anthropic $1.02
Gemini 3.6 Flash (high) Google $1.04
Qwen3.7 Max Alibaba $1.05
GPT-5.6 Terra (max) OpenAI $1.15
GLM-5.2 (max) Z AI $1.30
GPT-5.5 (high) OpenAI $1.35
Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic $1.36
GPT-5.6 Sol (high) OpenAI $1.37
Gemini 3.5 Flash (high) Google $1.74
Magistral Medium 1.2 Mistral $1.81
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic $1.94
GPT-5.5 (xhigh) OpenAI $2.10
GPT-5.6 Sol (xhigh) OpenAI $2.11
GPT-5.4 (xhigh) OpenAI $2.14
Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort) Anthropic $2.14
Kimi K3 (max) Kimi $2.15
GPT-4 OpenAI $2.68
Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic $3.03
GPT-5.6 Sol (max) OpenAI $3.49
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic $4.34
Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic $4.70
Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic $4.90
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic $4.96
Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic $6.77
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic $7.75

Table 11: GDPval-AA v2 Average Turns per Task

Model Creator Turns
DeepSeek V3 (Dec '24) DeepSeek 3
GPT-5.5 Instant (June 2026) OpenAI 4
GPT-5.5 (Non-reasoning) OpenAI 5
o3-mini (high) OpenAI 5
GPT-5.6 Luna (Non-reasoning) OpenAI 7
GPT-5.6 Luna (low) OpenAI 7
GPT-5.5 (low) OpenAI 7
GPT-5.4 mini (Non-Reasoning) OpenAI 7
Claude 3.5 Haiku Anthropic 7
GPT-5.6 Sol (low) OpenAI 9
GPT-5.6 Terra (low) OpenAI 9
GPT-5.6 Sol (Non-reasoning) OpenAI 9
Kimi K3 (low) Kimi 9
Gemini 3.1 Flash-Lite Google 10
GPT-5.6 Luna (medium) OpenAI 11
K2 Think V2 MBZUAI Institute of Foundation Models 11
Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic 12
Mistral Small 4 (Reasoning) Mistral 12
Nova 2.0 Pro Preview (low) Amazon 12
Qwen3 235B A22B 2507 (Reasoning) Alibaba 12
GPT-5.6 Terra (medium) OpenAI 13
GPT-5.6 Terra (Non-reasoning) OpenAI 13
Nova 2.0 Lite (high) Amazon 13
Nemotron Cascade 2 30B A3B NVIDIA 13
MiMo-V2.5 Xiaomi 13
Qwen3 Next 80B A3B (Reasoning) Alibaba 13
DeepSeek V3 0324 DeepSeek 13
Grok 4.3 (high) SpaceXAI 13
Qwen3 32B (Reasoning) Alibaba 13
Claude Sonnet 5 (Adaptive Reasoning, Low Effort) Anthropic 14
Grok 4.3 (Non-reasoning) SpaceXAI 14
Qwen3.7 Max Alibaba 14
Qwen3 14B (Reasoning) Alibaba 14
Claude 4.5 Haiku (Reasoning) Anthropic 15
Grok 4.5 (high) SpaceXAI 15
GPT-5.5 (medium) OpenAI 15
GPT-5.6 Sol (medium) OpenAI 16
Llama 4 Scout Meta 16
Mistral Large 3 Mistral 16
Nova 2.0 Pro Preview (medium) Amazon 16
Qwen3.7 Plus Alibaba 16
GPT-5.1 (high) OpenAI 16
Grok Build 0.1 0616 SpaceXAI 16
Qwen3 8B (Reasoning) Alibaba 16
Qwen3 30B A3B 2507 (Reasoning) Alibaba 16
Muse Spark 1.1 (xhigh) Meta 17
Gemini 3.1 Pro Preview Google 17
MiMo-V2.5-Pro Xiaomi 17
GPT-5 mini (high) OpenAI 17
Claude 4.5 Sonnet (Reasoning) Anthropic 17
Llama 3.3 Instruct 70B Meta 18
Gemma 4 31B (Reasoning) Google 18
Gemma 4 31B (Non-reasoning) Google 18
DeepSeek V4 Pro (Reasoning, High Effort) DeepSeek 18
Gemini 2.5 Pro Google 19
Nova 2.0 Pro Preview (Non-reasoning) Amazon 19
Claude 4 Sonnet (Reasoning) Anthropic 19
Kimi K2.5 (Reasoning) Kimi 19
GLM-5.1 (Reasoning) Z AI 19
GLM-4.6 (Reasoning) Z AI 19
Devstral 2 Mistral 20
Ring-2.6-1T InclusionAI 20
Celeris-1 Celeris 20
GPT-5.5 (high) OpenAI 20
Muse Spark Meta 21
Gemini 3.5 Flash-Lite Google 21
Granite 4.1 30B IBM 21
Mercury 2 Inception 21
Hy3 Tencent 21
GPT-5 (high) OpenAI 21
Mistral Small 3.1 Mistral 21
DeepSeek V4 Flash (Reasoning, High Effort) DeepSeek 21
MiniMax-M2.7 MiniMax 21
GPT-5.6 Terra (high) OpenAI 22
GPT-5.6 Luna (high) OpenAI 22
Gemma 4 26B A4B (Reasoning) Google 22
Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA 22
Qwen3.6 27B (Reasoning) Alibaba 22
Qwen3.5 122B A10B (Reasoning) Alibaba 22
Gemma 4 E2B (Reasoning) Google 23
Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic 23
DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek 23
NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) NVIDIA 23
Qwen3.5 397B A17B (Reasoning) Alibaba 23
Qwen3.5 122B A10B (Non-reasoning) Alibaba 23
Devstral Small 2 Mistral 24
Step 3.7 Flash StepFun 24
Command A+ Cohere 24
Qwen3.6 Plus Alibaba 24
DeepSeek V3.2 (Reasoning) DeepSeek 24
GPT-5.6 Sol (high) OpenAI 25
Claude Sonnet 5 (Adaptive Reasoning, Medium Effort) Anthropic 25
KAT Coder Pro V2 KwaiKAT 25
GPT-5.5 (xhigh) OpenAI 25
DeepSeek R1 (Jan '25) DeepSeek 25
GPT-5.6 Terra (xhigh) OpenAI 26
Ministral 3 8B Mistral 26
Qwen3.6 35B A3B (Reasoning) Alibaba 26
Qwen3.5 35B A3B (Non-reasoning) Alibaba 26
Agnes 2.5 Pro Alpha Sapiens AI 26
DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek 26
GLM-4.7 (Reasoning) Z AI 26
Mistral Medium 3.5 Mistral 27
North Mini Code Cohere 27
GPT-4 OpenAI 27
gpt-oss-120b (high) OpenAI 29
GPT-5.6 Luna (xhigh) OpenAI 29
K-EXAONE (Reasoning) LG AI Research 29
Qwen3 Coder Next Alibaba 30
DeepSeek V3.1 Terminus (Reasoning) DeepSeek 30
GPT-5.6 Sol (xhigh) OpenAI 31
KAT-Coder-Pro V1 KwaiKAT 31
GLM-5.2 (max) Z AI 31
Qwen3.6 27B (Non-reasoning) Alibaba 31
Mistral Medium 3.1 Mistral 31
Kimi K2.7 Code Kimi 32
LongCat 2.0 LongCat 32
GPT-5.4 mini (xhigh) OpenAI 32
DiffusionGemma 26B A4B Google 33
Gemini 3.6 Flash (high) Google 33
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic 33
Ling 2.6 Flash InclusionAI 33
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic 34
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic 34
Kimi K2.6 Kimi 34
GPT-5.6 Luna (max) OpenAI 35
JT-4.1 Flash 236B A21B China Mobile 35
Nex-N2-Pro Nex AGI 35
Llama 4 Maverick Meta 36
Magistral Medium 1.2 Mistral 36
Motif 3 (Beta) Motif Technologies 36
gpt-oss-120b (low) OpenAI 37
GPT-5.6 Terra (max) OpenAI 37
Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic 38
Claude Sonnet 5 (Adaptive Reasoning, High Effort) Anthropic 38
Kimi K3 (max) Kimi 38
GLM-5.2 (Non-reasoning) Z AI 39
GPT-4.1 mini OpenAI 40
GPT-5.4 (xhigh) OpenAI 41
GPT-5.4 nano (xhigh) OpenAI 42
Inkling (xhigh) Thinking Machines 43
G9v3-3B AI9Stars 44
gpt-oss-20b (high) OpenAI 45
GPT-5.6 Sol (max) OpenAI 45
Granite 4.1 3B IBM 45
Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic 45
DeepSeek V4 Flash 0731 (Reasoning, Max Effort) DeepSeek 46
Gemini 3.5 Flash (high) Google 47
Claude Sonnet 5 (Non-reasoning, High Effort) Anthropic 48
EXAONE 4.5 33B LG AI Research 49
Trinity Large Thinking Arcee AI 49
G9v3-39A5B AI9Stars 49
Ministral 3 14B Mistral 50
Nemotron 3 Nano Omni 30B A3B Reasoning NVIDIA 50
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 51
NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA 51
NVIDIA Nemotron 3 Nano 4B NVIDIA 53
MiniMax-M3 MiniMax 54
Qwen3.6 35B A3B (Non-reasoning) Alibaba 54
HyperNova 60B 2605 Multiverse Computing 55
Gemma 4 12B (Reasoning) Google 56
Gemma 4 E4B (Reasoning) Google 58
Solar Pro 3 Upstage 58
Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 60
Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic 60
GPT-4o mini OpenAI 61
Magistral Small 1.2 Mistral 68
Ministral 3 3B Mistral 78
Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic 94
Qwen3.5 9B (Reasoning) Alibaba 95
GPT-4.1 nano OpenAI 101
Qwen3.5 2B (Reasoning) Alibaba 118
NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) NVIDIA 157
Llama 3.1 Instruct 8B Meta 158
Mistral Small 3.2 Mistral 160
MiMo-V2-Flash (Non-reasoning) Xiaomi 177
MiniCPM-V 4.6 1.3B OpenBMB 209
Qwen3.5 0.8B (Reasoning) Alibaba 233
Qwen3.5 0.8B (Non-reasoning) Alibaba 238
Phi-4 Mini Instruct Microsoft 250
Nanbeige4.1-3B Nanbeige 250
Qwen3.5 2B (Non-reasoning) Alibaba 250
Gemma 3 27B Instruct Google 250
Gemma 3 12B Instruct Google 250
Inkling Small Thinking Machines N/A

Cross-Evaluation Summary: Top Models Across All Three Benchmarks

Model Creator AA-Briefcase Elo Omniscience Index GDPval-AA v2 Elo
Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic 1719 (#1) 31 (#3) 1852 (#1)
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic 1691 (#2) 30 (#4) 1819 (#2)
Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic 1606 (#3) 28 (#5) 1735 (#4)
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic 1574 (#4) 40 (#1) 1743 (#3)
Kimi K3 (max) Kimi 1541 (#5) 18 (#19) 1685 (#6)
GPT-5.6 Sol (max) OpenAI 1502 (#6) 22 (#13) 1730 (#5)
Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic 1470 (#7) 26 (#9) 1628 (#8)
Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic 1385 (#8) 15 (#28) 1600 (#10)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic 1342 (#9) 27 (#6) 1588 (#11)
Grok 4.5 (high) SpaceXAI 1314 (#10) 26 (#7) 1526 (#18)
Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic 1277 (#12) 26 (#8) 1490 (#22)
GLM-5.2 (max) Z AI 1252 (#13) 4 (#50) 1508 (#20)
Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic 1224 (#14) 23 (#11) 1454 (#26)
GPT-5.5 (xhigh) OpenAI 1150 (#16) 20 (#16) 1490 (#23)
MiniMax-M3 MiniMax 1107 (#17) 1 (#62) 1389 (#33)
GPT-5.5 (high) OpenAI 1099 (#18) 18 (#21) 1466 (#25)
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic 1075 (#20) 12 (#36) 1375 (#35)
GPT-5.5 (medium) OpenAI 1000 (#22) 17 (#24) 1373 (#36)
GLM-5.1 (Reasoning) Z AI 971 (#23) 2 (#61) 1257 (#50)
Gemini 3.6 Flash (high) Google 962 (#24) 24 (#10) 1423 (#28)
DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek 930 (#26) -10 (#102) 1304 (#40)
Inkling Small Thinking Machines 917 (#27) -9 (#95) 1268 (#46)
Qwen3.7 Max Alibaba 914 (#28) 14 (#31) 1271 (#44)
MiMo-V2.5-Pro Xiaomi 880 (#29) 4 (#52) 1264 (#47)
Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA 874 (#30) -1 (#72) 1162 (#66)
Gemini 3.5 Flash (high) Google 872 (#31) 23 (#12) 1343 (#39)
Muse Spark 1.1 (xhigh) Meta 869 (#34) 18 (#23) 1371 (#38)
Inkling (xhigh) Thinking Machines 842 (#35) 2 (#59) 1237 (#54)
DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek 833 (#36) -23 (#150) 1190 (#59)
Kimi K2.6 Kimi 819 (#37) 6 (#43) 1189 (#60)
Qwen3.6 27B (Reasoning) Alibaba 810 (#38) -20 (#138) 1140 (#72)
Grok 4.3 (high) SpaceXAI 759 (#39) 18 (#22) 1085 (#79)
GPT-5.4 mini (xhigh) OpenAI 717 (#40) -19 (#135) 1169 (#64)
Muse Spark Meta 642 (#41) 4 (#49) 1144 (#71)
Gemini 3.5 Flash-Lite Google 634 (#42) 7 (#42) 1137 (#74)
Claude 4.5 Haiku (Reasoning) Anthropic 611 (#43) -4 (#83) 911 (#97)
KAT-Coder-Pro V1 KwaiKAT 599 (#44) -37 (#208) 902 (#98)
Qwen3.5 397B A17B (Reasoning) Alibaba 554 (#45) -30 (#174) 961 (#91)
Mistral Medium 3.5 Mistral 516 (#46) -36 (#206) 931 (#95)
Gemini 3.1 Pro Preview Google 458 (#47) 33 (#2) 963 (#90)
Gemma 4 31B (Reasoning) Google 374 (#48) -45 (#254) 807 (#106)
Command A+ Cohere 369 (#49) -4 (#81) 713 (#116)
North Mini Code Cohere 238 (#50) -49 (#284) 540 (#139)
Gemini 3.1 Flash-Lite Google 231 (#51) -16 (#123) 645 (#124)
Solar Pro 3 Upstage 138 (#52) -54 (#329) 496 (#144)
K2 Think V2 MBZUAI Institute of Foundation Models 59 (#53) -34 (#189) 377 (#154)
gpt-oss-120b (high) OpenAI 8 (#54) -50 (#291) 798 (#107)
gpt-oss-20b (high) OpenAI 0 (#55) -64 (#394) 562 (#133)
Llama 4 Maverick Meta 0 (#56) -42 (#230) 4 (#175)
NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA 0 (#57) -42 (#235) 697 (#117)

Data extracted 2026-08-04 from artificialanalysis.ai page HTML and chart Flight records. Values originate from current page Flight records; N/A/missing source values are omitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment