Benchmark · KnowledgeSettled

MMLU-PRO

Updated 2025-10-17
Models tested
78
Top score
52.6
Qwen2-72B
Median
23.5
min 0.3
Top-5 spread
σ 1.4
Settled

Best score over time · one chart, every benchmark

MMLU-PRO34 MODELS · FRONTIER RUNNING MAX0255075100SCORE ↑Aug 24Nov 24Mar 25Jul 25Oct 25RELEASE DATE →benchgecko.ai/benchmark/hf-mmlu-pro · frontier
Only 34 models have been tested on MMLU-PRO · not enough history to compute a frontier yet.
Pink dots = frontier records · 1 totalClick to open model page

78 models tested · sorted by score

#ModelScore
1Alibaba Qwen logoQwen2-72B52.6
2Alibaba logoQwen2.5 32B Instruct51.9
3Alibaba Qwen logoQwen2.5 72B Instruct51.4
4
HA
Qwen2.5 72B Instruct Abliterated
50.4
5Microsoft logoPhi 448.6
6Meta logoLlama 3.3 70B Instruct48.1
7Meta logoLlama 3.1 70B Instruct47.9
8Alibaba logoQwen2.5 14B Instruct43.2
9DeepSeek logoR1 Distill Llama 70B41.6
10nousresearch logoHermes 3 70B Instruct41.4
11Meta logoMeta Llama 3 8B41.2
12DeepSeek logoDeepSeek R1 Distill Qwen 32B41.0
13DeepSeek logoR1 Distill Qwen 32B41.0
14DeepSeek logoDeepSeek R1 Distill Qwen 14B40.7
15Microsoft logoWizardLM-2 8x22B40.0
16
D
Dolphin 2.9.1 Yi 1.5 34b
39.1
17Google DeepMind logoGemma 2 27B38.4
18Alibaba Qwen logoQwen2.5 Coder 32B Instruct37.9
19
U
Yi 6B
37.9
20Alibaba Qwen logoQwen2.5 7B Instruct36.5
21z-ai logoGLM 4 32B 34.9
22Alibaba logoQwen2 VL 7B Instruct34.4
23Microsoft logoPhi 3 Mini 4k Instruct33.6
24Microsoft logoPhi 3.5 Mini Instruct32.9
25Alibaba logoQwen2.5 Coder 14B Instruct32.7
26Microsoft logoPhi 4 Mini Instruct32.6
27Microsoft logoPhi 4 Mini Instruct32.6
28Google DeepMind logoGemma 2 9B31.9
29Alibaba logoQwen2 7B Instruct31.6
30anthracite-org logoMagnum v4 72B31.4
31Meta logoLlama 3.1 8B Instruct30.9
32Meta logoMeta Llama 3 8B Instruct29.6
33Alibaba logoQwen2.5 Coder 7B Instruct26.1
34Alibaba Qwen logoQwen2.5 Coder 7B Instruct26.1
35
U
Stable Beluga 2
25.9
36Meta logoLlama 3.1 405B25.7
37Alibaba logoQwen2.5 3B Instruct25.1
38Meta logoLlama 3.2 3B Instruct24.4
39Meta logoLlama 3-70B24.0
40Meta logoLLaMA-13B23.1
41nousresearch logoHermes 2 Pro - Llama-3 8B22.8
42Mistral AI logoMistral 7B V0.122.4
43Google DeepMind logoGemma 2B21.6
44Alibaba logoQwen2.5 1.5B Instruct20.0
45Mistral AI logoMistral 7B Instruct V0.219.1
46Microsoft logoPhi 218.1
47Meta logoLlama 3 8B Instruct17.8
48Google DeepMind logoGemma 2 2b It17.2
49Alibaba logoQwen2 1.5B Instruct16.7
50Meta logoLlama 3.2 3B Instruct (free)16.5
51Mistral AI logoMistral 7B Instruct v0.115.7
52TII logoFalcon-180B15.4
53
U
StarCoder 2 15B
15.0
54DeepSeek logoDeepSeek R1 Distill Qwen 7B14.7
55Google DeepMind logoGemma 2 2b13.5
56
L
Vicuna 7b V1.5
12.7
57DeepSeek logoDeepSeek R1 Distill Llama 8B12.1
58Meta logoLlama 2 7b Hf9.6
59Meta logoLlama 3.2 1B Instruct8.2
60Alibaba logoQwen2 0.5B8.0
61Alibaba logoQwen2.5 0.5B Instruct8.0
62Microsoft logoPhi-1.57.7
63Meta logoLlama 2 7b Chat Hf7.6
64Alibaba logoQwen2 0.5B Instruct5.9
65
U
MPT-30B
2.3
66
U
RedPajama-INCITE-7B-Base
2.2
67Alibaba Qwen logoQwQ 32B2.2
68DeepSeek logoDeepSeek R1 Distill Qwen 1.5B2.1
69
D
Distilgpt2
2.1
70OpenAI logoGpt2 Medium2.0
71OpenAI logoGpt21.8
72OpenAI logoGpt2 Large1.6
73
U
INTELLECT-1
1.3
74eleutherai logoPythia 160m1.3
75
HF
SmolLM2 135M Instruct
1.3
76
T
TinyLlama 1.1B Chat V1.0
1.1
77
HF
SmolLM2 135M
1.1
78eleutherai logoGpt Neo 125m0.3

Same category · related evaluations