Which model leads on MMLU?

GPT-4o (2024-11-20) from OpenAI leads MMLU with a score of 84.1. The median score across 84 tested models is 61.8.

No · the top score is 84.1 out of 100 (84%). There is still meaningful room for improvement on MMLU.

Does MMLU predict performance on other benchmarks?

Yes · MMLU scores correlate 0.99 with ScienceQA across 5 shared models. Models that do well on MMLU tend to do well on ScienceQA.

How often is MMLU data refreshed?

BenchGecko pulls updates daily. New model scores on MMLU appear as soon as they are published by Epoch AI or the model provider.

Benchmark · KnowledgeSettled

MMLU

Name: MMLU Benchmark
Creator: BenchGecko
License: https://creativecommons.org/licenses/by/4.0/

Massive Multitask Language Understanding · 57 subjects spanning STEM, humanities, social sciences, and more. The standard benchmark for broad knowledge.

Updated 2025-04-15

Models tested

Top score

84.1

GPT-4o (2024-11-20)

Median

61.8

min 1.1

Top-5 spread

σ 0.9

Settled

The Frontier

Best score over time · one chart, every benchmark

Chart type

Only 8 models have been tested on MMLU · not enough history to compute a frontier yet.

Pink dots = frontier records · 1 totalClick to open model page

Full rankings

84 models tested · sorted by score

#	Model	Score	Price
1	GPT-4o (2024-11-20)· OpenAI	84.1	$2.50
2	DeepSeek V3· DeepSeek	82.9	$0.32
3	Claude 3.5 Sonnet· Anthropic	82.0	n/a
4	GPT-4 (older v0314)· OpenAI	81.9	$30.00
5	Llama 3.3 70B Instruct· Meta	81.7	$0.71
6	Llama 3.3 70B Instruct (free)· Meta	81.7	$0.00
7	Gemini 1.5 Pro (May 2024)· Google DeepMind	81.2	n/a
8	Qwen2.5 72B Instruct· Alibaba Qwen	80.4	$0.36
9	Phi 4· Microsoft	79.7	$0.07
10	Claude 3 Opus· Anthropic	79.5	n/a
11	Llama 3.1 405B· Meta	79.3	n/a
12	GPT-4o (2024-08-06)· OpenAI	79.1	$2.50
13	GPT-4o (2024-05-13)· OpenAI	78.9	$5.00
14	Gemini 1.5 Pro (Feb 2024)· Google DeepMind	76.9	n/a
15	Qwen2-72B· Alibaba Qwen	76.5	n/a
16	GPT-4 Turbo (batch)· OpenAI	76.5	$5.00
17	GPT-4 Turbo· OpenAI	76.5	$10.00
18	GPT-4o-mini (2024-07-18)· OpenAI	75.7	$0.15
19	GPT-4o-mini· OpenAI	75.7	$0.15
20	Llama 3.2 90B· Meta	73.7	n/a
21	Llama 3.1 70B Instruct· Meta	73.5	$0.72
22	Mistral Large 2407· Mistral AI	73.3	$2.00
23	Gemini 2.0 Flash (Dec 2024)· Google DeepMind	72.9	n/a
24	Gemini 2.0 Flash· Google DeepMind	72.9	$0.10
25	Llama 3 70B Instruct· Meta	72.4	$0.51
26	Llama 3-70B· Meta	72.4	n/a
27	Qwen2.5 Coder 32B Instruct· Alibaba Qwen	72.1	$0.66
28	Claude 2· Anthropic	71.3	n/a
29	DeepSeek-V2 (MoE-236B, May 2024)· DeepSeek	71.2	n/a
30	phi-3-medium 14B· Microsoft	70.7	n/a
31	Gemini 1.5 Flash (May 2024)· Google DeepMind	70.5	n/a
32	Mixtral 8x22B Instruct· Mistral AI	70.4	$2.00
33	Claude 3 Sonnet· Anthropic	67.9	n/a
34	phi-3-small 7.4B· Microsoft	67.6	n/a
35	Gemma 2 27B· Google DeepMind	67.6	$0.65
36	Qwen2.5 Coder 14B Instruct· Alibaba	66.9	n/a
37	Claude 3.5 Haiku· Anthropic	65.7	n/a
38	Claude 3 Haiku· Anthropic	65.1	$0.25
39	Claude 2.1· Anthropic	64.7	n/a
40	Claude Instant· Anthropic	64.5	n/a
41	Qwen2.5 7B Instruct· Alibaba Qwen	63.9	$0.10
42	Gemma 2 9B· Google DeepMind	62.8	$0.03
43	Mixtral 8x7B Instruct· Mistral AI	60.8	$0.54
44	Falcon-180B· TII	60.8	n/a
45	Mixtral 8x7B· Mistral AI	60.8	n/a
46	Gemini 1.0 Pro· Google DeepMind	60.0	n/a
47	Command R (08-2024)· Cohere	59.2	$0.15
48	Mistral Large· Mistral AI	58.4	$2.00
49	Llama 3 8B Instruct· Meta	58.4	$0.14
50	phi-3-mini 3.8B· Microsoft	58.4	n/a
51	U Stable Beluga 2· Unknown	58.1	n/a
52	Qwen2.5 Coder 7B Instruct· Alibaba	57.3	n/a
53	Qwen2.5 Coder 7B Instruct· Alibaba Qwen	57.3	$0.03
54	GPT-3.5 Turbo (older v0613)· OpenAI	56.4	$1.00
55	Qwen-14B· Alibaba Qwen	55.1	n/a
56	U StarCoder 2 15B· Unknown	52.1	n/a
57	U Yi 6B· Unknown	52.0	n/a
58	Mistral 7B V0.1· Mistral AI	50.0	n/a
59	DeepSeek-Coder-V2-Lite-Base· DeepSeek	47.3	n/a
60	U Nemotron-4 15B· Unknown	44.9	n/a
61	Phi 2· Microsoft	44.5	n/a
62	Falcon 2 11B· TII	44.5	n/a
63	Llama 3.1 8B Instruct· Meta	41.5	$0.02
64	Llama 2-13B· Meta	40.8	n/a
65	U Baichuan 2-7B· Unknown	38.9	n/a
66	Qwen2.5 Coder 1.5B Instruct· Alibaba	38.1	n/a
67	U internlm-20b· Unknown	34.7	n/a
68	U INTELLECT-1· Unknown	33.2	n/a
69	U MPT-30B· Unknown	30.5	n/a
70	LLaMA-13B· Meta	30.3	n/a
71	U Baichuan1-7B· Unknown	23.1	n/a
72	Gemma 2B· Google DeepMind	23.1	n/a
73	Qwen2.5 Coder 0.5B Instruct· Alibaba	22.7	n/a
74	CodeQwen1.5-7B· Alibaba Qwen	20.7	n/a
75	DeepSeek Coder 33B· DeepSeek	19.2	n/a
76	Phi-1.5· Microsoft	16.8	n/a
77	DeepSeek Coder 6.7B· DeepSeek	15.2	n/a
78	U XGen-7B· Unknown	15.1	n/a
79	open_llama_7b· Meta	6.5	n/a
80	Qwen-1_8B· Alibaba Qwen	4.3	n/a
81	U RedPajama-INCITE-7B-Base· Unknown	1.7	n/a
82	Cerebras-GPT-13B· OpenAI	1.6	n/a
83	U Dolly 2.0-12b· Unknown	1.6	n/a
84	DeepSeek Coder 1.3B· DeepSeek	1.1	n/a