Which model leads on HellaSwag?

GPT-4 Turbo (batch) from OpenAI leads HellaSwag with a score of 93.7. The median score across 43 tested models is 70.8.

Is HellaSwag saturated?

No · the top score is 93.7 out of 100 (94%). There is still meaningful room for improvement on HellaSwag.

Does HellaSwag predict performance on other benchmarks?

Yes · HellaSwag scores correlate 0.99 with Aider · Code Editing across 7 shared models. Models that do well on HellaSwag tend to do well on Aider · Code Editing.

How often is HellaSwag data refreshed?

BenchGecko pulls updates daily. New model scores on HellaSwag appear as soon as they are published by Epoch AI or the model provider.

Benchmark · ReasoningCompetitive

HellaSwag

Name: HellaSwag Benchmark
Creator: BenchGecko
License: https://creativecommons.org/licenses/by/4.0/

HellaSwag · tests commonsense reasoning by asking models to predict the most plausible continuation of everyday scenarios.

Updated 2025-04-15

Models tested

Top score

93.7

GPT-4 Turbo (batch)

Median

70.8

min 20.9

Top-5 spread

σ 4.1

Competitive

The Frontier

Best score over time · one chart, every benchmark

Chart type

Only 2 models have been tested on HellaSwag · not enough history to compute a frontier yet.

Pink dots = frontier records · 0 totalClick to open model page

Full rankings

43 models tested · sorted by score

#	Model	Score	Price
1	GPT-4 Turbo (batch)· OpenAI	93.7	$5.00
2	GPT-4 Turbo· OpenAI	93.7	$10.00
3	Llama 3.1 405B· Meta	85.6	n/a
4	Falcon-180B· TII	85.3	n/a
5	DeepSeek V3· DeepSeek	85.2	$0.32
6	DeepSeek-V2 (MoE-236B, May 2024)· DeepSeek	82.8	n/a
7	U PaLM 2-L· Unknown	82.4	n/a
8	Mixtral 8x7B Instruct· Mistral AI	82.3	$0.54
9	Mixtral 8x7B· Mistral AI	82.3	n/a
10	Qwen2.5 72B Instruct· Alibaba Qwen	79.7	$0.36
11	U Stable Beluga 2· Unknown	78.8	n/a
12	U PaLM 2-M· Unknown	78.7	n/a
13	Qwen2.5 Coder 32B Instruct· Alibaba Qwen	77.3	$0.66
14	Falcon 2 11B· TII	77.2	n/a
15	phi-3-medium 14B· Microsoft	76.5	n/a
16	U Nemotron-4 15B· Unknown	76.5	n/a
17	U PaLM 2-S· Unknown	76.0	n/a
18	Mistral 7B V0.1· Mistral AI	74.7	n/a
19	Llama 2-13B· Meta	74.3	n/a
20	Qwen2.5 Coder 14B Instruct· Alibaba	73.6	n/a
21	LLaMA-13B· Meta	72.3	n/a
22	U internlm-20b· Unknown	70.8	n/a
23	phi-3-small 7.4B· Microsoft	69.3	n/a
24	Qwen2.5 Coder 7B Instruct· Alibaba	69.1	n/a
25	Qwen2.5 Coder 7B Instruct· Alibaba Qwen	69.1	$0.03
26	phi-3-mini 3.8B· Microsoft	68.9	n/a
27	U MPT-30B· Unknown	68.5	n/a
28	U Yi 6B· Unknown	65.9	n/a
29	U XGen-7B· Unknown	65.6	n/a
30	open_llama_7b· Meta	62.4	n/a
31	U INTELLECT-1· Unknown	61.9	n/a
32	Gemma 2B· Google DeepMind	61.9	n/a
33	Qwen2.5-Coder-3B· Alibaba Qwen	61.2	n/a
34	U Dolly 2.0-12b· Unknown	61.1	n/a
35	U RedPajama-INCITE-7B-Base· Unknown	60.4	n/a
36	U Baichuan 2-7B· Unknown	57.3	n/a
37	Qwen2.5 Coder 1.5B Instruct· Alibaba	49.1	n/a
38	Cerebras-GPT-13B· OpenAI	45.9	n/a
39	U vicuna-13b-v1.1· Unknown	43.7	n/a
40	Phi 2· Microsoft	38.1	n/a
41	Qwen2.5 Coder 0.5B Instruct· Alibaba	31.2	n/a
42	Phi-1.5· Microsoft	30.1	n/a
43	U stablelm-tuned-alpha-7b· Unknown	20.9	n/a