HellaSwag
HellaSwag · tests commonsense reasoning by asking models to predict the most plausible continuation of everyday scenarios.
The Frontier
Best score over time · one chart, every benchmark
Full rankings
42 models tested · sorted by score
| # | Model | Score |
|---|---|---|
| 1 | 93.7 | |
| 2 | 85.6 | |
| 3 | 85.3 | |
| 4 | 85.2 | |
| 5 | 82.8 | |
| 6 | U PaLM 2-L | 82.4 |
| 7 | 82.3 | |
| 8 | 82.3 | |
| 9 | 79.7 | |
| 10 | U Stable Beluga 2 | 78.8 |
| 11 | U PaLM 2-M | 78.7 |
| 12 | 77.3 | |
| 13 | 77.2 | |
| 14 | U Nemotron-4 15B | 76.5 |
| 15 | 76.5 | |
| 16 | U PaLM 2-S | 76.0 |
| 17 | 74.7 | |
| 18 | 74.3 | |
| 19 | 73.6 | |
| 20 | 72.3 | |
| 21 | U internlm-20b | 70.8 |
| 22 | 69.3 | |
| 23 | 69.1 | |
| 24 | 69.1 | |
| 25 | 68.9 | |
| 26 | U MPT-30B | 68.5 |
| 27 | U Yi 6B | 65.9 |
| 28 | U XGen-7B | 65.6 |
| 29 | 62.4 | |
| 30 | U INTELLECT-1 | 61.9 |
| 31 | 61.9 | |
| 32 | 61.2 | |
| 33 | U Dolly 2.0-12b | 61.1 |
| 34 | U RedPajama-INCITE-7B-Base | 60.4 |
| 35 | U Baichuan 2-7B | 57.3 |
| 36 | 49.1 | |
| 37 | 45.9 | |
| 38 | U vicuna-13b-v1.1 | 43.7 |
| 39 | 38.1 | |
| 40 | 31.2 | |
| 41 | 30.1 | |
| 42 | U stablelm-tuned-alpha-7b | 20.9 |
Score distribution
Where models cluster
Correlated benchmarks
Pearson r · original research
Benchmarks that track with HellaSwag
Pearson correlation across models scored on both benchmarks. Closer to 1 = strongly predictive.
Frequently asked
About HellaSwag
What does HellaSwag measure?
HellaSwag · tests commonsense reasoning by asking models to predict the most plausible continuation of everyday scenarios. 42 AI models have been tested on it. Scores range from 20.9 to 93.7 out of 100.
Which model leads on HellaSwag?
GPT-4 Turbo from OpenAI leads HellaSwag with a score of 93.7. The median score across 42 tested models is 70.1.
Is HellaSwag saturated?
No · the top score is 93.7 out of 100 (94%). There is still meaningful room for improvement on HellaSwag.
Does HellaSwag predict performance on other benchmarks?
Yes · HellaSwag scores correlate 0.99 with Aider · Code Editing across 7 shared models. Models that do well on HellaSwag tend to do well on Aider · Code Editing.
How often is HellaSwag data refreshed?
BenchGecko pulls updates daily. New model scores on HellaSwag appear as soon as they are published by Epoch AI or the model provider.
- Category
- Reasoning
- Max score
- 100
- Models
- 42
- Updated
- 2025-04-15
Top on HellaSwag
GPT-4 Turbo · 93.7Llama 3.1 405B · 85.6Falcon-180B · 85.3DeepSeek V3 · 85.2DeepSeek-V2 (MoE-236B, May 2024) · 82.8More reasoning benchmarks
Same category · related evaluations