ARC AI2
AI2 Reasoning Challenge · tests grade-school level science knowledge with multiple-choice questions requiring reasoning beyond simple retrieval.
The Frontier
Best score over time · one chart, every benchmark
Distribution
Where models cluster
Correlated benchmarks
Pearson r · original research
Benchmarks that track with ARC AI2
Pearson correlation across models scored on both benchmarks. Closer to 1 = strongly predictive.
Full rankings
35 models tested · sorted by score
| # | Model | Score |
|---|---|---|
| 1 |  DeepSeek V3 | 93.7 |
| 2 |  Llama 3.1 405B | 93.7 |
| 3 |  Qwen2.5 72B Instruct | 92.7 |
| 4 | DeepSeek-V2 (MoE-236B, May 2024) | 89.6 |
| 5 |  phi-3-medium 14B | 88.8 |
| 6 | phi-3-small 7.4B | 87.6 |
| 7 |  GPT-3.5 Turbo (older v0613) | 83.2 |
| 8 |  Mixtral 8x7B Instruct | 83.1 |
| 9 |  Claude Instant | 81.7 |
| 10 | U Stable Beluga 2 | 81.5 |
| 11 | phi-3-mini 3.8B | 79.9 |
| 12 | Qwen-14B | 79.2 |
| 13 | Llama 3 8B Instruct | 77.1 |
| 14 | Mistral 7B V0.1 | 71.5 |
| 15 | Phi 2 | 67.9 |
| 16 | Qwen2.5 Coder 32B Instruct | 60.7 |
| 17 |  Falcon-180B | 57.1 |
| 18 | Qwen2.5 Coder 7B Instruct | 47.9 |
| 19 | Llama 2-13B | 47.1 |
| 20 | U Nemotron-4 15B | 40.7 |
| 21 | U INTELLECT-1 | 39.4 |
| 22 | LLaMA-13B | 36.9 |
| 23 | U MPT-30B | 34.1 |
| 24 | U Yi 6B | 33.7 |
| 25 | U StarCoder 2 15B | 29.6 |
| 26 | Qwen2.5 Coder 1.5B Instruct | 26.9 |
| 27 | Phi-1.5 | 25.9 |
| 28 | DeepSeek Coder 33B | 22.9 |
| 29 |  Gemma 2B | 22.8 |
| 30 | U XGen-7B | 21.6 |
| 31 | U Dolly 2.0-12b | 19.5 |
| 32 | DeepSeek Coder 6.7B | 15.2 |
| 33 | U Baichuan 2-7B | 10.0 |
| 34 | Cerebras-GPT-13B | 9.9 |
| 35 | DeepSeek Coder 1.3B | 0.5 |
Frequently asked
Pulled from the ARC AI2 dataset · updated daily
What does ARC AI2 measure?
ARC AI2 is a knowledge benchmark in the BenchGecko catalog. 35 AI models have been tested on it. Scores range from 0.5 to 93.7 out of 100.
Which model leads on ARC AI2?
DeepSeek V3 from DeepSeek leads ARC AI2 with a score of 93.7. The median score across 35 tested models is 47.9.
Is ARC AI2 saturated?
No · the top score is 93.7 out of 100 (94%). There is still meaningful room for improvement on ARC AI2.
Does ARC AI2 predict performance on other benchmarks?
Yes · ARC AI2 scores correlate 0.90 with Chatbot Arena Elo · Overall across 5 shared models. Models that do well on ARC AI2 tend to do well on Chatbot Arena Elo · Overall.
How often is ARC AI2 data refreshed?
BenchGecko pulls updates daily. New model scores on ARC AI2 appear as soon as they are published by Epoch AI or the model provider.
More knowledge benchmarks
Same category · related evaluations