HellaSwag

HellaSwag — tests commonsense reasoning by asking models to predict the most plausible continuation of everyday scenarios.

knowledge

Models Tested

85.6

Top Score

69.8

Average Score

Rankings

#	Model	Score
1	Llama 3.1-405BMeta	85.6
2	T Falcon-180BTII	85.3
3	DeepSeek V3DeepSeek	85.2
4	DeepSeek-V2 (MoE-236B, May 2024)DeepSeek	82.8
5	Mixtral 8x7B InstructMistral AI	82.3
6	Llama 2-70BMeta	80.4
7	T Falcon-40BTII	80.4
8	Qwen2.5 72B InstructAlibaba Qwen	79.7
9	LLaMA-65BMeta	78.9
10	U Stable Beluga 2unknown	78.8
11	Qwen2.5 Coder 32B InstructAlibaba Qwen	77.3
12	T Falcon 2 11BTII	77.2
13	LLaMA-33BMeta	77.1
14	U Nemotron-4 15Bunknown	76.5
15	phi-3-medium 14BMicrosoft	76.5
16	Gemma 7BGoogle	76.3
17	Mistral 7B v0.1Mistral	74.7
18	Llama 2-13BMeta	74.3
19	LLaMA-13BMeta	72.3
20	T Falcon-7BTII	70.8
21	Llama 2-7BMeta	69.6
22	phi-3-small 7.4BMicrosoft	69.3
23	Qwen2.5 Coder 7B InstructAlibaba Qwen	69.1
24	phi-3-mini 3.8BMicrosoft	68.9
25	U MPT-7Bunknown	68.5
26	LLaMA-7BMeta	68.3
27	U Yi 6Bunknown	65.9
28	U XGen-7Bunknown	65.6
29	U INTELLECT-1unknown	61.9
30	Gemma 2BGoogle	61.9
31	U Baichuan2-13Bunknown	61.1
32	U Dolly 2.0-12bunknown	61.1
33	U Baichuan 2-7Bunknown	57.3
34	Qwen2.5-Coder (1.5B)Alibaba	49.1
35	Cerebras-GPT-13BOpenAI	45.9
36	Phi-2Microsoft	38.1
37	Phi-1.5Microsoft	30.1