BBH

BIG-Bench Hard — a curated subset of 23 challenging tasks from BIG-Bench where language models previously failed to outperform average humans.

reasoning

Models Tested

83.3

Top Score

42.3

Average Score

Rankings

#	Model	Score
1	DeepSeek V3DeepSeek	83.3
2	Llama 3.1-405BMeta	77.2
3	phi-3-medium 14BMicrosoft	75.2
4	Qwen2.5 72B InstructAlibaba Qwen	73.1
5	phi-3-small 7.4BMicrosoft	72.1
6	DeepSeek-V2 (MoE-236B, May 2024)DeepSeek	71.7
7	GPT-4 TurboOpenAI	66.8
8	GPT-4 Turbo (older v1106)OpenAI	66.8
9	GPT-4 (older v0314)OpenAI	66.8
10	GPT-4OpenAI	66.8
11	U Yi-34Bunknown	62.3
12	phi-3-mini 3.8BMicrosoft	62.3
13	U Stable Beluga 2unknown	59.1
14	Llama 2-70BMeta	53.2
15	Phi-2Microsoft	45.9
16	U Nemotron-4 15Bunknown	44.9
17	LLaMA-65BMeta	44.5
18	Llama 2-13BMeta	44.3
19	Mistral 7B v0.1Mistral	41.5
20	Gemma 7BGoogle	40.1
21	Qwen-14BAlibaba	40.0
22	LLaMA-33BMeta	33.3
23	U Baichuan2-13Bunknown	32.0
24	U Yi 6Bunknown	29.6
25	Qwen-7BAlibaba	26.7
26	Llama 2-34BMeta	25.5
27	U Baichuan 2-7Bunknown	22.1
28	Llama 2-7BMeta	18.9
29	U MPT-30Bunknown	17.3
30	LLaMA-13BMeta	17.2
31	T Falcon-40BTII	16.1
32	U MPT-7Bunknown	14.1
33	Gemma 2BGoogle	13.6
34	U INTELLECT-1unknown	13.1
35	LLaMA-7BMeta	11.3
36	U Baichuan1-7Bunknown	10.0
37	T Falcon-7BTII	5.0