GSM8K

Grade School Math 8K — 8,500 linguistically diverse grade-school math word problems that require multi-step reasoning to solve.

math

Models Tested

91.3

Top Score

52.8

Average Score

Rankings

#	Model	Score
1	GPT-4o-mini (2024-07-18)OpenAI	91.3
2	GPT-4o-miniOpenAI	91.3
3	Qwen2.5 Coder 32B InstructAlibaba Qwen	91.1
4	GPT-4 TurboOpenAI	90.0
5	GPT-4 Turbo (older v1106)OpenAI	90.0
6	GPT-4 (older v0314)OpenAI	90.0
7	GPT-4OpenAI	90.0
8	Qwen2.5 Coder 7B InstructAlibaba Qwen	86.7
9	Claude InstantAnthropic	86.7
10	Gemma 2 9BGoogle DeepMind	84.9
11	Mistral NemoMistral AI	84.2
12	Llama 3.1 8B InstructMeta	82.4
13	Gemini 1.5 Flash (May 2024)Google	82.4
14	U Yi-34Bunknown	76.0
15	Mixtral 8x7B InstructMistral AI	74.4
16	Llama 2-70BMeta	69.6
17	U Stable Beluga 2unknown	69.6
18	Qwen2.5-Coder (1.5B)Alibaba	65.8
19	Qwen-14BAlibaba	61.3
20	U StarCoder 2 15Bunknown	57.7
21	LLaMA-65BMeta	54.4
22	Mistral 7B v0.1Mistral	54.4
23	T Falcon-180BTII	54.4
24	T Falcon 2 11BTII	53.8
25	U Baichuan2-13Bunknown	52.8
26	Qwen-7BAlibaba	51.7
27	Gemma 7BGoogle	46.4
28	U Nemotron-4 15Bunknown	46.0
29	U Yi 6Bunknown	44.9
30	LLaMA-33BMeta	44.1
31	Llama 2-34BMeta	42.2
32	U INTELLECT-1unknown	38.6
33	Llama 2-13BMeta	36.9
34	DeepSeek Coder 33BDeepSeek	35.4
35	U MPT-30Bunknown	34.4
36	T Falcon-40BTII	33.8
37	U StarCoder 2 7Bunknown	32.7
38	U Baichuan 2-7Bunknown	24.6
39	U StarCoder 2 3Bunknown	21.6
40	DeepSeek Coder 6.7BDeepSeek	21.3
41	LLaMA-13BMeta	20.6
42	Gemma 2BGoogle	17.7
43	Llama 2-7BMeta	16.7
44	LLaMA-7BMeta	11.0
45	U Baichuan1-7Bunknown	9.2
46	U MPT-7Bunknown	9.1
47	T Falcon-7BTII	6.8
48	DeepSeek Coder 1.3BDeepSeek	4.4