Benchmarks/OTIS Mock AIME 2024-2025

OTIS Mock AIME 2024-2025

OTIS Mock AIME 2024–2025 — simulated American Invitational Mathematics Examination problems testing advanced problem-solving skills.

math

105

Models Tested

96.1

Top Score

42.9

Average Score

Rankings

#	Model	Score
1	GPT-5.2 ChatOpenAI	96.1
2	GPT-5.2OpenAI	96.1
3	Gemini 3.1 Pro PreviewGoogle DeepMind	95.6
4	GPT-5.4OpenAI	95.3
5	Claude Opus 4.6Anthropic	94.4
6	Gemini 3 Flash PreviewGoogle DeepMind	92.8
7	M Kimi K2.5moonshotai	92.2
8	GPT-5 ChatOpenAI	91.4
9	GPT-5OpenAI	91.4
10	Gemini 3 ProGoogle	91.4
11	gpt-oss-120b (free)OpenAI	88.9
12	gpt-oss-120bOpenAI	88.9
13	gpt-oss-20b (free)OpenAI	88.9
14	gpt-oss-20bOpenAI	88.9
15	GPT-5.1OpenAI	88.6
16	GPT-5.1 ChatOpenAI	88.6
17	DeepSeek V3.2DeepSeek	87.8
18	GPT-5 MiniOpenAI	86.7
19	Qwen3 235B A22B Thinking 2507Alibaba Qwen	86.7
20	Claude Opus 4.5Anthropic	86.1
21	Claude Sonnet 4.6Anthropic	85.8
22	Gemini 2.5 ProGoogle DeepMind	84.7
23	Grok 4xAI	84.0
24	o3OpenAI	83.9
25	ZA GLM 4.7z-ai	83.3
26	M Kimi K2 Thinkingmoonshotai	83.0
27	o4 MiniOpenAI	81.7
28	GPT-5 NanoOpenAI	81.1
29	ZA GLM 5 Turboz-ai	80.0
30	ZA GLM 5z-ai	80.0
31	Claude Sonnet 4.5Anthropic	77.8
32	Grok 3 MinixAI	77.8
33	Grok 3 Mini BetaxAI	77.8
34	o3 MiniOpenAI	76.9
35	Qwen3 MaxAlibaba Qwen	73.3
36	o1OpenAI	73.3
37	Gemini 2.5 FlashGoogle DeepMind	73.0
38	Claude Sonnet 4Anthropic	71.1
39	Claude Opus 4.1Anthropic	68.9
40	Claude Haiku 4.5Anthropic	66.6
41	DeepSeek-R1 (May 2025)DeepSeek	66.4
42	Claude Opus 4Anthropic	64.4
43	Claude 3.7 SonnetAnthropic	57.7
44	Claude 3.7 Sonnet (thinking)Anthropic	57.7
45	Gemini 2.0 Flash Thinking (Jan 2025)Google	57.7
46	Grok 3xAI	55.5
47	Grok 3 BetaxAI	55.5
48	DeepSeek-R1DeepSeek	53.3
49	o1-miniOpenAI	46.9
50	GPT-4.1 MiniOpenAI	44.7
51	GPT-4.1OpenAI	38.3
52	GPT-4.5OpenAI	37.7
53	Mistral Medium 3Mistral AI	32.1
54	o1-previewOpenAI	31.0
55	GPT-4.1 NanoOpenAI	28.8
56	Gemini 1.5 Pro (Sept 2024)Google	23.0
57	Llama 4 MaverickMeta	20.5
58	Gemma 3 4B (free)Google DeepMind	19.6
59	Gemma 3 4BGoogle DeepMind	19.6
60	Gemma 3 12B (free)Google DeepMind	19.6
61	Gemma 3 12BGoogle DeepMind	19.6
62	Gemma 3 27B (free)Google DeepMind	19.6
63	Gemma 3 27BGoogle DeepMind	19.6
64	Gemini 1.5 Flash (Sep 2024)Google	16.2
65	Qwen2.5-MaxAlibaba	16.0
66	DeepSeek V3DeepSeek	15.8
67	Phi 4Microsoft	13.7
68	Grok-2 (Dec 2024)xAI	11.4
69	Llama 3.1-405BMeta	9.6
70	Mistral Large 2407Mistral AI	8.4
71	Qwen2.5 72B InstructAlibaba Qwen	8.0
72	Llama 4 ScoutMeta	7.7
73	Mistral Large 2411Mistral AI	7.7
74	GPT-4o-mini (2024-07-18)OpenAI	6.8
75	GPT-4o-miniOpenAI	6.8
76	Gemini 1.5 Pro (Feb 2024)Google	6.7
77	GPT-4 Turbo PreviewOpenAI	6.6
78	Claude 3.5 SonnetAnthropic	6.4
79	GPT-4o (2024-08-06)OpenAI	6.3
80	GPT-4o (2024-05-13)OpenAI	6.3
81	GPT-4oOpenAI	6.3
82	GPT-4o (extended)OpenAI	6.3
83	GPT-4o (2024-11-20)OpenAI	6.2
84	Llama 3.3 70B Instruct (free)Meta	5.0
85	Llama 3.3 70B InstructMeta	5.0
86	Claude 3 OpusAnthropic	4.6
87	Claude 3.5 HaikuAnthropic	4.2
88	Llama 3 70B InstructMeta	4.2
89	Gemini 1.5 Flash (May 2024)Google	3.8
90	Llama 3.1 70B InstructMeta	3.5
91	Llama 3.2 90BMeta	2.5
92	Llama 3.1 8B InstructMeta	2.4
93	Claude 3 SonnetAnthropic	2.4
94	Claude 2Anthropic	2.4
95	Mistral LargeMistral AI	1.9
96	Claude 2.1Anthropic	1.9
97	Claude 3 HaikuAnthropic	1.7
98	Gemma 2 27BGoogle DeepMind	1.3
99	GPT-4 TurboOpenAI	1.0
100	GPT-4 Turbo (older v1106)OpenAI	1.0
101	GPT-4 (older v0314)OpenAI	1.0
102	GPT-4OpenAI	1.0
103	Gemini 1.0 ProGoogle	1.0
104	Llama 3 8B InstructMeta	0.7
105	Gemma 2 9BGoogle DeepMind	0.5

Related Benchmarks

FrontierMath-2025-02-28-Private

60 models

FrontierMath-Tier-4-2025-07-01-Private

39 models