Benchmarks/GPQA diamond

GPQA diamond

Graduate-Level Google-Proof QA (Diamond set) — expert-crafted questions in physics, biology, and chemistry that are difficult even for domain PhDs.

knowledge

115

Models Tested

92.1

Top Score

46.1

Average Score

Rankings

#	Model	Score
1	Gemini 3.1 Pro PreviewGoogle DeepMind	92.1
2	GPT-5.4OpenAI	91.1
3	Gemini 3 ProGoogle	90.2
4	GPT-5.2 ChatOpenAI	88.5
5	GPT-5.2OpenAI	88.5
6	Claude Opus 4.6Anthropic	87.4
7	ZA GLM 5 Turboz-ai	83.8
8	ZA GLM 5z-ai	83.8
9	GPT-5.1OpenAI	83.5
10	GPT-5.1 ChatOpenAI	83.5
11	M Kimi K2.5moonshotai	83.5
12	Claude Sonnet 4.6Anthropic	83.2
13	Grok 4xAI	82.7
14	GPT-5 ChatOpenAI	81.6
15	GPT-5OpenAI	81.6
16	Claude Opus 4.5Anthropic	81.4
17	Gemini 2.5 ProGoogle DeepMind	80.4
18	M Kimi K2 Thinkingmoonshotai	79.0
19	DeepSeek V3.2DeepSeek	77.9
20	ZA GLM 4.7z-ai	77.8
21	Gemini 3 Flash PreviewGoogle DeepMind	77.6
22	Claude Sonnet 4.5Anthropic	76.4
23	o3OpenAI	75.8
24	Qwen3 235B A22B Thinking 2507Alibaba Qwen	73.4
25	Claude 3.7 SonnetAnthropic	73.0
26	Claude 3.7 Sonnet (thinking)Anthropic	73.0
27	o4 MiniOpenAI	72.8
28	Claude Sonnet 4Anthropic	72.3
29	Claude Opus 4.1Anthropic	69.7
30	o3 MiniOpenAI	69.4
31	o1OpenAI	69.0
32	DeepSeek-R1 (May 2025)DeepSeek	68.4
33	Grok 3 MinixAI	68.3
34	Claude Opus 4Anthropic	68.3
35	Grok 3 Mini BetaxAI	68.3
36	gpt-oss-120b (free)OpenAI	67.7
37	gpt-oss-120bOpenAI	67.7
38	gpt-oss-20b (free)OpenAI	67.7
39	gpt-oss-20bOpenAI	67.7
40	Grok 3xAI	67.7
41	Grok 3 BetaxAI	67.7
42	GPT-5 MiniOpenAI	66.7
43	Qwen3 MaxAlibaba Qwen	63.5
44	DeepSeek-R1DeepSeek	62.3
45	Claude Haiku 4.5Anthropic	61.6
46	Qwen3 235B A22BAlibaba Qwen	60.9
47	GPT-5 NanoOpenAI	59.3
48	GPT-4.5OpenAI	58.3
49	Llama 4 MaverickMeta	56.0
50	GPT-4.1OpenAI	55.9
51	GPT-4.1 MiniOpenAI	54.5
52	Gemini 2.0 ProGoogle	54.2
53	o1-miniOpenAI	49.8
54	Mistral Medium 3Mistral AI	46.0
55	Gemini 1.5 Pro (Sept 2024)Google	43.0
56	Gemini 2.0 Flash Thinking (Jan 2025)Google	42.8
57	DeepSeek V3DeepSeek	42.0
58	Qwen2.5-MaxAlibaba	41.5
59	Phi 4Microsoft	41.4
60	Claude 3.5 SonnetAnthropic	38.7
61	Grok-2 (Dec 2024)xAI	38.4
62	Llama 4 ScoutMeta	35.8
63	Mistral Large 2411Mistral AI	35.1
64	Llama 3.1-405BMeta	34.5
65	o1-previewOpenAI	33.8
66	GPT-4o (2024-08-06)OpenAI	32.3
67	GPT-4o (2024-05-13)OpenAI	32.3
68	GPT-4oOpenAI	32.3
69	GPT-4o (extended)OpenAI	32.3
70	Qwen2.5 72B InstructAlibaba Qwen	32.2
71	Mistral Large 2407Mistral AI	32.0
72	GPT-4.1 NanoOpenAI	31.9
73	Gemma 3 4B (free)Google DeepMind	31.8
74	Gemma 3 4BGoogle DeepMind	31.8
75	Gemma 3 12B (free)Google DeepMind	31.8
76	Gemma 3 12BGoogle DeepMind	31.8
77	Gemma 3 27B (free)Google DeepMind	31.8
78	Gemma 3 27BGoogle DeepMind	31.8
79	GPT-4o (2024-11-20)OpenAI	30.5
80	Llama 3.3 70B Instruct (free)Meta	29.9
81	Llama 3.3 70B InstructMeta	29.9
82	Gemini 1.5 Flash (Sep 2024)Google	29.8
83	Claude 3 OpusAnthropic	29.6
84	GPT-4 Turbo PreviewOpenAI	28.8
85	Gemini 1.5 Pro (Feb 2024)Google	27.8
86	Llama 3.1 70B InstructMeta	25.6
87	Llama 3.2 90BMeta	21.4
88	Qwen2-72BAlibaba	21.0
89	Claude 3 SonnetAnthropic	20.8
90	Llama 3 70B InstructMeta	20.8
91	Gemini 1.5 Flash (May 2024)Google	20.5
92	Mistral LargeMistral AI	18.4
93	Claude 3.5 HaikuAnthropic	17.5
94	GPT-4o-mini (2024-07-18)OpenAI	17.0
95	GPT-4o-miniOpenAI	17.0
96	Gemma 2 27BGoogle DeepMind	15.3
97	Claude 3 HaikuAnthropic	15.1
98	Claude 2Anthropic	12.9
99	Mixtral 8x22B InstructMistral AI	12.1
100	Gemini 1.0 ProGoogle	11.9
101	Claude 2.1Anthropic	10.6
102	GPT-4 TurboOpenAI	7.5
103	GPT-4 Turbo (older v1106)OpenAI	7.5
104	GPT-4 (older v0314)OpenAI	7.5
105	GPT-4OpenAI	7.5
106	Mixtral 8x7B InstructMistral AI	7.5
107	Mistral NemoMistral AI	6.5
108	phi-3-medium 14BMicrosoft	3.5
109	Gemma 2 9BGoogle DeepMind	3.3
110	GPT-3.5 Turbo (older v0613)OpenAI	2.9
111	GPT-3.5 Turbo InstructOpenAI	2.9
112	GPT-3.5 TurboOpenAI	2.9
113	Llama 2-70BMeta	1.8
114	Llama 3 8B InstructMeta	1.4
115	Llama 3.1 8B InstructMeta	1.3

Related Benchmarks