Which model leads on WeirdML?

GPT-6 Astra from OpenAI leads WeirdML with a score of 93.6. The median score across 120 tested models is 43.8.

Is WeirdML saturated?

No · the top score is 93.6 out of 100 (94%). There is still meaningful room for improvement on WeirdML.

Does WeirdML predict performance on other benchmarks?

Yes · WeirdML scores correlate 0.96 with Professional Reasoning · Legal across 5 shared models. Models that do well on WeirdML tend to do well on Professional Reasoning · Legal.

How often is WeirdML data refreshed?

BenchGecko pulls updates daily. New model scores on WeirdML appear as soon as they are published by Epoch AI or the model provider.

Benchmark · CodeSettled

WeirdML

Name: WeirdML Benchmark
Creator: BenchGecko
License: https://creativecommons.org/licenses/by/4.0/

WeirdML · tests models on unusual and adversarial machine learning tasks that require creative problem-solving beyond standard patterns.

Updated 2026-09-04

Models tested

120

Top score

93.6

GPT-6 Astra

Median

43.8

min 1.7

Top-5 spread

σ 1.4

Settled

The Frontier

Best score over time · one chart, every benchmark

Chart type

Frontier on WeirdML rose from 25.1 to 93.6 in 22 months · +68.4 points · latest leader GPT-6 Astra from OpenAI.

Pink dots = frontier records · 14 totalClick to open model page

Full rankings

120 models tested · sorted by score

#	Model	Score	Price
1	GPT-6 Astra· OpenAI	93.6	$10.00
2	Claude Fable 5.1· Anthropic	92.9	$10.00
3	Claude Fable 5· Anthropic	91.9	$10.00
4	Claude Opus 5· Anthropic	91.8	$5.00
5	GPT-5.6 Sol· OpenAI	89.4	$4.00
6	GPT-5.5· OpenAI	84.9	$5.00
7	Gemini 3.8 Flash· Google DeepMind	84.8	$0.82
8	Claude Opus 4.8· Anthropic	82.9	$5.00
9	Kimi K3· moonshotai	82.6	$0.19
10	GPT-5.3-Codex· OpenAI	79.3	$1.75
11	GPT-5.6 Terra· OpenAI	78.3	$2.00
12	Claude Opus 4.6· Anthropic	77.9	$5.00
13	Claude Opus 4.7· Anthropic	76.4	$5.00
14	GLM 5.3· z-ai	75.4	$0.07
15	GPT-5.2· OpenAI	72.2	$1.75
16	Gemini 3.1 Pro Preview· Google DeepMind	72.1	$2.00
17	GLM 5.2· z-ai	70.1	$0.01
18	Gemini 3 Pro· Google DeepMind	69.9	n/a
19	Claude Sonnet 5· Anthropic	68.8	$2.00
20	Grok 4.6· xAI	67.3	$2.00
21	DeepSeek V4 Pro 0813· DeepSeek	66.2	$0.11
22	Claude Sonnet 4.6· Anthropic	66.1	$3.00
23	Claude Opus 4.5· Anthropic	63.7	$5.00
24	DeepSeek V4 Flash 0731· DeepSeek	63.0	$0.04
25	Gemini 3.5 Flash· Google DeepMind	62.6	$1.50
26	Gemini 3 Flash Preview· Google DeepMind	61.6	$0.50
27	GPT-5.6 Luna· OpenAI	60.9	$0.20
28	GPT-5.1· OpenAI	60.8	$1.25
29	GPT-5· OpenAI	60.7	$1.25
30	GPT-5 Pro· OpenAI	60.4	$15.00
31	GPT-5.4 Mini· OpenAI	60.3	$0.75
32	o3 Pro· OpenAI	58.2	$20.00
33	GPT-5.4· OpenAI	57.4	$2.50
34	GPT-5.4 Pro· OpenAI	57.4	$30.00
35	GLM 5.1· z-ai	57.1	$1.33
36	Gemini 3.6 Flash· Google DeepMind	56.1	$0.75
37	Kimi K2.6· moonshotai	55.9	$0.65
38	Kimi K2.7 Code· moonshotai	54.1	$0.60
39	Gemini 2.5 Pro· Google DeepMind	54.0	$1.25
40	GPT-5 Mini· OpenAI	52.7	$0.25
41	o4 Mini· OpenAI	52.6	$1.10
42	o4 Mini (batch)· OpenAI	52.6	$0.55
43	o3· OpenAI	52.4	$2.00
44	Grok 4.20· xAI	52.3	$1.25
45	Gemma 4 31B· Google DeepMind	52.3	$0.20
46	Gemini 3.1 Flash Lite· Google DeepMind	52.2	$0.25
47	Grok 4.3· xAI	49.9	$1.25
48	GPT-5.4 Nano· OpenAI	49.2	$0.20
49	DeepSeek V4 Pro· DeepSeek	48.9	$0.88
50	gpt-oss-120b· OpenAI	48.2	$0.15
51	GLM 5· z-ai	48.2	$1.00
52	Claude Sonnet 4.5· Anthropic	47.7	$3.00
53	o1-preview· OpenAI	47.6	n/a
54	Grok 4.5· xAI	46.4	$2.00
55	Claude Sonnet 4· Anthropic	46.1	$3.00
56	Grok 4· xAI	45.7	n/a
57	DeepSeek V4 Flash· DeepSeek	45.6	$0.09
58	Kimi K2.5· moonshotai	45.6	$0.53
59	Claude Haiku 4.5· Anthropic	45.4	$1.00
60	o1· OpenAI	43.8	$15.00
61	Claude Opus 4· Anthropic	43.7	$15.00
62	Mistral Medium 3.5· Mistral AI	43.7	$1.50
63	o3 Mini (batch)· OpenAI	43.7	$0.55
64	o3 Mini· OpenAI	43.7	$1.10
65	Nemotron 3 Ultra· NVIDIA	43.5	$0.60
66	Grok 4 Fast· xAI	42.9	n/a
67	Kimi K2 Thinking· moonshotai	42.8	$0.60
68	Claude Opus 4.1· Anthropic	42.8	$15.00
69	Grok-3 mini· xAI	42.6	n/a
70	R1 0528· DeepSeek	41.6	$0.50
71	Qwen3 235B A22B Thinking 2507· Alibaba Qwen	41.0	$0.45
72	Gemini 2.5 Flash· Google DeepMind	41.0	$0.30
73	gpt-oss-20b· OpenAI	40.9	$0.03
74	DeepSeek V3.2 Exp· DeepSeek	39.5	$0.27
75	GPT-4.5· OpenAI	39.4	n/a
76	Kimi K2 0711· moonshotai	39.4	$0.57
77	GPT-4.1· OpenAI	39.0	$2.00
78	Gemini 3.5 Flash Lite· Google DeepMind	39.0	$0.30
79	Qwen3 235B A22B Instruct 2507· Alibaba Qwen	38.7	$0.14
80	DeepSeek V3.1· DeepSeek	38.4	$0.25
81	GPT-5 Nano· OpenAI	38.1	$0.05
82	GPT-4.1 Mini· OpenAI	37.6	$0.40
83	Qwen3 235B A22B· Alibaba Qwen	37.3	$0.46
84	Grok 3· xAI	37.2	n/a
85	MiniMax M2.7· minimax	37.0	$0.21
86	R1· DeepSeek	36.5	$0.70
87	o1-mini· OpenAI	36.3	n/a
88	DeepSeek V3· DeepSeek	36.1	$0.32
89	Gemini 2.5 Flash Lite· Google DeepMind	35.2	$0.10
90	Gemma 4 26B A4B · Google DeepMind	35.2	$0.08
91	Qwen3.6 35B A3B· Alibaba Qwen	34.5	$0.10
92	Claude 3.5 Sonnet· Anthropic	31.0	n/a
93	Claude 3.5 Haiku· Anthropic	30.7	n/a
94	Qwen3 30B A3B· Alibaba Qwen	29.8	$0.12
95	Gemini 2.0 Flash (Dec 2024)· Google DeepMind	25.8	n/a
96	Gemini 2.0 Flash· Google DeepMind	25.8	$0.10
97	GPT-4o (2024-11-20)· OpenAI	25.1	$2.50
98	Gemini 1.5 Flash (May 2024)· Google DeepMind	24.9	n/a
99	Llama 4 Maverick· Meta	24.5	$0.35
100	Grok-2 (Dec 2024)· xAI	22.2	n/a
101	Gemini 1.5 Pro (Feb 2024)· Google DeepMind	22.2	n/a
102	Gemini 1.5 Pro (May 2024)· Google DeepMind	22.2	n/a
103	Llama 3.1 405B· Meta	21.4	n/a
104	Claude 3 Opus· Anthropic	19.2	n/a
105	GPT-4.1 Nano· OpenAI	19.0	$0.10
106	Qwen2.5 72B Instruct· Alibaba Qwen	16.0	$0.36
107	Llama 3.3 70B Instruct· Meta	14.4	$0.71
108	Llama 3.3 70B Instruct (free)· Meta	14.4	$0.00
109	GPT-4 Turbo (batch)· OpenAI	12.4	$5.00
110	GPT-4 Turbo· OpenAI	12.4	$10.00
111	GPT-4o-mini (2024-07-18)· OpenAI	11.8	$0.15
112	GPT-4o-mini· OpenAI	11.8	$0.15
113	Qwen2-72B· Alibaba Qwen	11.3	n/a
114	Claude 3 Sonnet· Anthropic	10.2	n/a
115	Claude 3 Haiku· Anthropic	9.8	$0.25
116	Llama 3.1 70B Instruct· Meta	9.0	$0.72
117	Claude 2.1· Anthropic	7.1	n/a
118	GPT-3.5 Turbo (older v0613)· OpenAI	3.5	$1.00
119	Mixtral 8x22B Instruct· Mistral AI	3.2	$2.00
120	Llama 3.1 8B Instruct· Meta	1.7	$0.02