Compare · ModelsLive · 2 picked · head to head
Grok 3 vs Grok 4
Side by side · benchmarks, pricing, and signals you can act on.
Winner summary
Grok 4 wins on 17/18 benchmarks
Grok 4 wins 17 of 18 shared benchmarks. Leads in coding · agentic · reasoning.
Category leads
coding·Grok 4agentic·Grok 4reasoning·Grok 4knowledge·Grok 4math·Grok 4language·Grok 4
Head to head · from the data · data as of 2026-11-15
- Benchmarks · Grok 3 leads on 1 of 18 shared benchmarks (HELM · WildBench), Grok 4 on 17 (Aider polyglot, APEX-Agents, ARC-AGI, ARC-AGI-2, 13 more)
- Price history · Grok 3 OpenRouter input list price had no move of 5% or more in the 90 days to 2026-11-15 · Grok 4 OpenRouter input list price had no move of 5% or more in the 90 days to 2026-11-15
Sources · scores from public leaderboards, list prices and provider offers from OpenRouter, price moves from the BenchGecko changelog (5% or more, held in the next daily check), Gecko tests run by BenchGecko.
Hype vs Reality
Attention vs performance
Grok 3
#171 by perf·no signal
Grok 4
#98 by perf·#16 by attention
Vendor risk
Who is behind the model
xAI
$250.0B·Tier 1
xAI
$250.0B·Tier 1
Head to head
18 benchmarks · 2 models
Grok 3Grok 4
Aider polyglot
Grok 4 leads by +26.3
Aider Polyglot · measures how well AI models can edit code across multiple programming languages using the Aider coding assistant framework.
Grok 3
53.3
Grok 4
79.6
APEX-Agents
Grok 4 leads by +13.1
APEX-Agents · evaluates AI agents on complex, multi-step tasks requiring planning, tool use, and autonomous decision-making in realistic environments.
Grok 3
2.1
Grok 4
15.2
ARC-AGI
Grok 4 leads by +61.2
ARC-AGI · the original Abstraction and Reasoning Corpus, testing whether AI can solve novel visual pattern recognition tasks without memorization.
Grok 3
5.5
Grok 4
66.7
ARC-AGI-2
Grok 4 leads by +16.0
ARC-AGI-2 · the second iteration of the Abstraction and Reasoning Corpus, testing novel pattern recognition and abstract reasoning without prior training data.
Grok 3
0.0
Grok 4
16.0
Balrog
Grok 4 leads by +14.1
Balrog · benchmarks AI agents on text-based adventure games, testing language understanding, strategic planning, and long-horizon reasoning.
Grok 3
29.5
Grok 4
43.6
Fiction.LiveBench
Grok 4 leads by +36.1
Fiction.LiveBench · a continuously updated benchmark using recently published fiction to test reading comprehension and reasoning, preventing data contamination.
Grok 3
58.3
Grok 4
94.4
FrontierMath-2025-02-28-Private
Grok 4 leads by +15.9
FrontierMath (Feb 2025) · original research-level math problems created by mathematicians, testing capabilities at the boundary of current AI mathematical reasoning.
Grok 3
3.8
Grok 4
19.7
FrontierMath-Tier-4-2025-07-01-Private
Grok 4 leads by +2.0
FrontierMath Tier 4 (Jul 2025) · the most challenging tier of frontier mathematics, containing problems that push the absolute limits of AI mathematical reasoning.
Grok 3
0.1
Grok 4
2.1
GPQA diamond
Grok 4 leads by +18.4
Graduate-Level Google-Proof QA (Diamond set) · expert-crafted questions in physics, biology, and chemistry that are difficult even for domain PhDs.
Grok 3
67.7
Grok 4
86.1
HELM · GPQA
Grok 4 leads by +7.6
Grok 3
65.0
Grok 4
72.6
HELM · IFEval
Grok 4 leads by +6.5
Grok 3
88.4
Grok 4
94.9
HELM · MMLU-Pro
Grok 4 leads by +6.3
Grok 3
78.8
Grok 4
85.1
HELM · Omni-MATH
Grok 4 leads by +13.9
Grok 3
46.4
Grok 4
60.3
HELM · WildBench
Grok 3 leads by +5.2
Grok 3
84.9
Grok 4
79.7
Lech Mazur Writing
Grok 4 leads by +4.3
Lech Mazur Writing · evaluates creative writing ability, assessing prose quality, narrative coherence, and stylistic sophistication.
Grok 3
76.4
Grok 4
80.7
OTIS Mock AIME 2024-2025
Grok 4 leads by +28.5
OTIS Mock AIME 2024-2025 · simulated American Invitational Mathematics Examination problems testing advanced problem-solving skills.
Grok 3
55.5
Grok 4
84.0
SimpleBench
Grok 4 leads by +29.3
SimpleBench · tests fundamental reasoning capabilities with straightforward problems designed to expose gaps in basic logical and spatial thinking.
Grok 3
23.3
Grok 4
52.6
WeirdML
Grok 4 leads by +8.5
WeirdML · tests models on unusual and adversarial machine learning tasks that require creative problem-solving beyond standard patterns.
Grok 3
37.2
Grok 4
45.7
Full benchmark table
| Benchmark | Grok 3 | Grok 4 |
|---|---|---|
Aider polyglot Aider Polyglot · measures how well AI models can edit code across multiple programming languages using the Aider coding assistant framework. | 53.3 | 79.6 |
APEX-Agents APEX-Agents · evaluates AI agents on complex, multi-step tasks requiring planning, tool use, and autonomous decision-making in realistic environments. | 2.1 | 15.2 |
ARC-AGI ARC-AGI · the original Abstraction and Reasoning Corpus, testing whether AI can solve novel visual pattern recognition tasks without memorization. | 5.5 | 66.7 |
ARC-AGI-2 ARC-AGI-2 · the second iteration of the Abstraction and Reasoning Corpus, testing novel pattern recognition and abstract reasoning without prior training data. | 0.0 | 16.0 |
Balrog Balrog · benchmarks AI agents on text-based adventure games, testing language understanding, strategic planning, and long-horizon reasoning. | 29.5 | 43.6 |
Fiction.LiveBench Fiction.LiveBench · a continuously updated benchmark using recently published fiction to test reading comprehension and reasoning, preventing data contamination. | 58.3 | 94.4 |
FrontierMath-2025-02-28-Private FrontierMath (Feb 2025) · original research-level math problems created by mathematicians, testing capabilities at the boundary of current AI mathematical reasoning. | 3.8 | 19.7 |
FrontierMath-Tier-4-2025-07-01-Private FrontierMath Tier 4 (Jul 2025) · the most challenging tier of frontier mathematics, containing problems that push the absolute limits of AI mathematical reasoning. | 0.1 | 2.1 |
GPQA diamond Graduate-Level Google-Proof QA (Diamond set) · expert-crafted questions in physics, biology, and chemistry that are difficult even for domain PhDs. | 67.7 | 86.1 |
HELM · GPQA | 65.0 | 72.6 |
HELM · IFEval | 88.4 | 94.9 |
HELM · MMLU-Pro | 78.8 | 85.1 |
HELM · Omni-MATH | 46.4 | 60.3 |
HELM · WildBench | 84.9 | 79.7 |
Lech Mazur Writing Lech Mazur Writing · evaluates creative writing ability, assessing prose quality, narrative coherence, and stylistic sophistication. | 76.4 | 80.7 |
OTIS Mock AIME 2024-2025 OTIS Mock AIME 2024-2025 · simulated American Invitational Mathematics Examination problems testing advanced problem-solving skills. | 55.5 | 84.0 |
SimpleBench SimpleBench · tests fundamental reasoning capabilities with straightforward problems designed to expose gaps in basic logical and spatial thinking. | 23.3 | 52.6 |
WeirdML WeirdML · tests models on unusual and adversarial machine learning tasks that require creative problem-solving beyond standard patterns. | 37.2 | 45.7 |
Pricing · per 1M tokens · projected $/mo at 10M tokens
Grok 3 vs Grok 4 · questions answered from the data
Grok 3 leads on 1 of 18 shared benchmarks (HELM · WildBench), Grok 4 on 17 (Aider polyglot, APEX-Agents, ARC-AGI, ARC-AGI-2, 13 more)
People also compared