Compare · ModelsLive · 2 picked · head to head
Gemini 1.5 Flash (May 2024) vs gpt-oss-120b
Side by side · benchmarks, pricing, and signals you can act on.
Winner summary
gpt-oss-120b wins on 9/9 benchmarks
gpt-oss-120b wins 9 of 9 shared benchmarks. Leads in arena · knowledge · language.
Category leads
arena·gpt-oss-120bknowledge·gpt-oss-120blanguage·gpt-oss-120bmath·gpt-oss-120breasoning·gpt-oss-120bcoding·gpt-oss-120b
Hype vs Reality
Attention vs performance
Gemini 1.5 Flash (May 2024)
#105 by perf·no signal
gpt-oss-120b
#108 by perf·no signal
Best value
gpt-oss-120b
Gemini 1.5 Flash (May 2024)
—
no price
gpt-oss-120b
428.3 pts/$
$0.11/M
Vendor risk
Who is behind the model
Google DeepMind
$4.00T·Tier 1
OpenAI
$840.0B·Tier 1
Head to head
9 benchmarks · 2 models
Gemini 1.5 Flash (May 2024)gpt-oss-120b
Chatbot Arena Elo · Overall
gpt-oss-120b leads by +68.7
Gemini 1.5 Flash (May 2024)
1285.1
gpt-oss-120b
1353.8
GPQA diamond
gpt-oss-120b leads by +47.2
Graduate-Level Google-Proof QA (Diamond set) · expert-crafted questions in physics, biology, and chemistry that are difficult even for domain PhDs.
Gemini 1.5 Flash (May 2024)
20.5
gpt-oss-120b
67.7
HELM · GPQA
gpt-oss-120b leads by +24.7
Gemini 1.5 Flash (May 2024)
43.7
gpt-oss-120b
68.4
HELM · IFEval
gpt-oss-120b leads by +0.5
Gemini 1.5 Flash (May 2024)
83.1
gpt-oss-120b
83.6
HELM · MMLU-Pro
gpt-oss-120b leads by +11.7
Gemini 1.5 Flash (May 2024)
67.8
gpt-oss-120b
79.5
HELM · Omni-MATH
gpt-oss-120b leads by +38.3
Gemini 1.5 Flash (May 2024)
30.5
gpt-oss-120b
68.8
HELM · WildBench
gpt-oss-120b leads by +5.3
Gemini 1.5 Flash (May 2024)
79.2
gpt-oss-120b
84.5
OTIS Mock AIME 2024-2025
gpt-oss-120b leads by +85.1
OTIS Mock AIME 2024-2025 · simulated American Invitational Mathematics Examination problems testing advanced problem-solving skills.
Gemini 1.5 Flash (May 2024)
3.8
gpt-oss-120b
88.9
WeirdML
gpt-oss-120b leads by +23.3
WeirdML · tests models on unusual and adversarial machine learning tasks that require creative problem-solving beyond standard patterns.
Gemini 1.5 Flash (May 2024)
24.9
gpt-oss-120b
48.2
Full benchmark table
| Benchmark | Gemini 1.5 Flash (May 2024) | gpt-oss-120b |
|---|---|---|
Chatbot Arena Elo · Overall | 1285.1 | 1353.8 |
GPQA diamond Graduate-Level Google-Proof QA (Diamond set) · expert-crafted questions in physics, biology, and chemistry that are difficult even for domain PhDs. | 20.5 | 67.7 |
HELM · GPQA | 43.7 | 68.4 |
HELM · IFEval | 83.1 | 83.6 |
HELM · MMLU-Pro | 67.8 | 79.5 |
HELM · Omni-MATH | 30.5 | 68.8 |
HELM · WildBench | 79.2 | 84.5 |
OTIS Mock AIME 2024-2025 OTIS Mock AIME 2024-2025 · simulated American Invitational Mathematics Examination problems testing advanced problem-solving skills. | 3.8 | 88.9 |
WeirdML WeirdML · tests models on unusual and adversarial machine learning tasks that require creative problem-solving beyond standard patterns. | 24.9 | 48.2 |
Pricing · per 1M tokens · projected $/mo at 10M tokens
| Model | Input | Output | Context | Projected $/mo |
|---|---|---|---|---|
| — | — | — | — | |
| $0.04 | $0.18 | 131K tokens (~66 books) | $0.74 |