Compare · ModelsLive · 2 picked · head to head
Phi 3 Mini 4k Instruct vs Gemma 2 9B
Side by side · benchmarks, pricing, and signals you can act on.
Winner summary
Gemma 2 9B wins on 5/7 benchmarks
Gemma 2 9B wins 5 of 7 shared benchmarks. Leads in arena · general · knowledge.
Category leads
arena·Gemma 2 9Bgeneral·Gemma 2 9Bknowledge·Gemma 2 9Blanguage·Gemma 2 9Bmath·Gemma 2 9Breasoning·Phi 3 Mini 4k Instruct
Hype vs Reality
Attention vs performance
Phi 3 Mini 4k Instruct
#233 by perf·no signal
Gemma 2 9B
#193 by perf·no signal
Vendor risk
Who is behind the model
Microsoft
$3.00T·Big Tech
Google DeepMind
$4.00T·Tier 1
Head to head
7 benchmarks · 2 models
Phi 3 Mini 4k InstructGemma 2 9B
Chatbot Arena Elo · Overall
Gemma 2 9B leads by +137.6
Phi 3 Mini 4k Instruct
1127.4
Gemma 2 9B
1265.0
BBH (HuggingFace)
Gemma 2 9B leads by +5.6
Phi 3 Mini 4k Instruct
36.6
Gemma 2 9B
42.1
GPQA
Gemma 2 9B leads by +3.8
Phi 3 Mini 4k Instruct
11.0
Gemma 2 9B
14.8
IFEval
Gemma 2 9B leads by +19.6
Phi 3 Mini 4k Instruct
54.8
Gemma 2 9B
74.4
MATH Level 5
Gemma 2 9B leads by +3.1
Phi 3 Mini 4k Instruct
16.4
Gemma 2 9B
19.5
MMLU-PRO
Phi 3 Mini 4k Instruct leads by +1.6
Phi 3 Mini 4k Instruct
33.6
Gemma 2 9B
31.9
MUSR
Phi 3 Mini 4k Instruct leads by +3.4
Phi 3 Mini 4k Instruct
13.1
Gemma 2 9B
9.7
Full benchmark table
| Benchmark | Phi 3 Mini 4k Instruct | Gemma 2 9B |
|---|---|---|
Chatbot Arena Elo · Overall | 1127.4 | 1265.0 |
BBH (HuggingFace) | 36.6 | 42.1 |
GPQA | 11.0 | 14.8 |
IFEval | 54.8 | 74.4 |
MATH Level 5 | 16.4 | 19.5 |
MMLU-PRO | 33.6 | 31.9 |
MUSR | 13.1 | 9.7 |
Pricing · per 1M tokens · projected $/mo at 10M tokens
| Model | Input | Output | Context | Projected $/mo |
|---|---|---|---|---|
| — | — | — | — | |
| $0.03 | $0.09 | 8K tokens (~4 books) | $0.45 |