Compare · ModelsLive · 2 picked · head to head
StarCoder 2 15B vs Meta Llama 3 8B Instruct
Side by side · benchmarks, pricing, and signals you can act on.
Winner summary
Meta Llama 3 8B Instruct wins on 4/6 benchmarks
Meta Llama 3 8B Instruct wins 4 of 6 shared benchmarks. Leads in general · language · math.
Category leads
general·Meta Llama 3 8B Instructknowledge·StarCoder 2 15Blanguage·Meta Llama 3 8B Instructmath·Meta Llama 3 8B Instructreasoning·StarCoder 2 15B
Hype vs Reality
Attention vs performance
StarCoder 2 15B
#204 by perf·no signal
Meta Llama 3 8B Instruct
#115 by perf·no signal
Vendor risk
Who is behind the model
U
Unknown
private · undisclosed
Meta AI
$1.50T·Tier 1
Head to head
6 benchmarks · 2 models
StarCoder 2 15BMeta Llama 3 8B Instruct
BBH (HuggingFace)
Meta Llama 3 8B Instruct leads by +7.9
StarCoder 2 15B
20.4
Meta Llama 3 8B Instruct
28.2
GPQA
StarCoder 2 15B leads by +1.9
StarCoder 2 15B
3.1
Meta Llama 3 8B Instruct
1.2
IFEval
Meta Llama 3 8B Instruct leads by +46.3
StarCoder 2 15B
27.8
Meta Llama 3 8B Instruct
74.1
MATH Level 5
Meta Llama 3 8B Instruct leads by +2.7
StarCoder 2 15B
6.0
Meta Llama 3 8B Instruct
8.7
MMLU-PRO
Meta Llama 3 8B Instruct leads by +14.6
StarCoder 2 15B
15.0
Meta Llama 3 8B Instruct
29.6
MUSR
StarCoder 2 15B leads by +1.3
StarCoder 2 15B
2.9
Meta Llama 3 8B Instruct
1.6
Full benchmark table
| Benchmark | StarCoder 2 15B | Meta Llama 3 8B Instruct |
|---|---|---|
BBH (HuggingFace) | 20.4 | 28.2 |
GPQA | 3.1 | 1.2 |
IFEval | 27.8 | 74.1 |
MATH Level 5 | 6.0 | 8.7 |
MMLU-PRO | 15.0 | 29.6 |
MUSR | 2.9 | 1.6 |
Pricing · per 1M tokens · projected $/mo at 10M tokens
| Model | Input | Output | Context | Projected $/mo |
|---|---|---|---|---|
U StarCoder 2 15B | — | — | — | — |
| — | — | — | — |