Compare · ModelsLive · 2 picked · head to head
StarCoder 2 15B vs vicuna-13b-v1.1
Side by side · benchmarks, pricing, and signals you can act on.
Winner summary
StarCoder 2 15B wins on 2/3 benchmarks
StarCoder 2 15B wins 2 of 3 shared benchmarks. Leads in knowledge · math.
Category leads
knowledge·StarCoder 2 15Bmath·StarCoder 2 15B
Hype vs Reality
Attention vs performance
StarCoder 2 15B
#240 by perf·no signal
vicuna-13b-v1.1
#212 by perf·no signal
Vendor risk
Who is behind the model
U
Unknown
private · undisclosed
U
Unknown
private · undisclosed
Head to head
3 benchmarks · 2 models
StarCoder 2 15Bvicuna-13b-v1.1
ARC AI2
StarCoder 2 15B leads by +5.3
AI2 Reasoning Challenge · tests grade-school level science knowledge with multiple-choice questions requiring reasoning beyond simple retrieval.
StarCoder 2 15B
29.6
vicuna-13b-v1.1
24.3
GSM8K
StarCoder 2 15B leads by +29.6
Grade School Math 8K · 8,500 linguistically diverse grade-school math word problems that require multi-step reasoning to solve.
StarCoder 2 15B
57.7
vicuna-13b-v1.1
28.1
Winogrande
vicuna-13b-v1.1 leads by +13.0
WinoGrande · large-scale commonsense reasoning benchmark where models must resolve ambiguous pronouns in carefully constructed sentence pairs.
StarCoder 2 15B
28.6
vicuna-13b-v1.1
41.6
Full benchmark table
| Benchmark | StarCoder 2 15B | vicuna-13b-v1.1 |
|---|---|---|
ARC AI2 AI2 Reasoning Challenge · tests grade-school level science knowledge with multiple-choice questions requiring reasoning beyond simple retrieval. | 29.6 | 24.3 |
GSM8K Grade School Math 8K · 8,500 linguistically diverse grade-school math word problems that require multi-step reasoning to solve. | 57.7 | 28.1 |
Winogrande WinoGrande · large-scale commonsense reasoning benchmark where models must resolve ambiguous pronouns in carefully constructed sentence pairs. | 28.6 | 41.6 |
Pricing · per 1M tokens · projected $/mo at 10M tokens
| Model | Input | Output | Context | Projected $/mo |
|---|---|---|---|---|
U StarCoder 2 15B | — | — | — | — |
U vicuna-13b-v1.1 | — | — | — | — |