Compare · ModelsLive · 2 picked · head to head
Meta Llama 3 8B Instruct vs Qwen2 7B Instruct
Side by side · benchmarks, pricing, and signals you can act on.
Winner summary
Qwen2 7B Instruct wins on 22/25 benchmarks
Qwen2 7B Instruct wins 22 of 25 shared benchmarks. Leads in general · knowledge · language.
Category leads
general·Qwen2 7B Instructknowledge·Qwen2 7B Instructlanguage·Qwen2 7B Instructmath·Qwen2 7B Instructreasoning·Qwen2 7B Instruct
Hype vs Reality
Attention vs performance
Meta Llama 3 8B Instruct
#113 by perf·no signal
Qwen2 7B Instruct
#92 by perf·no signal
Best value
Pricing unknown
Meta Llama 3 8B Instruct
—
no price
Qwen2 7B Instruct
—
no price
Vendor risk
Who is behind the model
Meta AI
$1.50T·Tier 1
Alibaba (Qwen)
$293.0B·Tier 1
Head to head
25 benchmarks · 2 models
Meta Llama 3 8B InstructQwen2 7B Instruct
BBH (HuggingFace)
Qwen2 7B Instruct leads by +9.6
Meta Llama 3 8B Instruct
28.2
Qwen2 7B Instruct
37.8
GPQA
Qwen2 7B Instruct leads by +5.2
Meta Llama 3 8B Instruct
1.2
Qwen2 7B Instruct
6.4
IFEval
Meta Llama 3 8B Instruct leads by +17.3
Meta Llama 3 8B Instruct
74.1
Qwen2 7B Instruct
56.8
MATH Level 5
Qwen2 7B Instruct leads by +19.0
Meta Llama 3 8B Instruct
8.7
Qwen2 7B Instruct
27.6
MMLU-PRO
Qwen2 7B Instruct leads by +2.0
Meta Llama 3 8B Instruct
29.6
Qwen2 7B Instruct
31.6
MUSR
Qwen2 7B Instruct leads by +5.8
Meta Llama 3 8B Instruct
1.6
Qwen2 7B Instruct
7.4
JCommonsenseQA
Qwen2 7B Instruct leads by +1.4
Meta Llama 3 8B Instruct
87.7
Qwen2 7B Instruct
89.1
JMMLU
Qwen2 7B Instruct leads by +9.8
Meta Llama 3 8B Instruct
46.7
Qwen2 7B Instruct
56.5
JNLI
Qwen2 7B Instruct leads by +20.2
Meta Llama 3 8B Instruct
61.1
Qwen2 7B Instruct
81.3
JSQuAD
Qwen2 7B Instruct leads by +0.1
Meta Llama 3 8B Instruct
89.5
Qwen2 7B Instruct
89.6
LLM-JP · Overall
Qwen2 7B Instruct leads by +2.1
Meta Llama 3 8B Instruct
49.6
Qwen2 7B Instruct
51.7
MMMLU · Arabic
Qwen2 7B Instruct leads by +10.2
Meta Llama 3 8B Instruct
40.5
Qwen2 7B Instruct
50.7
MMMLU · Bengali
Qwen2 7B Instruct leads by +7.0
Meta Llama 3 8B Instruct
36.4
Qwen2 7B Instruct
43.4
MMMLU · German
Qwen2 7B Instruct leads by +3.6
Meta Llama 3 8B Instruct
53.5
Qwen2 7B Instruct
57.1
MMMLU · Spanish
Qwen2 7B Instruct leads by +4.4
Meta Llama 3 8B Instruct
55.8
Qwen2 7B Instruct
60.2
MMMLU · French
Qwen2 7B Instruct leads by +5.0
Meta Llama 3 8B Instruct
55.8
Qwen2 7B Instruct
60.8
MMMLU · Hindi
Qwen2 7B Instruct leads by +3.7
Meta Llama 3 8B Instruct
41.4
Qwen2 7B Instruct
45.1
MMMLU · Indonesian
Qwen2 7B Instruct leads by +3.1
Meta Llama 3 8B Instruct
51.0
Qwen2 7B Instruct
54.1
MMMLU · Italian
Qwen2 7B Instruct leads by +5.7
Meta Llama 3 8B Instruct
53.3
Qwen2 7B Instruct
59.0
MMMLU · Japanese
Qwen2 7B Instruct leads by +14.3
Meta Llama 3 8B Instruct
42.3
Qwen2 7B Instruct
56.6
MMMLU · Korean
Qwen2 7B Instruct leads by +7.5
Meta Llama 3 8B Instruct
46.5
Qwen2 7B Instruct
54.0
MMMLU · Portuguese
Qwen2 7B Instruct leads by +4.6
Meta Llama 3 8B Instruct
55.5
Qwen2 7B Instruct
60.1
MMMLU · Swahili
Meta Llama 3 8B Instruct leads by +3.2
Meta Llama 3 8B Instruct
37.5
Qwen2 7B Instruct
34.3
MMMLU · Yoruba
Meta Llama 3 8B Instruct leads by +0.8
Meta Llama 3 8B Instruct
31.0
Qwen2 7B Instruct
30.2
MMMLU · Chinese
Qwen2 7B Instruct leads by +10.4
Meta Llama 3 8B Instruct
51.4
Qwen2 7B Instruct
61.8
Full benchmark table
| Benchmark | Meta Llama 3 8B Instruct | Qwen2 7B Instruct |
|---|---|---|
BBH (HuggingFace) | 28.2 | 37.8 |
GPQA | 1.2 | 6.4 |
IFEval | 74.1 | 56.8 |
MATH Level 5 | 8.7 | 27.6 |
MMLU-PRO | 29.6 | 31.6 |
MUSR | 1.6 | 7.4 |
JCommonsenseQA | 87.7 | 89.1 |
JMMLU | 46.7 | 56.5 |
JNLI | 61.1 | 81.3 |
JSQuAD | 89.5 | 89.6 |
LLM-JP · Overall | 49.6 | 51.7 |
MMMLU · Arabic | 40.5 | 50.7 |
MMMLU · Bengali | 36.4 | 43.4 |
MMMLU · German | 53.5 | 57.1 |
MMMLU · Spanish | 55.8 | 60.2 |
MMMLU · French | 55.8 | 60.8 |
MMMLU · Hindi | 41.4 | 45.1 |
MMMLU · Indonesian | 51.0 | 54.1 |
MMMLU · Italian | 53.3 | 59.0 |
MMMLU · Japanese | 42.3 | 56.6 |
MMMLU · Korean | 46.5 | 54.0 |
MMMLU · Portuguese | 55.5 | 60.1 |
MMMLU · Swahili | 37.5 | 34.3 |
MMMLU · Yoruba | 31.0 | 30.2 |
MMMLU · Chinese | 51.4 | 61.8 |
Pricing · per 1M tokens · projected $/mo at 10M tokens
| Model | Input | Output | Context | Projected $/mo |
|---|---|---|---|---|
| Meta Llama 3 8B Instruct | — | — | — | — |
| Qwen2 7B Instruct | — | — | — | — |