Better than 26% of all models
Context
N/A
Input $/1M
TBD
Output $/1M
TBD
Type
text-generation
License
Open Source
Benchmarks
25 tested
Data updated today
About
Meta-llama text generation model. 1407K downloads on HuggingFace.
Tested on 25 benchmarks with 45.2% average. Top scores: JSQuAD (89.5%), JCommonsenseQA (87.7%), IFEval (74.1%).
Capabilities
reasoning
1.6
#178 globally
math
8.7
#189 globally
knowledge
15.4
#201 globally
language
53.0
#102 globally
general
28.2
#32 globally
Benchmark Scores
Compare AllTested on 25 benchmarks · Ranked across 5 categories
Score Distribution (all 231 models)
0255075100
▲ You are here
reasoningCompare reasoning →
MUSR
1.6—HuggingFace MuSR (Multi-Step Reasoning). Tests multi-hop reasoning requiring chaining multiple facts together.
mathCompare math →
MATH Level 5
8.7—HuggingFace evaluation of MATH Level 5 problems. Competition math requiring advanced reasoning and proof construction.
knowledgeCompare knowledge →
MMLU-PRO
29.6—HuggingFace MMLU-Pro. Harder version of MMLU with 10 answer choices instead of 4 and more challenging questions.
GPQA
1.2—HuggingFace evaluation of GPQA (Graduate-Level Google-Proof Q&A). PhD-level science questions that cannot be easily searched.
vs Devstral 2 2512
vs Claude 3 Sonnet
vs Phi 2 Excellent (85+) Good (70-85) Average (50-70) Below (<50)
Links
Info
Research
Documentation
Community
Source Code
BenchGecko API
meta-llama-meta-llama-3-8b-instruct
Specifications
- Typetext-generation
- ContextN/A
- ReleasedApr 2024
- LicenseOpen Source
- StatusActive
Available On
MetaTBDLearn More
Share & Export
Frequently Asked Questions
Meta Llama 3 8B Instruct is an open-source text-generation AI model by Meta, released in April 2024. It has an average benchmark score of 31.7.