Claude 3.5 Haiku
開発元 Anthropic · リリース日 2024-11-04
37.2
平均スコア
$0.80/1M
入力料金
$4.00/1M
出力料金
200K tokens (~100 books)
コンテキストウィンドウ
multimodal
タイプ
Tested on 17 benchmarks with 37.2% average. Top scores: HELM — IFEval (79.2%), HELM — WildBench (76.0%), Lech Mazur Writing (73.5%).
ベンチマークスコア
| ベンチマーク | カテゴリ | スコア | Bar |
|---|---|---|---|
| HELM — IFEval | language | 79.2 | |
| HELM — WildBench | reasoning | 76.0 | |
| Lech Mazur Writing | knowledge | 73.5 | |
| MMLU | knowledge | 65.7 | |
| HELM — MMLU-Pro | knowledge | 60.5 | |
| MATH level 5 | math | 46.4 | |
| HELM — GPQA | knowledge | 36.3 | |
| GeoBench | knowledge | 34.0 | |
| CadEval | coding | 32.0 | |
| WeirdML | coding | 30.7 | |
| Aider polyglot | coding | 28.0 | |
| HELM — Omni-MATH | math | 22.4 | |
| Balrog | knowledge | 19.3 | |
| GPQA diamond | knowledge | 17.5 | |
| SimpleQA Verified | knowledge | 6.7 | |
| OTIS Mock AIME 2024-2025 | math | 4.2 | |
| FrontierMath-2025-02-28-Private | math | 0.3 |