OpenAI o4-mini is a compact reasoning model in the o-series, optimized for fast, cost-efficient performance while retaining strong multimodal and agentic capabilities. It supports tool use and demonstrates competitive reasoning...
Tested on 26 benchmarks with 53.2% average. Top scores: MATH level 5 (97.8%), HELM — IFEval (92.9%), HELM — WildBench (85.4%).
Qwen2.5 7B Instruct scores 57.4 (101% as good) at $0.04/1M input · 96% cheaper
Multi-language code editing from Aider. Tests editing ability across Python, JavaScript, TypeScript, Java, C++, Go, Rust, and more.
Computer-aided design evaluation. Tests understanding of CAD concepts, 3D modeling, and engineering design principles.
Unusual and adversarial machine learning challenges. Tests robustness of reasoning about edge cases in ML systems.
Stanford HELM WildBench evaluation. Tests reasoning on challenging real-world tasks.
Abstraction and Reasoning Corpus. Tests fluid intelligence through novel visual pattern recognition puzzles. Core measure of general intelligence.
Deceptively simple questions that humans find easy but AI models often get wrong. Tests common sense and reasoning gaps.
Competition-level math from AMC, AIME, and olympiad problems. Level 5 is the hardest tier, requiring creative problem-solving.
Mock AIME (American Invitational Mathematics Exam) problems from OTIS. Tests mathematical competition performance.
Stanford HELM evaluation of mathematical reasoning across diverse problem types.
- Typemultimodal
- Context200K tokens (~100 books)
- ReleasedApr 2025
- LicenseProprietary
- StatusActive
- Cost / Message~$0.007