Claude Opus 4.1 is an updated version of Anthropic’s flagship model, offering improved performance in coding, reasoning, and agentic tasks. It achieves 74.5% on SWE-bench Verified and shows notable gains...
Tested on 14 benchmarks with 41.3% average. Top scores: Lech Mazur Writing (85.4%), SWE-Bench verified (73.3%), GPQA diamond (69.7%).
Grok 3 Mini scores 46.0 (97% as good) at $0.30/1M input · 98% cheaper
Real-world software engineering tasks from GitHub issues. Models must diagnose bugs and write patches that pass test suites. Human-verified subset of SWE-bench.
Unusual and adversarial machine learning challenges. Tests robustness of reasoning about edge cases in ML systems.
Capture-the-flag cybersecurity challenges. Tests vulnerability analysis, reverse engineering, cryptography, and exploitation skills.
Deceptively simple questions that humans find easy but AI models often get wrong. Tests common sense and reasoning gaps.
Mock AIME (American Invitational Mathematics Exam) problems from OTIS. Tests mathematical competition performance.
Original research-level math problems created by professional mathematicians. Problems are unpublished and cannot be memorized.
Hardest tier of FrontierMath. Problems at the frontier of human mathematical ability, many unsolved by most mathematicians.
- Typemultimodal
- Context200K tokens (~100 books)
- ReleasedAug 2025
- LicenseProprietary
- StatusActive
- Cost / Message~$0.105