LiveBench

LiveBench was created in 2024 to combat benchmark contamination. Frontier models increasingly see benchmark problems during training, which inflates scores. LiveBench releases new tasks every month across six categories: reasoning, coding, math, data analysis, instruction following, and language. Only recent task releases count toward scores, ensuring models can't have seen them.

LiveBench ships monthly new-task releases. The leaderboard shows both "recent" scores (last 3 months) and "all-time" scores. Categories weight equally: reasoning (logic puzzles, planning), coding (novel problems beyond HumanEval), mathematics (AMC-style), data analysis (table-based questions), instruction following (strict format compliance), and language (complex writing). Frontier models in 2026: Claude Mythos Preview leads "recent" at ~73%, GPT-5 at ~71%, DeepSeek V3.2 at ~65%. The "all-time" leaderboard ages out contaminated scores as tasks are deprecated.

LiveBench's contamination-resistance works through strict release cadence and objective grading. Each monthly drop contains 30-60 new tasks per category, hand-crafted by the LiveBench team and held out for ~2 months before public evaluation. Grading is deterministic · no LLM-as-judge, only verifiable answers. The "recent" score averages the last 90 days of tasks. This directly addresses Schaeffer et al.'s critique that static benchmarks lose signal once they're in pretraining data.