Reasoning Model

Reasoning models like o1, o3, Claude Extended Thinking, and DeepSeek R1 produce a private chain of thought before their final answer. This costs more tokens and latency but sharply improves scores on benchmarks like GPQA Diamond, ARC-AGI, and MATH. You pay ~10× more per answer but get dramatically better correctness on hard problems.

Reasoning models are trained with reinforcement learning on chain-of-thought traces that maximize a correctness reward. At inference time they allocate a variable "thinking budget" of tokens that are generated but not shown to the user. OpenAI o1 spends between 1K and 100K reasoning tokens per problem depending on difficulty. Claude Extended Thinking exposes the reasoning transparently and charges for it. DeepSeek R1 open-sourced both the weights and the RL recipe, making reasoning models accessible to anyone. The trade-off is direct: more thinking tokens equals higher cost and latency but better accuracy on benchmarks that require multi-step inference. For pure recall or chat, reasoning models are overkill.

A reasoning model is fine-tuned with RL against a reward that measures final-answer correctness on problems with verifiable answers (MATH, code, etc.). The policy learns to emit long intermediate traces that decompose the problem, self-verify partial results, and backtrack from dead ends. Test-time compute becomes a new scaling axis: OpenAI reported that o1 quality scales log-linearly with thinking tokens up to ~10K tokens per problem. Serving costs mirror this: o1-preview charges $15/M input + $60/M output, with hidden reasoning tokens billed as output. DeepSeek R1 (671B MoE reasoning model) demonstrated that the recipe generalizes: pure outcome-reward RL on a strong base model yields emergent reasoning behavior without any process-level supervision. Open questions: how far does test-time compute scale before diminishing returns, and whether reasoning-style RL harms general capabilities.