Model drift
Model drift is a change in a model's behavior while its name stays the same · caused by new weights, system prompts, safety filters or serving changes behind a stable API name.
Text reviewed October 5, 2026
Model drift is a change in a model's behavior while its name stays the same · caused by new weights, system prompts, safety filters or serving changes behind a stable API name.
Basic
Providers sometimes update a model without changing its name. Users notice when answers, refusals or formatting change. Drift can improve a model or make it worse for a specific use, which is why teams pin versions and re-test regularly.
Deep
BenchGecko's Model Drift Index sends the same fixed probes to the same models every week and compares each run with the previous one: exact-answer accuracy, refusals and how the model names its maker. Small differences are expected because even temperature 0 is not perfectly deterministic, so drift is flagged only on large moves.
Expert
Sources of drift include weight updates, quantization changes, system prompt edits, moderation layers and routing to different hardware or providers. Dated snapshots in the API name, where offered, are the main protection.
Depending on why you're here
- ·When an AI changes without telling anyone
- ·Pin dated model versions when the API offers them
- ·Re-run your evals after provider updates
- ·Watch /gecko-tests/model-drift-index
- ·Silent changes are an operational risk for AI products
- ·Measure with fixed probes over time
- ·Separate real drift from sampling noise