SWE-Bench Verified (Bash Only)
SWE-Bench Verified (Bash Only) · a curated subset of SWE-bench where models fix real Python repository bugs using only bash commands, no agent frameworks.
The Frontier
Best score over time · one chart, every benchmark
Full rankings
19 models tested · sorted by score
| # | Model | Score |
|---|---|---|
| 1 | 74.4 | |
| 2 | 71.8 | |
| 3 | 70.6 | |
| 4 | 67.6 | |
| 5 | 66.0 | |
| 6 | 65.0 | |
| 7 | 64.9 | |
| 8 | 63.4 | |
| 9 | 59.8 | |
| 10 | 58.4 | |
| 11 | 52.8 | |
| 12 | 45.0 | |
| 13 | 39.6 | |
| 14 | 34.8 | |
| 15 | 26.0 | |
| 16 | 23.9 | |
| 17 | 21.6 | |
| 18 | 21.0 | |
| 19 | 9.1 |
Score distribution
Where models cluster
Correlated benchmarks
Pearson r · original research
Benchmarks that track with SWE-Bench Verified (Bash Only)
Pearson correlation across models scored on both benchmarks. Closer to 1 = strongly predictive.
Frequently asked
About SWE-Bench Verified (Bash Only)
What does SWE-Bench Verified (Bash Only) measure?
SWE-Bench Verified (Bash Only) · a curated subset of SWE-bench where models fix real Python repository bugs using only bash commands, no agent frameworks. 19 AI models have been tested on it. Scores range from 9.1 to 74.4 out of 100.
Which model leads on SWE-Bench Verified (Bash Only)?
Claude Opus 4.5 from Anthropic leads SWE-Bench Verified (Bash Only) with a score of 74.4. The median score across 19 tested models is 58.4.
Is SWE-Bench Verified (Bash Only) saturated?
No · the top score is 74.4 out of 100 (74%). There is still meaningful room for improvement on SWE-Bench Verified (Bash Only).
Does SWE-Bench Verified (Bash Only) predict performance on other benchmarks?
Yes · SWE-Bench Verified (Bash Only) scores correlate 0.99 with SWE-Bench verified across 11 shared models. Models that do well on SWE-Bench Verified (Bash Only) tend to do well on SWE-Bench verified.
How often is SWE-Bench Verified (Bash Only) data refreshed?
BenchGecko pulls updates daily. New model scores on SWE-Bench Verified (Bash Only) appear as soon as they are published by Epoch AI or the model provider.
- Category
- Code
- Max score
- 100
- Models
- 19
- Updated
- 2025-12-10
Top on SWE-Bench Verified (Bash Only)
Claude Opus 4.5 · 74.4GPT-5.2 · 71.8Claude Sonnet 4.5 · 70.6Claude Opus 4 · 67.6GPT-5.1 · 66.0More code benchmarks
Same category · related evaluations