74 tasks.
A broader test of terminal work.
The July 2026 release brought a new task set covering software, science, machine learning and other domains, with reference runs and adversarial review.
Read the 3.0 releaseBenchmark watch / Public research
What the models get done.
Resolution rate is only part of the picture. Look at the agent, effort setting, token use and cost alongside the result.
74 tasks.
66 tasks.
Less noise from the environment.
Released in August 2026, 4.0 removes eight tasks, fixes task issues and recalibrates compute and time limits. The official release notes set the agent timeout to eight hours.
Read the 4.0 releaseSnapshot · 23 September 2026
Performance & resources
Resolution rate (%) · higher is better
Published configurations
| Model | Agent / effort | Resolution rate | Run cost | Tokens |
|---|---|---|---|---|
| Codexmax | 58.2%±2.8 pp | $3,300 | 1.5B | |
| Claude Codemax | 57.9%±3.8 pp | $6,200 | 2.7B | |
| Claude Codexhigh | 53.9%±3.2 pp | $6,100 | 6.9B | |
| Claude Codemax | 44.5%±3.8 pp | $7,300 | 3.8B | |
| Claude Codemax | 41.8%±3.2 pp | $2,700 | 8.7B | |
| Codexmax | 37.3%±3.8 pp | $2,500 | 4.4B | |
| Claude Codemax | 23.6%±3.6 pp | $6,500 | 6.4B | |
| Codexmax | 21.5%±3.3 pp | $1,700 | 5.7B | |
| Grok Buildhigh | 20.3%±3.1 pp | $3,600 | 4B | |
| mini-SWE-agenthigh | 19.1%±3.4 pp | $1,800 | 17.2B | |
| Codexmax | 17.3%±2.8 pp | $300 | 11.6B | |
| Grok Buildhigh | 12.4%±2.6 pp | $2,100 | 3.4B | |
| Claude Codemax | 12.4%±3.1 pp | $9,600 | 21.6B | |
| mini-SWE-agenthigh | 11.2%±2.4 pp | $1,300 | 11.1B |
Source: Snorkel AI’s Terminal-Bench 4.0 leaderboard. Figures are a dated snapshot, not live data or Clorvin test results. Published cost figures are rounded.
Read efficiency in context.
Cost and tokens are reported run totals, not prices per task. Agent harnesses and effort settings differ. Version 4.0 also changes tasks and resource limits, so a score change from 3.0 is not a like-for-like measure of model improvement. Use the source run details before making a purchasing or model-selection decision.
Want to inspect the task side?
Task Library contains original Clorvin tasks, with new briefs, downloadable packages and locally recorded reference checks.
Explore Task Library →Work with Clorvin
Tell us what needs doing.
Share the scope, the expected volume and your timeline.
We’ll work out where our team can help.