scores near the frontier
at a third of the cost
78.8% mean accuracy on six core coding benchmarks, at 63% lower total cost than Claude Code with the Claude family, on the same 1,344 real tasks.
- Claude Code + ML.ai model
- Baseline, Claude family
- Claude Fable
- Claude Code + ML.ai model
- Baseline, Claude family
- Claude Fable
across eight
public benchmarks
Share of tasks ML.ai solved on each benchmark, and where it ranked among the setups that ran it. Hover a card for every score.
Every setup ran the same 1,344-task snapshot, scored by each benchmark’s own tests, one run per task. ML.ai’s mean across the six core benchmarks is 78.8%. Pro and Multimodal ran on the two ML.ai-model setups only.
matches Claude Code on accuracy
at about a third of the cost
Each setup plotted by mean accuracy across the six core benchmarks against its average cost per task. Up and to the left is better.
Same accuracy as Claude Code
for 36% of the cost per task
78.8% · $0.031 per taskSame accuracy as Claude Code for 36% of the cost per task
- 78.8%
Cost per task$0.031
- Claude Code + ML.ai model76.1%
Cost per task$0.053
- Claude Code, Claude family (baseline)78.4%
Cost per task$0.085
- Claude Code, Claude Fable82.6%
Cost per task$0.333
Mean accuracy on the six core benchmarks. Bars show cost per task.
Where the 63%
saving comes from
Start from Claude Code with the Claude family and swap one part at a time.
Select a step to trace it on the chart
- Claude Code, Claude family78.4% mean accuracy
- Claude Code + ML.ai model76.1% mean accuracy
- agent and model78.8% mean accuracy
What each task costs on
Average spend to attempt one task, pass or fail. Grey shows the same ML.ai model running inside Claude Code.
Cost per task, by benchmark
ML.ai against Claude Code running the same ML.ai model
- + Claude Code
SWE-bench Verified−41% per task
$0.048
$0.082
SWE-Lancer Diamond−41% per task
$0.024
$0.041
Terminal-Bench−41% per task
$0.047
$0.080
AgentBench-OS−46% per task
$0.007
$0.013
LiveCodeBench v6−42% per task
$0.056
$0.096
MBPP+−43% per task
$0.004
$0.007
Claude Code + ML.ai model+ Claude Code- VerifiedSWE-bench Verified−41% per task
- LancerSWE-Lancer Diamond−41% per task
- TerminalTerminal-Bench−41% per task
- AgentBenchAgentBench-OS−46% per task
- LiveCodeLiveCodeBench v6−42% per task
- MBPP+MBPP+−43% per task
FAQs
What is being compared?
Four setups ran on the same 1,344 coding tasks: ML.ai’s own agent and model, Claude Code with the ML.ai model, Claude Code with the Claude family, and Claude Fable.
Why does ML.ai cost less per task?
Two things show up in the runs. The ML.ai model charges $1.50 per million output tokens against $10 to $50 for the larger Claude models, and the ML.ai harness spends about 42% less per task than Claude Code with the same model.
Should I start with the drop-in or the full stack?
Putting the ML.ai model inside Claude Code cuts cost per task from $0.085 to $0.053, at 2.3 points lower accuracy than the baseline. The full ML.ai agent and model costs $0.031 per task and scores 0.4 points above the baseline, so the full stack gives the bigger gain.
How reliable are the smaller benchmarks?
Each task ran once, so on the smaller sets a gap of a point or two can be noise. Treat close single-benchmark results as a tie and lean on the mean across all six.
Will I get the same results on my own codebase?
Treat public benchmarks as a guide. Results depend on your languages, repo size and test coverage, so the quickest check is to run ML.ai on a few real tasks from your own backlog.
Trynow
No waitlist. Sign in and Infinite is ready to run your first task.