Anyone can claim a better coding agent. We publish what our harness adds to the same underlying model — on public benchmarks, with the method, the sample size and the date in plain sight.
Results
Every pair below is the same model on both sides. The left number is that model working bare, over raw API access. The right number is the same model inside Erevos-03. Nothing about the weights changes — only what surrounds them.
6/8→8/8
Knowledge layer
Curated, continuously updated engineering knowledge is the difference between an agent that guesses and one that fixes.
Internal Skeldyne eval, 8-task suite, identical model and prompts — the knowledge layer the only variable.
4/20→18/20
Test-driven verify loop
With no test oracle the base model lands 4 of 20; with the harness checking every edit against the project's own tests, it lands 18 — four and a half times as many.
Internal Skeldyne eval, 20-instance SWE-bench Verified slice, identical model — the verification harness the only variable. August 2026.
9/20→15/20
Terminal-Bench
Real work in a live terminal — building kernels, cracking archives, resolving broken environments. The bare model over raw API access solves 9 of 20; Erevos-03 with the full Skeldyne stack lands 15 — same model, the stack is the difference.
Official Terminal-Bench harness and grader, terminal-bench-core 0.1.1, 20-task slice, identical model, August 2026.
Why the same model solves more
A model alone gets one shot at a guess. An agent with a harness gets to look things up, check its work, and try again when the check fails. That is the entire gap above.
◈
It looks things up
A curated knowledge layer supplies verified fixes and current package behaviour at the moment they are relevant — instead of recalling how a library behaved a year ago.
✓
It checks its own work
Every edit is run against the project's own tests. A failing check is not the end of the task; it is the input to the next attempt. This single mechanism is the 4/20 → 18/20 line.
⌕
It finds the right file
Semantic code search locates code by meaning rather than by name, so an unfamiliar repository stops being a wall — which is most of what a live-terminal benchmark actually tests.
⎇
It works in parallel
Wide searches and independent subtasks go to focused workers and come back folded into one plan, so a long task does not degrade into a long conversation.
This is also why we quote our numbers against the bare model rather than against other companies' agents: the comparison we can run honestly, repeatedly and with one variable at a time is the one against ourselves.
Method
The rules we hold ourselves to on every number published here:
1
Same model both sides
The baseline and the Erevos-03 run use identical weights and identical prompts. If the model changed, the comparison would be meaningless.
2
One variable at a time
Each row isolates a single component — the knowledge layer, or the verify loop — rather than reporting the whole stack and attributing the gain wherever it flatters us.
3
Official harness where one exists
Terminal-Bench is run through the official harness and grader (terminal-bench-core 0.1.1), not a re-implementation of our own scoring.
4
Raw fractions, always
We publish 15/20, not "75% — industry leading". The denominator is the honest part, so it stays visible.
ModelIdentical across baseline and Erevos-03
Terminal-BenchOfficial harness · core 0.1.1 · 20-task slice
SWE-benchVerified · 20-instance slice
Knowledge evalInternal · 8-task suite
MeasuredAugust 2026
What we don't claim
The fastest way to lose your trust would be to overstate this page, so here is the other side of it.
·
These are slices, not full suites
20 instances of SWE-bench Verified and 20 Terminal-Bench tasks. Enough to show a real effect, not enough to claim a leaderboard position — and we do not claim one.
·
Two of the three are our own evals
The knowledge-layer and verify-loop suites are internal. We describe exactly what they contain and what was held constant, but nobody else has graded them.
·
A benchmark is not your codebase
Benchmark tasks are self-contained and well-specified. Your repository has history, conventions and a build that only works on Tuesdays. Treat these as evidence that the harness helps, not as a forecast of your week.
·
We don't rank ourselves against other agents
Running someone else's product fairly is harder than running our own, and a comparison we cannot repeat cleanly is not one worth publishing.
Try it on your own code
The only benchmark that decides anything is your own repository. Install the CLI, point it at a bug you already understand, and see whether it closes the loop.
A typical task costs a few cents, and credits never expire — see pricing.