Do Autonomous Coding Agents Actually Finish Real Work? Completion Is the Claim, Speed Is the Distraction

TL;DR: Sometimes, on bounded work, with a human still carrying the definition of done. The public evidence is much stronger for speed of activity than for completion of real work: agents start tasks, generate code, and open pull requests readily, while the harder questions, was the work merged, was it right, did it survive, are measured far less often. The contrasting evidence class matters here: alongside vendor claims sits a randomised trial by METR that found experienced developers slower with AI tools on real tasks. When the best controlled evidence and the marketing point in opposite directions, the honest verdict is unproven for autonomous completion at general scale.

Key Takeaways

• Finishing means merged, correct, and durable, not attempted, drafted, or demoed. Most agent reporting stops 2 or 3 steps before that line.
• Speed evidence and completion evidence are different bodies of proof. A tool can be fast at producing code and still poor at closing tasks.
• Autonomous is doing quiet work in the pitch. Many reported successes include human scoping, human review, and human rescue that the headline leaves out.
• The METR randomised trial is the contrasting evidence class named in this post's research queue: controlled measurement on real tasks that found experienced developers slower with AI tools, against the grain of vendor claims.
• The practical test is local: track an agent's tasks from assignment to merged and surviving, on your own repository, before scaling its autonomy.

What counts as finishing real work?

A finished task in a real codebase has a life cycle: the work was scoped correctly, implemented completely, passed the project's own checks, survived review, merged, and did not quietly cause a follow up fix, a revert, or a workaround that erased its value. Finishing means reaching the end of that cycle with the original intent intact. Anything earlier, a plausible diff, an opened pull request, a demo that worked once, is progress toward finishing, and progress is not nothing. It is just not the claim being sold when a product is described as autonomous.

Real work also has properties demos avoid. Requirements are incomplete and sometimes wrong. The codebase contains conventions nobody wrote down. The correct change touches a file the task description never mentioned, and the incorrect change passes the obvious tests. This is the territory explored in what actually breaks in coding agents: the failure modes are not mainly syntax or fluency, they are judgement calls about scope, context, and when to stop. An agent that finishes demos and stalls on this territory has demonstrated generation, not completion.

Why is the evidence so heavily weighted toward speed?

Because speed is easy to count and flattering to report. Tasks attempted, tokens generated, pull requests opened, and self reported time saved can all be measured inside the product, by the vendor, on the vendor's chosen tasks. Completion requires following work into repositories, review queues, and production, over time, often on customer code the vendor cannot publish about. The evidence base is therefore lopsided by construction, not necessarily by bad faith: the measurable thing gets measured, and the marketing inherits the bias.

Self perception widens the gap. Developers using AI tools commonly feel faster, and that feeling is real data about experience, but it is not the same as measured throughput. This site's earlier measurement work, we measured AI coding productivity, sits in the assistant era of this question, where a human drives and the tool suggests. Autonomous agents raise the stakes on the same measurement problem: when the human stops driving, who notices that the work drifted, stalled, or solved the wrong problem confidently. A feeling of speed around an agent dashboard is even easier to manufacture than around an editor, because activity is constant and visibly busy.

What does the contrasting controlled evidence say?

The research queue for this post names the contrast explicitly: vendor claims on one side, the METR randomised trial on the other. METR, an independent evaluator, ran a randomised controlled trial in which experienced developers worked on real tasks in real codebases, with and without AI tools, and the trial found the developers were slower with the tools than without them, even though the developers themselves expected and perceived the opposite. This post states that finding only in those general terms and claims no precise figure from it, because the direction and the method are what matter for the argument here.

Sit with what that design buys. Randomisation removes the selection effect in vendor case studies, where successful users volunteer and failed pilots stay quiet. Real tasks remove the benchmark effect, where tidy problems with clean tests flatter automation. Experienced developers remove the excuse that the users were novices. A trial with those properties finding a slowdown does not prove agents never finish work. It proves that the burden of proof sits with the completion claim, and that perception is an unreliable instrument for this exact question, since the trial's participants misjudged their own speed in the flattering direction.

Vendor benchmarks are not worthless, but they answer a narrower question: performance on the vendor's task set, under the vendor's conditions, usually with the vendor's scaffolding tuned for the attempt. Generalising from that to autonomous completion on an arbitrary production codebase is the leap the evidence does not currently support.

Where do agents plausibly finish work today?

The credible zone is bounded work with machine checkable done states: small, well specified changes, test covered fixes, mechanical migrations, and tasks where the project can verify the result automatically and a human can reject failures cheaply. In that zone, completion is observable, the cost of an agent wandering is contained, and autonomy can be widened as the local record earns it. The pattern matches how this site treats agent claims generally: believe the denominator backed, locally verified version, and discount the universal version.

Outside that zone, on ambiguous features, cross cutting refactors, and work whose correctness only a domain expert can judge, the evidence for unsupervised finishing is thin, and the failure cost is asymmetric. A human reviewer still has to reconstruct the agent's intent to check its work, which can consume the time the agent was meant to save, the same arithmetic that haunts the productivity question generally. None of this says do not use agents. It says grade their autonomy by measured completion on your tasks, not by fluency on anyone's demo.

FAQ

Do agents finish any real work at all?

Yes, in the bounded, checkable zone described above, teams do get agent work merged and kept, and dismissing that would be as sloppy as overclaiming it. The dispute is about scope and reliability: how wide the zone is, how much human scoping and review the successes quietly include, and whether results transfer to messier work. Those are empirical questions with local answers.

Why not trust benchmark completion rates?

Because benchmark tasks are selected, specified, and scored in ways production tasks are not, and because a benchmark completion still skips review, merge politics, and durability over time. A benchmark can show a capability exists. It cannot show that the capability survives your codebase, your conventions, and your definition of done, which is the claim a buyer actually needs.

How should a team test an agent's finishing ability?

Assign real, bounded tasks with explicit done criteria, then track the full cycle: merged or not, review burden, follow up fixes, and reverts, across enough tasks that a few wins cannot carry the verdict. Compare against the same work done the previous way. Expand autonomy only where the completion record, not the activity record, justifies it.

Bottom Line

Autonomous coding agents finish some real work, in bounded areas with checkable outcomes and humans still defining done. The general claim, that they finish real work autonomously at scale, remains unproven: the published evidence skews toward speed and activity, and the strongest controlled evidence named here, the METR randomised trial, found experienced developers slower with AI tools on real tasks despite feeling faster. Treat completion as a property to measure on your own repository, task by task, and let agents earn wider autonomy from that record. Fluency is not finishing, and a busy dashboard is not a merged result.