Search "how much faster are AI-assisted developers" and you'll land on two credible, well-designed studies that contradict each other. Microsoft, GitHub, and MIT researchers measured a 55.8% speedup on a controlled coding task. METR measured developers running 19% slower on real work they already knew well. Neither result is wrong. They're measuring different tasks, different developers, and different definitions of "faster."
That distinction matters more than either headline number, because it's the only thing that tells you whether either number applies to your team.
Why the headline numbers contradict each other
Most coverage of this topic picks a side. One camp cites Microsoft's 55.8% and calls the case closed. Another cites METR's 19% and calls AI coding assistants overhyped. Both are cherry-picking from a body of research that, read together, tells a consistent story: AI assistance produces large, repeatable speedups on bounded, well-specified tasks, and it produces smaller gains, no gains, or losses on complex work inside codebases a developer already knows cold.
The studies also differ in who they measured. Microsoft's experiment and the enterprise RCT below used working professionals on a single assigned task under lab-like conditions. METR used experienced open source maintainers doing their normal jobs on codebases they'd worked in for years. DORA surveys entire organizations about delivery outcomes, not individual coding speed. None of these studies is measuring "AI-assisted developers" as one undifferentiated group, and no single number can describe all of them at once.
The studies, compared
| Study | Sample | Task type | Result |
|---|---|---|---|
| Microsoft/GitHub/MIT (2023) | Professional developers, randomized to Copilot or no Copilot | Single greenfield task: build an HTTP server in JavaScript, as fast as possible | 55.8% faster with Copilot |
| Google enterprise RCT (2024) | 96 full-time Google engineers | Maintenance task: add logging to an existing 474-line, 10-file internal repo | raw completion time 96 vs 114 min (p=.038); covariate-adjusted estimate ~21% but p=.086, wide confidence interval |
| METR (2025) | 16 experienced open source maintainers, 246 tasks | Real maintenance work on large, familiar codebases | 19% slower with AI enabled |
| DORA 2024 | Annual cross-industry survey | Organization-level delivery outcomes, self-reported | Individual productivity, flow, and satisfaction up; delivery stability and throughput down |
| DORA 2025 | Annual cross-industry survey, updated sample | Organization-level delivery outcomes, self-reported | Delivery throughput now positively correlated with AI adoption |
| NAV IT (2025) | Longitudinal case study, one organization | Commit-based activity before and after Copilot rollout | No statistically significant change in measured activity; developers felt more productive |
Where the speedup is real
Microsoft's 55.8%, on a controlled, from-scratch task
The Microsoft Research study ran a randomized controlled trial: developers were assigned to build an HTTP server in JavaScript as quickly as possible, half with GitHub Copilot enabled and half without. The Copilot group finished 55.8% faster. This is the number everyone quotes, and it's accurate for exactly what it measured: a single, self-contained, greenfield task with no existing codebase to understand, no legacy conventions to follow, and no teammates to coordinate with. It's the best case for AI-assisted coding, not the average case.
The enterprise RCT's ~21%, and why it isn't the same kind of task
The most commonly misdescribed study in this space is Google's own internal randomized controlled trial, published as a 2024 preprint. It's frequently summarized as a greenfield productivity study. It isn't. Ninety-six full-time Google engineers were assigned to patch an existing internal repository, roughly 474 lines across 10 files, by adding logging functionality. That's maintenance and enhancement work inside code someone else wrote, not a blank-slate build. Call it bounded and well-specified; don't call it greenfield.
The raw completion-time difference was significant: 96 minutes with AI against 114 without, at p=.038. The paper's headline figure, about 21%, is the estimate after controlling for factors known to influence time on task, and that one sits at p=.086, short of the conventional threshold, with a wide confidence interval. The direction of the effect held; the certainty around its size didn't. One more detail worth a sentence: the study tested whether senior engineers benefited more than juniors did, the finding that would cut against the common assumption that AI tools mostly help less experienced developers catch up, but the effect wasn't statistically significant (p=.706), and the paper rejected that hypothesis outright.
Where the speedup disappears or reverses
METR's 19% slower, and the perception gap that makes it worse
METR's July 2025 study is the one every pro-AI argument has to answer, so here it is plainly: 16 experienced open source developers completed 246 real tasks in codebases they knew well, and with AI assistance enabled, they were on average 19% slower than without it. Before the study, these developers forecast a 24% speedup from using AI. Afterward, despite measurably running slower, they believed they'd been about 20% faster. That gap between felt speed and measured speed is the study's most uncomfortable finding, and it's the reason METR's result is still being argued over on Hacker News months after publication.
The scope matters as much as the number. These weren't developers encountering unfamiliar code; they were maintainers working in repositories they already understood at a level no AI tool can match in a single session. That's a materially different situation from Microsoft's from-scratch HTTP server or Google's bounded logging patch, and it's exactly the kind of work where a maintainer's existing mental model competes with, rather than complements, an AI suggestion.
NAV IT's null result
A 2025 longitudinal case study tracked commit-based activity at one organization before and after GitHub Copilot adoption and found no statistically significant change in measured output. Developers in the same study reported feeling more productive. Put next to METR, this is the second data point showing that developer self-report and measured output can diverge, in either direction, and that neither one alone tells you what's actually happening to delivery.
What's happening at the team and organization level
DORA 2024: individual gains, organizational costs
The 2024 DORA report found that AI adoption raised individual developer productivity, flow, and job satisfaction, while lowering software delivery stability and throughput at the team level. That's a genuinely odd combination: developers felt and were more effective individually, while the systems they shipped through got less stable and slower to move code through. The likely mechanism, though DORA doesn't reduce it to one cause, is that faster individual output without matching investment in review capacity, testing, and deployment pipelines just moves the bottleneck downstream.
DORA 2025: the reversal
The 2025 DORA report found delivery throughput now positively correlated with AI adoption, a reversal of the 2024 finding. This is the strongest piece of macro-level evidence in the pro-speed case, because it's not a single controlled task; it's a cross-industry survey showing the organizational cost from 2024 has started to close. DORA's own framing is worth repeating honestly: AI amplifies existing team strengths and weaknesses rather than fixing them outright, and instability concerns from the prior year haven't fully resolved. Teams with strong existing engineering practices got more out of AI adoption in 2025; teams without them didn't automatically get better by adding a coding assistant.
So how much faster, honestly
Put the six studies side by side and a pattern holds up better than any single number: large, repeatable speedups show up on bounded, well-scoped, self-contained work, whether that's Microsoft's greenfield HTTP server or the enterprise RCT's bounded maintenance patch. The trend at the organization level is now favorable, per DORA 2025, after a rockier 2024. And the gains shrink, disappear, or reverse on complex work inside large, mature codebases handled by developers who already know that code better than any assistant can learn it in a session, which is exactly what METR and NAV IT both found.
The practical question for an engineering leader isn't whether to adopt AI tooling. It's whether you can tell, task by task, which bucket a given piece of work falls into, and whether you're measuring outcomes or just asking people how fast they felt. METR's forecasting gap and NAV IT's self-report mismatch both point the same direction: felt speed is not a reliable stand-in for measured speed, and any team serious about the question needs to look at cycle time and delivered output, not just developer sentiment.
This is also why the senior engineer's job has shifted from writing code to reviewing and architecting it rather than typing it faster. If AI assistance mostly compresses the bounded, well-specified parts of the job, the parts that don't compress, judgment calls inside unfamiliar systems, architectural tradeoffs, reviewing someone else's (or an AI's) output for correctness, become the part that actually differentiates a senior hire. It's why we screen specifically for the ability to review and improve AI-generated code rather than testing raw typing speed, and it factors directly into how AI tool use gets evaluated in a senior React hire. None of that is a claim about how fast any particular engineer works with AI tools; it's a claim about what to test for once the easy part of the job gets faster for everyone.
It's also worth separating this from the productivity problems that predate any AI tool. Context switching, unclear priorities, and interruption-heavy workdays were destroying engineering throughput long before Copilot existed, and no coding assistant fixes a team that's constantly pulled in six directions. AI tooling can compress the time a focused developer spends on a bounded task; it does nothing for a developer who never gets an uninterrupted hour to use it in. If you want to see what engineer-led vetting looks like in practice, that's the other half of this: hiring engineers who can tell the difference between the work AI speeds up and the work it doesn't, and who don't need a study to know which is which.
Frequently asked questions
Do AI coding assistants actually make developers faster?
On bounded, well-specified tasks, yes, and the effect is large: Microsoft's controlled study found a 55.8% speedup on a from-scratch coding task, and Google's internal RCT put the covariate-adjusted effect at roughly 21% on a bounded maintenance patch, though that estimate didn't clear conventional significance. On complex work inside large, familiar codebases, the effect shrinks or reverses, per METR and NAV IT. There's no single "AI-assisted developers are X% faster" answer that covers both situations honestly.
Why did the METR study find developers were slower with AI?
METR studied 16 experienced open source maintainers working in codebases they already knew well, not developers encountering unfamiliar code. With AI assistance enabled, they completed 246 real tasks 19% slower on average than without it, despite forecasting a 24% speedup beforehand and believing afterward that they'd been about 20% faster. The gap suggests that on deeply familiar, complex work, AI suggestions can compete with a maintainer's existing mental model rather than speed it up.
Has DORA changed its position on AI and software delivery?
Yes. The 2024 DORA report found AI adoption raised individual productivity and job satisfaction while lowering delivery stability and throughput at the organizational level. The 2025 report found delivery throughput now positively correlates with AI adoption, a reversal of the prior year. DORA frames the shift as AI amplifying whatever strengths or weaknesses a team already had, not as a uniform improvement, and notes that instability concerns from 2024 haven't fully resolved.
Does AI speed up coding but not overall delivery?
That was the pattern in 2024: individual coding speed and satisfaction went up while team-level delivery throughput and stability went down, most likely because faster individual output without matching investment in review and deployment capacity just shifts the bottleneck further downstream. The 2025 DORA data shows that gap closing, with throughput now moving in the same direction as individual productivity, though the report is clear that this depends on the strength of a team's existing engineering practices going in.