Are AI-assisted developers actually faster?

Where the speed numbers come from

The 55.8% figure comes from a randomized controlled trial by Microsoft Research, GitHub, and MIT. Developers used GitHub Copilot to build a self-contained HTTP server in JavaScript from scratch, a bounded, well-specified task with a clear finish line. That's the scenario where AI assistance shows its clearest gains.

A separate randomized trial inside Google found AI tooling sped up a task by roughly 21%, unadjusted. It's worth being precise about what that study actually tested: 96 Google engineers editing an existing 474-line, 10-file repository, not building something new. It was maintenance work, not greenfield development. The unadjusted result was statistically significant (p=.038), but once researchers adjusted for other factors, the effect lost significance and the confidence interval widened considerably. Treat the 21% as suggestive, not settled.

At the macro level, Google's 2025 DORA report found that AI adoption now correlates positively with team-level delivery throughput. That's a reversal of DORA's own 2024 findings, which had linked AI use to lower delivery throughput and stability even as individual developers reported feeling more productive. DORA hasn't published the 2024 percentages we'd need to compare the two years directly, so the honest comparison is directional: the correlation flipped from negative to positive.

Where the gains disappear

The clearest counterexample comes from METR, which ran a randomized trial with 16 experienced open-source maintainers working on 246 tasks in codebases they already knew well. With AI assistance, they were 19% slower, not faster. Before the study, they'd forecast a 24% speedup. Afterward, they still believed they'd gained roughly 20%. The gap between what they experienced and what they believed is the whole point: on complex work inside large, familiar codebases, AI assistance can slow experienced developers down while feeling like it's speeding them up.

A longitudinal case study of developers at NAV IT found something similar in the field: after Copilot adoption, commit-based activity showed no statistically significant change, even though developers reported feeling more productive.

What this means for planning

The pattern across these studies is consistent. Large, reliable speedups show up on well-scoped, bounded tasks, the kind with a clear spec and a defined finish line. Gains shrink or reverse on complex work inside large, mature codebases handled by developers who already have deep context on that code. If your team's day-to-day work looks more like the second case than the first, don't assume Copilot-style numbers will transfer.

That also means knowing which kind of work your team is actually doing, and hiring for the judgment to tell the difference, matters as much as the tooling. For how to evaluate that judgment in a candidate, see engineer-led vetting. It also changes what a senior engineer's job looks like day to day; see how to hire software engineers for how that role has shifted toward reviewing and architecting AI-generated output.