Code Was Never the Bottleneck: What 400 Orgs Learned From a Year of AI Coding Tools

Most engineering organizations now have a budget line for AI tooling. Few can say what it bought.

DX has published data from more than 400 organizations on what AI coding tools did to software delivery in 2026. The median productivity gain was 7.7%. Deployment frequency rose steadily, but perceived delivery speed, meaning how fast developers feel they ship, moved only 4.5% across a full year of heavy investment.

So AI does help. The trouble is where it helps: code generation speeds up one stage of the pipeline, and most of the time goes somewhere else.

What the data shows

Justin Reock of DX presented the research at AI Engineer on September 30. Five findings stand out.

• Quality is spreading out. Change failure rates have become more volatile. Some organizations now ship 50% more defects than their own baseline, against an industry-wide rise of 4%. AI helps some teams and hurts others, and the gap between them keeps growing.
• Pull requests are getting bigger. Average PR size rose 64%, from 44 lines to 72. Developers rate the code as slightly more maintainable (+4%) yet feel less confident shipping it (−6%). Generated code reads well line by line, but reviewers lose track of how it fits the wider system, and larger PRs take longer to review.
• Code generation covers 14–16% of the value stream. Meetings, context switching, waiting for reviews and working around undocumented systems make up the rest. A faster code writer changes none of that.
• Junior engineers use AI most and gain the least. They consume more tokens than senior engineers for roughly the same time saved. Knowing when to trust, override or redirect a model takes judgment that comes with experience.
• Small companies save more time than enterprises. Their codebases tend to be more modular, their documentation more current and their tests more reliable. The state of the platform limits the gains more than the choice of model does.

Where the time actually goes

If writing code had been the main constraint, years of better IDEs, snippets and boilerplate generators would already have fixed delivery speed. They didn't, because the slow parts were elsewhere: unclear requirements, meeting load, flaky tests, undocumented dependencies and slow reviews. Those are organizational problems.

Faster code generation can make them worse. More PRs queue for the same reviewers. More generated code runs against the same brittle test suite. Every review asks the reviewer to hold more context, because every PR is larger.

That is why "give everyone Copilot" plateaus. It adds capacity to a stage that was never short of it and leaves the real constraint where it was.

Spread AI skills across teams

Rania Khalaf, Chief AI Officer at WSO2, spoke at the same conference about her first two years in the role. She describes it as three modes whose balance shifts as a company matures: Scientist (experimentation), Architect (strategy and reshaping the product) and Coach (internal evangelism and customer enablement).

Her most useful point is about measurement. She judges AI fluency by whether knowledge sits in a central team or is spread across product teams, not by how many tokens the company consumes.

Token counts and usage dashboards show that adoption happened. They don't show whether the organization ships better software. Khalaf recommends an AI lead inside every product team rather than one central group that holds all the expertise. That matches DX's findings: the brittle tests and undocumented services live inside product teams, so that is where the skills need to be.

What to do next quarter

If you are deciding where the next AI tooling budget goes, the data points to a different checklist from "which model is smartest":

1. Compare cohorts before scaling. Measure AI users against non-users on PR cycle time, change failure rate and review push-back, not on usage. If heavy users aren't ahead on those numbers, more adoption won't help.
2. Check platform readiness before rolling out agents widely. Modular code, current documentation and reliable tests decide how efficiently agents use tokens and how far people trust their output. Much of the gap between enterprises and small companies in DX's data comes from this.
3. Put structural limits on PR size. Enforce them with automated checks and add architectural review gates. Asking reviewers to try harder doesn't scale.
4. Give AI ownership to product teams, following Khalaf's model, so a central AI team doesn't become both the bottleneck and the only place the knowledge lives.
5. Record where automation fails. Tag each failure by cause, such as missing context, poor steering or hallucination, and map the causes to use cases. Over time this shows which kinds of work suit current agents and which don't.

How we build for this

This is also how we design Software Factories. Every change an agent makes arrives as a small pull request with tests, goes through the same review as human code, and is measured after it ships. The model matters less than the pipeline around it.

A 7.7% gain is what you get when a strong tool is aimed at a stage that wasn't the constraint. Before adding more AI, find out what is actually slow in your delivery process, and whether AI reaches it at all.

Sources

1. Justin Reock, DX: "The State of AI in Software Development: Data from 400+ Orgs"↗, AI Engineer, 30 September 2026
2. Rania Khalaf, WSO2: "The Chief AI Officer: Scientist, Architect, Coach"↗, AI Engineer, 30 September 2026