Study Finds AI Coding Agents Fail to Boost Software Output
A major new study tracking engineering metrics across hundreds of firms reveals that AI coding tools significantly increase raw code generation, but fail to increase overall software output due to a human review bottleneck.
Radwan AbdulhadiEditor and technology writer
The Rise of AI Code Generation and Its Hidden Costs
Anyone who has even tangentially associated with computer programming knows that modern AI coding assistants and agents can be incredibly efficient at generating huge amounts of functional code . However, developers also know better than to trust the accuracy of that code , meaning that substantial effort must be dedicated to reviewing any AI-generated output.
A recent study examining actual coding practices across hundreds of firms discovered that human code review creates a substantial bottleneck for the overall efficiency of AI coding tools. The research found little evidence that firms increase software output or reduce employment by implementing these technologies, as efficiency gains during the coding phase are quickly absorbed by downstream production constraints.
Analyzing Millions of Work Events Across Software Firms
To reach these conclusions, Harvard University researchers Fiona Chen and James Stratton utilized aggregated analytics data from Jellyfish , a platform that tracks the granular output of engineering teams. Their dataset encompassed 300 million individual work events, including commits, pull requests, and issue management software data spanning more than 700,000 employees across over 700 software development firms from 2021 through March 2026.
The researchers employed a combination of directly measured AI usage metrics and GitHub activity analysis to determine when specific companies integrated AI coding assistants—which help autocomplete human-authored code—or AI coding agents, which autonomously write and submit code based on prompts. They then applied a difference-of-differences regression model to key variables both before and after the introduction of these tools.
More Lines of Code, Same Software Output
The raw code metrics show stark and clear increases. According to the study, the introduction of AI coding agents at a firm results in an average 30 percent increase in total lines of code generated, a 20 percent rise in total commits, and a 23 percent increase in pull requests.
Despite this massive surge in raw code, the extra volume does not translate directly into enhanced software output at the firm level. Tracking data from issue management tools like Jira showed that the resolution rate for Issues and Epics—representing wholesale software features—did not experience any statistically significant changes after AI tools were adopted.
The Code Review Bottleneck
The core reason behind this discrepancy lies directly within the code review process. The average review time between submitting a pull request and merging it into the codebase balloons by 49 percent on average after AI agents are introduced into a company's workflow.
Granular data further illustrates this friction. The share of pull requests requiring changes nearly doubled, and the number of reviewer comments per pull request increased by 35 percent following the shift toward AI agents. In response, firms saw a 14 percent increase in the share of workers performing code reviews, though researchers noted they could not attribute significant employment changes directly to AI.
Looking Ahead at AI Coding Workflows
While AI technologies could theoretically assist with the review process itself, the study found that this impact has remained marginal so far. By March 2026, although 80 percent of measured firms utilized some form of AI code review, AI agents were responsible for only 23.3 percent of review comments and 10.8 percent of pull requests, demonstrating that humans still handle the vast majority of verification work.
Because AI coding agents remain relatively new, software engineering teams are still navigating the learning curve regarding how best to deploy them. Over time, as teams gain deeper experience with these trade-offs, coding-versus-review dynamics may improve, but for now, AI-assisted coding functions as a double-edged sword where speed gains are neutralized by review overhead.
Sources
- Ars TechnicaAI coding agents generate more code, but not more software
Continue chronologically
Related entity coverage
Related stories
GitHub Stacked Pull Requests Reach General Availability
GitHub stacked pull requests are now generally available, introducing improvements for code reviews, automated merges, and branch management.
GitHub Actions API and UI Updates Query Count Results
GitHub has introduced updates to query results in the GitHub Actions API and UI to improve performance and provide more accurate record counts.
GitHub Intelligence Empowers AI Agents with Team Context
GitHub introduces GitHub intelligence, using code, context, and development history to help AI agents adapt to team workflows and conventions.
AI Agents Leak Over 13,000 Internal Screenshots
A new endpoint security report reveals that AI development agents have inadvertently uploaded over 13,000 internal screenshots to public repositories.
