Lines of Code Got a Better Publicist
This article skewers the prevalent 'lines of code' metric as a measure of AI's impact on developer productivity, arguing it's a vanity metric that has merely found a new publicist. It meticulously dissects conflicting research on AI's actual productivity gains, contrasting vendor hype with real-world outcomes and the subsequent, often dubious, rationale for layoffs. Hacker News found this a spicy take on AI adoption, resonating with developers skeptical of AI's proclaimed efficiency and the potential for mismeasurement to drive corporate decisions.
The Lowdown
The author critically examines the contemporary obsession with 'lines of code' as a measure of AI's contribution to developer productivity, likening it to a vanity metric with a rebranded public image. He argues that while the industry previously learned to dismiss raw line counts as ineffective for evaluating individual developers, AI vendors are now pushing similar volume-based metrics, such as 'percentage of code AI-generated,' to promote adoption.
- Historically, metrics like lines of code were discredited for assessing developer impact, with focus shifting to what actually shipped and its business value.
- Today, major AI companies (Google, Anthropic, OpenAI) heavily tout high percentages of AI-generated code, which the author argues are merely volume claims disguised as progress.
- This contrasts with earlier, more outcome-focused claims from companies like GitHub, which measured task completion speed.
- The author notes that while some studies show productivity gains for junior developers with AI, others indicate rising code churn, collapsing refactoring, or even slower performance for experienced developers.
- He highlights that Anthropic's internal research showed AI-assisted developers had lower code comprehension, despite the company's marketing of increased code output.
- These vanity metrics are not benign; they influence budgets, performance expectations, and headcount decisions, as seen in recent layoffs at Block and Atlassian, where AI was explicitly cited as a factor despite strong business performance.
- The author suggests that if AI genuinely delivers productivity gains, companies should use this capacity to deliver more value and accelerate roadmaps, rather than justify layoffs driven by other factors like over-hiring or investor pressure.
In conclusion, the author urges a pragmatic approach: embrace AI tools and stay current, but insist on measuring true engineering delivery through battle-tested outcome metrics like DORA, reliability, and customer value, rather than misleading volume-based 'AI vanity scores.'
The Gossip
Metrics Mismanagement & Goodhart's Law
Commenters widely agreed that measuring Lines of Code (LoC) is a poor metric, often citing Goodhart's Law ('When a measure becomes a target, it ceases to be a good measure'). They pointed out how this vanity metric is now being applied to AI, with examples like OpenAI's marketing of large LoC counts without describing value. There was also discussion about past '1M LoC' claims from Microsoft, with some defending the original intent as automated refactoring, while others saw it as further evidence of metric misuse by executives greedy for 'efficiencies that don't exist'.
Bottlenecks Beyond Bytes
A strong consensus emerged that 'writing code' has rarely, if ever, been the primary bottleneck in software development, especially for experienced engineers. Instead, the true hurdles lie in areas like defining requirements, design, code review, testing, and overall project management. Many argued that current organizational structures are not adapted to leverage faster code generation, thus limiting real productivity gains from AI and shifting the bottleneck to other parts of the software development lifecycle.
Nuanced Narratives on AI Necessity
While largely agreeing with the article's critique of AI metrics, some commenters expressed skepticism about the author's concluding advice that 'every engineer should be using AI daily.' Some felt it was an unsupported endorsement or even an 'ad' for AI, despite the preceding criticism. Others defended the author's position, highlighting the point about adapting to the rapid pace of change in the industry and encouraging curiosity and proficiency with new tools, akin to adopting higher-level languages or cloud computing.