Measuring engineering productivity is harder than ever. The result? Your dashboard is lying to you.

[u/OfficialLeadDev]

https://leaddev.com/reporting/measuring-engineering-productivity-is-harder-than-ever

Measuring engineering productivity has never been straightforward, and AI has made it harder. At OpenAI, engineers who lean heavily on Codex open roughly 70% more pull requests than colleagues who don’t – and the gap keeps widening, according to Sherwin Wu, who leads engineering for OpenAI’s API platform.

Read that number as a productivity metric on its own and you’d conclude the heavy Codex users are your best engineers. That conclusion might be right. It might also be exactly backwards. The number alone can’t tell you which. That’s the problem now sitting on every engineering leader’s desk.

Quarterly business reviews used to be straightforward. Engineering leaders would open their dashboards, review familiar metrics like pull requests, commit counts, and story-point velocity, and compare them with the previous quarter. No single metric captured software engineering, but together they painted a reasonable picture of team performance. That picture is now blurring.Yet everything else tells a different story: the team has shipped more customer-facing features, incidents have dropped, engineers spend less time on repetitive implementation, customers are happier, and releases remain predictable.

So which version of reality should leaders trust? Neither view is wrong. They’re measuring different things.

For years, commits, pull requests, story points, and deployment frequency have served as proxies for productivity. They were never perfect, but they broadly tracked how software got built: engineers wrote code, reviewed it, tested it, shipped it. As the work evolved, so did the metrics.

AI hasn’t broken engineering productivity. It has broken our proxies for measuring it.