How Faster Coding Shifts the Bottleneck to Debugging
This article was originally published on Sept. 17 and has been updated with new information.
For all the hype about developer productivity and AI coding tools, recent research has thrown a reality check on expectations. In July, METR reported that “when developers use AI tools, they take 19% longer than without” — even though they think it’s making them more productive.
Recently, Will Wilson, co-founder and CEO, and Akshay Shah, field CTO at Antithesis, joined TNS to unpack the disconnect between real and perceived productivity gains, including why the results from using large language models (LLMs) in software engineering are so mixed.
Here’s a hint: Generating code isn’t the biggest barrier to developer productivity, and evaluating the quality of code that AI generates is a bigger factor than most expect.
Problems With AI-Generated Code: Work Slop
Today’s AI tools are not yet as smart as humans, but they can still handle basic coding tasks, explained Wilson.
If you’re trying to do a common task with a lot of information available, “LLMs can really do something good for you right away,” Wilson said.
But “if your problem is something new that there aren’t a ton of examples of on the internet, if you’re trying to do something actually new, or if you’re just using a less commonly used technology, suddenly performance drops considerably.”
He offered an example: “If you turn [AI] loose on a million-line repository with a whole bunch of existing code and a whole bunch of existing stuff, and you’re like, ‘Hey, make this change, add this feature or whatever,’ they tend to have a lot more trouble because they’re not reasoning like human beings. They’re sort of looking at all the code and like, trying to grok it in some super-intuitive way and then spit out the change they think you want. But it’s not how a human being thinks.”
“We can no longer treat a piece of work as proof that you’ve thought about the work.”
—Akshay Shah, Antithesis
There’s a second-order effect to this, Shah said: work slop.
“Historically, we used the fact that you have written something down, or you have produced some code, as a kind of proof of work. Like, the act of producing the code is a testament to the thought and care that you’ve put into what you’re proposing.” But LLMs eliminate that.
“We can no longer treat a piece of work as proof that you’ve thought about the work. This really allows people to run amok and impose a lot of costs on their co-workers,” Shah explained.
One of those costs is having to fix the AI’s errors, Wilson said. “What you’re asking me to do is clean up after your AI’s mistakes. If you’re not willing to take the time to look at what it did, like, why am I taking my time to do that?”
Benefits of AI: Prototyping and Communication
Not that long ago, prototyping a new product was complicated and slow: A business user figured out what they needed, created a product brief to explain it, and then a designer turned that into a mock. Then the engineers had to translate that prototype into code that is “bulletproof, battle tested, scalable, not full of bugs” and works across dates, localities, languages and other use cases, Wilson said.
This took time, while the C-suite asked why the product wasn’t in production yet. Now, however, “non-programmers can whip up a prototype and say to somebody: ‘Look, this is what I mean. This is what I want.’”
“I think it’s going to help to smooth communication between technical people and, like, non-technical business users at a whole ton of companies, big time,” Wilson said.
How Faster Coding Shifts the Bottleneck to Debugging
If AI code generation isn’t delivering the efficiency and output AI has promised, watch How To Stop AI From Slowing Developers Down, now available on demand.
During this free webinar, Wilson and Shah discussed why AI isn’t boosting engineering productivity, the bottlenecks engineering organizations face and how to make the gains you’re looking for.
What You’ll Learn
By watching, you’ll leave with best practices, real-world examples and actionable tips including:
- Why AI code generation delivers only partial wins today, with real productivity still constrained by quality and context gaps.
- How speeding up coding can shift the bottleneck to debugging, testing and long-term maintainability.
- How to rethink workflows and tooling to integrate AI generation seamlessly into engineering practices.
Watch this free webinar today!
