2 weeks ago, my most experienced engineer wrote this message in our team’s channel
(published with his permission):
I know we're all very much enjoying the fact that we dont have to wait for code reviews, but I wonder if we should still consider doing them anyway. This is not only because I enjoy reviewing and dont get the chance to do it much, but also because I feel like it's one of the best ways we have of sharing our work with others, and it's also a great place to get discussions going on solutions.
I've recently done some cleanup (because it's the sort of thing I can do when my brain is half-fried), and I noticed a consistent pattern where one agent did a passable one-off job at a place where we did not have good pre-existing examples, and then multiple agents used their implementation as golden standard.
I feel like it's very important to remind ourselves that the fact that all the work is done by agents doesnt mean that we need to put less attention on quality. Those "passable" one-offs replicate themselves very fast, and they end up causing all sort of bugs (be it compounding perf hits, race conditions, or confused agents). Every line that goes into our codebase really does, literally, need to pass the standard of "what if 100 more people will do this exact same thing everywhere".
So how did we get here:
The 5-stage code review collapse
Up until five months ago, every PR in our team was reviewed by at least two people - the engineer who wrote the code, and one or more reviewers. Exactly how 99% of software engineering teams have worked since the 1970s.
Then in April, we moved to work on a brand new product, without any production to worry about. So we decided to make code reviews optional to help us move faster.
At first, most PRs still got reviewed. We all had this healthy fear of releasing something to production. But getting a PR to production in minutes instead of waiting for hours is addicting, so over a couple of months the number of code reviews dropped to almost zero.
Still, I expected everyone to fully read and understand the code their agents produced, and didn’t accept the ‘I don’t know, Claude wrote this’ excuse. We also added an agent review step in our coding workflow (a skill that spawned 3 separate agents with different instructions to review the code, and then reconciled their findings). So instead of 2 humans reviewing the code, we had one human (the author) and multiple agents.
But that too became a bottleneck. We are in a competitive market, and we need to move very fast. If I generate UI code that will be thrown away, why does it matter if I fully understand it?
So we split our tasks into 2 types:
Validation - the disposable parts that we will throw away in case we pivot. Things like onboarding flows, most UI, and features specific to the current product.
Infra - the capabilities and parts we would keep even if we pivoted.
We carefully reviewed the infra tasks, and only quickly glanced at validation ones (which were mostly frontend code).
That didn’t last long either.
The code review stage was what forced people to read the code carefully and REALLY understand what the agents wrote. You couldn’t skip it, as you wouldn’t be able to answer questions and comments on the PR (and sending your coding agent to answer them was frowned upon).
When we removed that forcing function, we all started to cut corners. We still looked at the code, but the level of our understanding started to diminish.
That’s how in 5 months we went from 100% of the code being reviewed by 2 people, to most of it being reviewed by 0:
It’s happening in every modern org
For organizations that use coding agents, the number of PRs created rose 3X(!) in the past 2 years. And that’s not just in greenfield projects, but across already established companies:
As I shared above, our case was even more drastic, due to the move to the greenfield project.
Here are our PRs in numbers. We jumped from 4 PRs/day to 28 PRs/day in August. You can also clearly see the drop in code reviews, from 100% to just 2% this month:
And it’s not just trivial PRs, the output itself increased by ~8X over that time. You can see we were static from January to April, and since then we have continued to accelerate:
When you have so much code generated, the old processes just break.
Is it really a problem?
Look at the software you use, written before coding agents existed.
Most of it sucks and is full of bugs. 90% of the codebases I’ve worked on were messy, and felt like an urgent refactor was needed (see ‘How refactoring almost ruined my marriage’). Ask an engineer what they think about the code they wrote 3 years ago, and you’ll probably hear it’s terrible.
So if you write a feature and it works well, should you still care about code quality?
I’m not asking this rhetorically - I don’t accept any process or past decision as a given. I never cared about beautiful code just for the sake of it, the outcomes were what mattered to me.
Even if the feature will have some small bugs, and the code will have duplications - it’s not that different from how it was before, and you at least produced it tens of times faster.
So it’s not like software engineers did a perfect job before LLMs came. Our baseline is mediocre at best.
A critical benefit of code reviews was the ability to align on how we work, which helped us write new code much faster. But when agents write all of the code, who cares if they need to work a bit harder?
Here’s why I believe you should:
The accelerated broken windows theory
You’ve probably heard about the broken windows theory, which comes from criminology. A broken window in the neighborhood that nobody replaces signals that no one cares, which invites more vandalism, and eventually worse crime.
Usually, that degradation of the neighborhood takes time. One window, then two, then some graffiti, then stealing cars, and so on. Same with codebases - they don’t start as shitty, it happens over years and years of patches upon patches.
With agents, that cycle comes down to months, or even days.
I’ve shared in “Explaining to business people why building software is still hard” about my experience in a hackathon, working with Lovable:
It’s 1 pm on the final day of the Hackathon, 5 hours left. For the last 2 hours, we made ZERO progress. I’m cursing the day Lovable was born. As soon as I fix one problem, another one appears, and our app is barely usable.
The start was very promising. I joined 2 engineering friends and 3 recruiters to build ‘HoneyCrew’, a smart referral system. We decided to go with Lovable so the recruiting team can maintain it themselves later.
On the first day, we just FLEW, completing 90% of the project. Scraping, scoring, Slack integration, admin - things looked almost ready.
On the 2nd (and final) day, we worked on some minor improvements, and things just… Completely broke. Endless bugs everywhere, slowness, everyone super stressed.
Things just deteriorated VERY fast, and we felt like we were playing an ‘agent whack-a-mole’, killing the same bugs over and over again:
Going back to what my engineer wrote the team in Slack:
One agent did a passable one-off job, and then multiple agents used their implementation as the golden standard. Those "passable" one-offs replicate themselves very fast, and they end up causing all sorts of bugs.
Let’s say you have a perfect codebase. You read every line, and made sure there are great .md files with rules, and everything is built well.
Then you need a quick fix for a bug. The agent does a workaround, and you don’t properly review it. It breaks one window. You blink, and the next thing you know, all the windows are broken. It just followed the existing pattern:
How the best teams are solving it
Ok, so it’s very clear that it’s impossible to continue reviewing code manually. But, unfortunately, NOT reviewing anything is also quite a mess.
Boris Cherny got asked this question. Here’s his answer:
Of course, a very biased answer.
Anthropic is the poster child of moving fast, but we can clearly see the effect of AI generating almost 100% of their own code. I have seriously considered switching to Codex, and I know many engineers who did. They provided a report back in April on the quality issues in Claude, but I feel they are playing the same whack-a-mole.
See the discussions in this “bug” for example, which is still open (and annoys the hell out of me):
My point is, no one has found a great way to solve it yet, even Anthropic.
The best advice I’ve heard was when I interviewed Andrew for his guest post:
Optional code reviews (but still do them on ~5% of PRs)
Do LOTS of plan reviews - another engineer going over the detailed plan the agent produced
I haven’t experimented with a good process for plan reviews yet, but planning to. I’d love to hear from anyone with interesting solutions for this challenge!
What I enjoyed reading this week:
Does Code Quality Still Matter in the Age of AI-Assisted Coding? Mark Heath answered this question beautifully (a 3-minute read).
What happens to engineers when AI writes all the code? Loved that one, I also believe engineers are not going anywhere.
Build the Busy Western Town. Netflix’s founder on tech debt.
