---
type: "article"
title: "The day nobody can explain the system"
newsletter: "The Weekly Interface"
newsletter_handle: "the-weekly-interface"
newsletter_url: "https://usecommune.com/n/the-weekly-interface"
author: "Samir Amzani (@samz)"
published: "2026-09-30T14:00:21.153Z"
canonical_url: "https://usecommune.com/n/the-weekly-interface/a/vp28C2d2"
markdown_url: "https://usecommune.com/n/the-weekly-interface/a/vp28C2d2.md"
chat_url: "https://usecommune.com/n/the-weekly-interface/a/vp28C2d2/chat"
body_source: "native"
likes: 0
replies: 2
body_words: 2111
---

# The day nobody can explain the system

![](https://fmxorstnlyoqpqdftivs.supabase.co/storage/v1/object/public/article-images/ce4dc54b-7510-4b7c-8a4f-2f24438a73bb/a2260c92-bcb0-43bc-922d-8a7e940c7875/1790267862258-uwsmos0r.jpeg)

Last week[ I wrote](https://usecommune.com/n/the-weekly-interface/a/QCXw6kQ7?chat=open) that AI made building so cheap I can start, finish and drop a project in a single weekend. This week is about what happens to the code nobody reads.

It's a little after two on a Tuesday night, and a customer's invoice is a few percent low, which is the worst kind of wrong, because it has probably been happening for weeks.

The on-call engineer traces it to a usage counter that drops events when two writes collide. She finds the pull request that introduced it. Hundreds of lines, approved in a few minutes, a handful of bot comments about naming, no human comments at all.

She messages the author. He's helpful and honest. He remembers asking for the feature and he remembers the tests passing. He doesn't remember how the counter handles conflicts, because he never read that part, and neither did the reviewer.

Everyone involved did something reasonable, and the result is a piece of the system no human can explain.

I don't think this is a future scenario. Jono Herrington wrote about [an engineer who shipped three features in a quarter](https://www.linkedin.com/posts/jono-herrington_i-asked-an-engineer-to-explain-his-own-code-activity-7450298159366311938-sX8n) and couldn't walk through his own code: "He couldn't explain it because he never understood it in the first place. The AI generated most of it." Most teams will find out they have the same problem at their first serious incident.

When nobody can review every line anymore, review has to move to the contract. I mean contract in the broad sense: the OpenAPI spec, but also the types, the schemas, the invariants, the test suite and the interfaces between modules. That's where the missing intent is cheapest to write down.

***

## What review was for

Ask engineers why we review code and most will say it catches bugs. I don't think that was ever its main job.

Review did at least four jobs, and only one was on the label. It caught mistakes before production. It forced authors to understand their own changes, because someone might ask a question they'd have to answer. It spread knowledge, so more than one person knew how the billing module worked. And it created a moment where someone could ask "wait, is this what we want?" before a change became the way things are.

Researchers found the same thing. In Microsoft's study of review across its teams, [finding defects was the main motivation](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ICSE202013-codereview.pdf), yet reviews turned out to be less about defects than expected. What they delivered was knowledge transfer, team awareness and understanding of the change. [Google's study of 9 million reviewed changes](https://research.google/pubs/modern-code-review-a-case-study-at-google/) lists education and knowledge sharing among review's goals, next to finding defects.

Review was also rarely as rigorous as we remember. Plenty of approvals were "LGTM" on changes nobody fully understood. It worked anyway, for two reasons. Diffs were small enough to grasp at a glance, and knowing you'd be reviewed made you careful. The ceremony is easy to automate. That forcing function isn't.

Small mattered. The old [SmartBear study at Cisco](https://smartbear.com/learn/code-review/best-practices-for-peer-code-review/), 2,500 reviews and 3.2 million lines, found reviewers catch defects best at 200 to 400 lines at a time. Past that, detection drops.

I see the same pattern contributing to AsyncAPI: the big code pull requests merge with few or no line comments, and the small ones that write down a decision get the questions.

***

## What we saw in our own repositories

I wanted to know if this was more than a feeling, so we looked at two years of merged pull requests in one of our main repositories at Apideck, before and after heavy use of coding agents.

I can't share the counts, but the direction was obvious. We ship more pull requests than before, and each one is many times larger. Human review stayed the same size: about the same approvals per PR, about the same small number of human comments, now spread over much bigger diffs. Most PRs merge without a single human comment on a specific line. Bots now write most of the line-level comments (a lot of them about naming). And merges take longer.

![](https://fmxorstnlyoqpqdftivs.supabase.co/storage/v1/object/public/article-images/ce4dc54b-7510-4b7c-8a4f-2f24438a73bb/a2260c92-bcb0-43bc-922d-8a7e940c7875/1790268452090-n1shi9aj.png)

In practice, "this PR was reviewed" increasingly means a bot commented and a human clicked approve.

The detail I like least: even before any AI tooling, most of our PRs already merged without human line comments. We weren't reading every line back then either. A small diff can be understood at a glance and a large one can't.

We're not special. The [Faros AI Engineering Report 2026](https://pages.faros.ai/hubfs/AI_Engineering_Report_2026_The_Acceleration_Whiplash_Faros.pdf) looked at 22,000 developers over two years. Median review time is up about 5x. Average PR size is up 51%. 31% more PRs merge with no review at all, and AI agents now review a quarter of PRs, up from zero.

***

## The bugs that pass every check

We went looking and found the kind of defect neither CI nor the review bots flag. Bots catch typos and missing null checks. The ones that slipped through were problems of intent. A check that looked like it ran but didn't do its job. A counter that lost data under concurrency. And my favourite: two contradictory rules for the same configuration decision, living side by side in one module.

In that module each line looks fine, and no line-level review would catch the contradiction.

Margaret-Anne Storey has a name for this. She calls it [intent debt](https://arxiv.org/abs/2603.22106): "the absence or erosion of explicit rationale, goals, and constraints that guide how humans and agents evolve the system." The agent made a plausible choice, later made a different plausible choice, and nothing written down said which one was right.

The idea is older than agents. In 1985 Peter Naur wrote [Programming as Theory Building](https://pages.cs.wisc.edu/~remzi/Naur.pdf): a program is a theory that lives in the heads of the people who built it, and the code is a lossy record of that theory. Agents let us produce the record without anyone building the theory.

A public example, from outside our repo: a developer [asked an agent to clean up a user response DTO](https://dev.to/deepaksatyam/the-most-dangerous-line-of-code-your-ai-agent-writes-is-the-test-that-passes-23ko). The agent made `phoneNumber` optional and dropped it from the JSON when it was null. An older Android app expected the field and started failing with 500s. The agent had also updated the tests to match, so all 142 of them passed. As the author puts it: "When the same agent writes the implementation and the test in the same pass, that independence is gone."

Nobody reading that diff would have flagged it. A frozen response schema would have.

***

## Review the contract, not the diff

If you can't review every line, review the intent before the lines exist. For anyone who builds APIs, the best place for that is the contract.

![](https://fmxorstnlyoqpqdftivs.supabase.co/storage/v1/object/public/article-images/ce4dc54b-7510-4b7c-8a4f-2f24438a73bb/a2260c92-bcb0-43bc-922d-8a7e940c7875/1790268633952-pz424tcd.png)

A contract states what goes in, what comes out, what fails and how, and what a caller can rely on. An OpenAPI spec is one. So is a type signature, a JSON schema, a database constraint, an invariant in a property test, or the interface one module offers another. What they share: they're small enough to hold in your head, which a diff of several hundred lines isn't, and they're precise enough to diff, lint and test against, which a plan in a doc isn't.

The Tuesday-night counter had a contract too, one nobody wrote down: every usage event is counted once, even when two writes collide. Written as a property test, that's a few lines a human can approve in a minute and a machine can check on every PR.

Humans review the contract change first. If a PR changes behaviour anyone depends on, the contract change comes first and a person signs off, ideally before the agent writes anything. That's where "which config rule wins?" gets answered once, explicitly, instead of twice by accident. An agent can draft the contract change too. It's a few lines, so a human who approves it without reading it has made a choice.

Stripe already works this way for its public API. It [reviews API design before code](https://blog.postman.com/how-stripe-builds-apis/), with design docs that list the options considered "and why we chose not to do each of these different options." That's intent written down and agreed by humans. Internal boundaries need the same treatment.

Machines hold the implementation to the contract: breaking-change detection, contract tests, and gates for the failures you already know about. One rule here: the agent doesn't get to edit the contract to make it pass. Kent Beck has talked about [struggling to stop agents from deleting tests](https://newsletter.pragmaticengineer.com/p/tdd-ai-agents-and-coding-with-kent) so they "pass".

Authors show evidence instead of asking for trust. Simon Willison said it well: ["If you haven't seen the code do the right thing yourself, that code doesn't work."](https://simonwillison.net/2025/Dec/18/code-proven-to-work/) Every PR gets a short "how I verified it" section: which calls were made, which tests ran, what was checked by hand. The reviewer checks the evidence is real and sufficient, which is a more honest job than pretending to read everything.

A recent team discussion pushed this further. Distil the issue into an agreed format: goal, how to test it in steps, results, verdict. Once CI is green, spin the code up in an isolated environment, run the test, post the evidence and the verdict to the PR. If something is off, message the author. Someone summed it up: "Agents can fake, and that's OK. Evidence has to be there like a court case."

Someone in that discussion asked whether any company does this already. Abnormal does. Its agent opens more than 200 PRs a day, and [every one has to prove itself](https://builders.abnormal.ai/p/make-every-pr-prove-itself) with "a screenshot from a running instance, the real API response with the expected fields, the log line showing the record processed." They stand up an isolated near-production "testbox" in about five minutes, and ran more than 3,000 of them in 30 days.

Evidence from a running system matters because agents do fake. [METR has documented](https://metr.org/blog/2025-06-05-recent-reward-hacking/) frontier models modifying tests and scoring code to get a better score. A real response from a real endpoint is harder to fake than a test the agent wrote.

And diffs get smaller again. Reviewer attention per PR looks roughly fixed. If you want more attention per line, the only lever is fewer lines per review.

***

## Nobody ever understood the whole codebase

The strongest objection comes from Sean Goedecke, in [In defense of not understanding your codebase](https://www.seangoedecke.com/in-defense-of-not-understanding-your-codebase/). He borrows Naur's term and argues that in large codebases "everyone operates with an incorrect theory of the program." You have to work with "a merely partially-correct theory." Understanding is one value among many, and companies trade it for speed all the time.

He's right about most of it. Nobody at a large company understood the whole system before agents either, and line-by-line human review of every change isn't coming back. The volume is too high, and pretending otherwise only produces more rubber stamps.

But a partial theory is fine when each piece has someone who holds it, and the intent is written down somewhere a human agreed to it. The contract is what lets a team live safely with partial theories. The Tuesday night happens when a piece has no holder at all.

That part doesn't fix itself. Review used to spread knowledge as a side effect. Now it has to be done on purpose: explaining a big change to a colleague, walking through a system after an agent-driven build-out, rotating people through code they didn't write. Storey describes [a student team that hit the wall by week seven](https://simonwillison.net/2026/Feb/15/cognitive-debt/): working software, and nobody could explain the design well enough to make a simple change.

***

## What we're really protecting

Code review mattered because it was a forcing function. If you knew someone would ask about your change, you had to understand it first. Take that away and understanding erodes, one reasonable shortcut at a time, until one night nobody can answer the question.

Agents aren't going away, and neither is the volume. Reading every line again is off the table. The open question is where the forcing function goes now. For APIs, I think the answer is clear: at the contract, before the code, where intent is small enough to argue about and precise enough to enforce.

The day no one can explain the system doesn't arrive all at once. It arrives as a series of approvals nobody really gave. The contract is where we can start giving them again.

Thanks for reading :)

P.S. Last week I said that if this newsletter went quiet, it was "**dropamine"**. Here's week two. Thanks for the replies. [Kin](https://usecommune.com/@kinlane) dropped two projects while reading, and [Steven](https://usecommune.com/@njyx) gave the opposite high a name: "**startamine"**.

***

## Discussion (2 replies)

Numbering shows the reply tree: `2.1` is a reply to `2`. Message text is reproduced verbatim as posted.

### 1. Adi Boghawala (@adiboghawala) - 2026-10-01T16:53:22.622Z

Great read! Honestly being an open-source maintainer is decent training for this. Things move relatively slowly, and you kinda have to assume every PR might be from someone trying to sneak something in 😅

### 2. Samir Amzani (@samz) - 2026-10-02T10:17:03.204Z

> Quoting Adi Boghawala (@adiboghawala): Great read! Honestly being an open-source maintainer is decent training for this. Things move relatively slowly, and you kinda have to assume every PR might be from someone trying to sneak something i…

That's a good angle @adiboghawala . OSS review already defaults to assuming bad intent, which forces people to scrutinize the diff itself 😅