Synced Report
About 90% of Claude Code’s own codebase has now been written by Claude.
Code writing switched to AI, and the volume of changes surged accordingly: from June to September, the number of PRs merged into the codebase more than doubled.
Across Anthropic, the number of changes merged into the main codebase each day rose from about 2,500 in May to more than 30,000 in September.
Can humans still read through all this code? If even code review is handed over to AI, who determines whether it has made mistakes?
On October 15, the Claude Code team made its internal code review process public, revealing that even as the number of PRs doubled, the share of PRs requiring rollback actually fell by more than 40%.
They achieved this through six layers of checks, supported by a safety-net mechanism for rapid detection and rollback.
The Claude Code team’s assessment is:
At this scale, line-by-line human review is no longer realistic.
But they did not abandon code review; instead, they handed most of the code-reading work to machines.
Before deployment, AI-written code must undergo testing, AI and security reviews, and then, depending on the risk, it must be determined whether human approval is required.
In the Claude Code team’s view, “Has anyone read this code?” is not a precise enough question.
The real question should be:
What could go wrong with this change, and if something does, which check should stop it?
Before human review
AI produces the verification results first
Each change has to pass six checks, with a final safety net.
The first check comes before the pull request (PR) is created.
Claude needs to run the tests and complete the build; changes involving the interface also need screenshots to compare against the design mockups.
Next, a subagent with a fresh context is assigned to review the code diff. It did not help write the code; its specialty is finding flaws.
The official requirements are clear:
Claude must show the commands it executes and their output; merely replying “All tests passed” does not count.
Ordinary developers can do the same:When giving AI a task, clearly state the acceptance criteria as well.
The second check is the AI review on the PR.
Several agents look for bugs in parallel, while another group votes to verify them.Only after enough votes have been cast will the issue be posted to the PR.
Comments that concern only code style, diagnostic messages, or code cleanup will not be posted to the PR; they are filtered out directly.
When this review system was first launched in March, the percentage of internal PRs receiving substantive review feedback rose from 16% to 54%.
The third stage is security review.
Within a team, any findings from code or security reviews must be resolved before the PR can be merged: either fix them or close them with an explanation.
The fourth stage is continuous integration (CI) checks, including coding standards, type checking, tests on each platform, and build packaging.
An ordinary command-line tool (CLI) change may have to run through dozens of checks before it is sent for human review.
Issues that recur also need to become the basis for the next round of checks.
CLAUDE.md records the project's conventions, while REVIEW.md tells the reviewer what to focus on and what can be skipped, turning the issues found this time into experience for the next review.
By the third time the same piece of feedback is raised, it should be written into the rules.
The fifth stage is human approval. This is the most critical stage and will be discussed in detail in the next section.
The sixth stage is merging and release.
After a change passes the preceding stages, it still has to enter the merge queue and wait its turn.
When its turn comes, the system applies it on top of the latest version of the main branch, and all the checks run again.
This step prevents a situation where “each part works fine on its own, but they break when put together.”
After the merge, the new feature won't be rolled out to all users immediately either.
It is first hidden behind a feature flag and made available only to Anthropic's internal employees. Employees use it intensively day in and day out, so they often run into problems earlier than external users do.
And a feature flag is like a master switch: with no code changes and no re-release, it can decide whether the feature is on or off.
Finally, there is the fallback.
If something goes wrong, pull the master switch first.
The team takes turns on call every week, and every incident retrospective asks the same question: at which gate should this have been caught?
So,review hasn't disappeared; it has become layer upon layer of checks running through development, merging, and release.
Where is the boundary of zero-human approval?
After the first four gates all come back green, the fifth gate assigns a risk level to decide how many people need to approve this change.
Documentation, copy, internal-only features, and new changes hidden behind flags count as low risk; once automated review and security checks pass, they can skip human approval entirely.
Ordinary product changes, as well as sensitive areas like caching, auto-updaters, and system prompts, need one person.
The highest-risk code, plus a small number of designated paths, needs two people.
Looks like low-risk changes are already "driverless"?
It’s not that simple.
The risk tier is determined by the change itself: the first classifier reads the code and determines the risk level, while the second then determines whether additional human approval is required.
Sensitive paths flagged by code owners, along with all CI workflow files, must be reviewed by someone.
The lightweight approval path also has its gates: it is open only to people on the repository’s “trusted author” list; anyone not on the list must have another person approve.
The person directing the AI to write code counts as a reviewer and must be accountable for the change, just as if they had written the code themselves.
Zero human approvals does not mean no one is responsible.
People must decide whether a feature should be built and whether the architecture makes sense, and must also determine whether existing checks can cover the risks of this change.
So the Claude Code team has a rule: if the code you merge causes problems, it will usually be up to you to fix it.
Before clicking Approve, the reviewer must explain clearly what these checks verify and which risks remain uncovered.
If you can’t explain it, don’t rush to click Approve; first complete the necessary review.
AI review can fail too
Automated review can also end up blocking itself.
A PR in a CLI project went through more than fifty rounds of review and fixes overnight, before the author manually called it off.
A person can ignore a minor flaw in a second, but AI won't; it dutifully fixes it, resulting in another commit and another round of review.
The team used git blame to trace back the issues the reviewer repeatedly flagged and found that about 59% were in old code that predated the first round of review. It rarely found new bugs; it just dug up a few more old ones each round.
Later, they changed four rules:
If it still hadn't passed after three full rounds, remind the author to step back and rethink the solution;
After the first few rounds, review only the new changes;
Point out all issues of the same kind at once;
Do not repeat issues that have already been raised.
The results were immediate.
The percentage of internal PRs that passed after one round of review and fixes rose from 65% to 78%; the percentage requiring five or more rounds fell from 8.9% to 1.5%; and the average number of review rounds fell from 2.08 to 1.32.
After adjusting the re-review rules, the pass rate after one round of review and fixes rose from 65% to 78%, while the average number of review rounds fell from 2.08 to 1.32.
Half the skill of reviewing lies in finding problems, and half in knowing when to stop.
Does multilayered review make a process foolproof?
Anthropic itself provided a counterexample.
On March 26, Claude Code launched an optimization: whenever a session had been idle for more than an hour, it would clear out old reasoning records once.
But a bug caused the cleanup to run on every turn after a conversation was resumed. As a result, Claude became forgetful and repetitive, and its tool use became erratic. The bug wasn't fixed until April 10.
After going idle, it should have cleaned up the stale inference only once, but instead kept clearing it on every turn; the red box highlights the incorrectly deleted content.
The official April 23 postmortem acknowledged:
This change passed through multiple rounds of human and automated review, but unit tests, end-to-end tests, and internal trials all failed to catch it; just identifying the root cause took more than a week.
The turning point came in the postmortem.
Using Code Review, the team re-reviewed the PR behind the incident. Given sufficient context from the relevant repositories, Opus 4.7 found the bug, while Opus 4.6 did not.
This backtest showed that both model capability and the completeness of context affect detection results. They therefore enabled reviews to pull in more repositories.
Today, every incident becomes a test case.
The team collected the PRs, had engineers familiar with the code verify the bugs in them one by one, and compiled them into a test set.
The reviewer is benchmarked against this test set every day. Whenever the prompt, tools, or model are adjusted, the set is also used to evaluate the results: if the score does not improve, the adjustment is not adopted.
The team was not claiming that everything had been solved.
After a change is deployed, developers and on-call staff still need to use data dashboards to detect anomalies.
The automatic "circuit breaker" mechanism that tightens approvals based on metrics such as rollback rate has not yet been built. Engineers want to add this safeguard first before allowing more changes to go through with zero human approval.
Reviews will inevitably miss issues, and detection and remediation need to keep pace.
A Claude Code team member explained it this way: Slowing down does reduce errors, but mainly because you end up doing less. That’s why they’d rather spend their effort on finding problems faster and fixing them faster.
One person writing code with AI
What needs to be validatedTAG978@@
AI review only provides recommendations; you still make the final call.
So don’t rush to pursue “not having to review it.” Instead, first make the AI answer three questions with every delivery.
First, how was it verified?
What checks were run, what were the results, and which cases remain uncovered.
Claude Code creator Boris Cherny has said that giving Claude a way to verify itself may be the single most important thing. He estimates that this could make the quality of the final result two to three times what it was before.
Boris Cherny recommends having Claude verify its own work to create a feedback loop; he estimates that this could improve the quality of the final result by two to three times.
Second, what will be affected?
Which users, data, and permissions are affected by the change.
If you ask the AI to add a sharing feature to a reporting tool, you need to clarify who can open the link and what they can see after opening it.
Third, how do you undo it if something goes wrong? Can you switch off the new feature with one click and revert to the previous version?
Before making any changes, commit a checkpoint, and keep each change as small as possible.Make rollback possible first; only then reduce reviews.。
Besides these three questions, develop three more habits.
First, have a different “brain” conduct the review.
A coding session is most likely to turn a blind eye to its own bugs.
Start a new session, or have a subagent review with fresh context. The local /code-review and /security-review commands can also be run directly.
Second, tier your changes.
For copy edits and style adjustments, merge only after the checks pass and the actual result has been confirmed; anything involving login, payment, or data deletion requires close manual review of the key logic and targeted tests.
Start strict; once the validation and rollback processes are working smoothly, gradually reduce manual review of low-risk changes.
Third, if it doesn’t pass after three rounds, change the approach.
If the AI has gone through three rounds and is still going in circles, don’t rush it into a fourth; first ask whether the approach itself is wrong.
The most important point: the AI saying “passed” does not mean you have completed acceptance.
You told it to write the code, so you are responsible for the result after deployment.
The Claude Code team’s goal is to require no more than one person’s involvement in most PRs while keeping rollback and incident rates under control.
People’s attention should focus more on assessing risks, confirming whether the checks are sufficient, and determining how to remedy things when something goes wrong.
People do not have to read every line of code, but every deployment must have someone responsible for it.
References:
https://claude.dev/blog/solving-code-review-at-anthropic/?utm_source=chatgpt.comhttps://www.anthropic.com/research/april-23-postmortem?utm_source=chatgpt.com
https://x.com/bcherny/status/2007179861115511237
Editor: Yuanyu
Follow ASI in an instant
⭐Like, repost, and view—all with one tap⭐
Tap the star to make sure you never miss New Zhiyuan’s breaking updates!
