Thinking Through AI Code Review's Quality Risk with the Swiss Cheese Model

Introduction

AI native development has become the norm, and handing much of PR review over to AI is now taken for granted. Flagging style violations, catching known vulnerability patterns, checking consistency with existing code — these are no longer tasks humans do by hand.

This is undeniably a productivity gain. But at the same time, a basic understanding of what review was actually for seems to be quietly eroding in practice. Only the parts AI can handle become visible, while the outline of everything else grows blurrier. This post untangles that structure and asks how AI and humans should divide the work of review.

Review Was Never Just One Function

The single word "review" has always bundled together several functions with very different natures, mostly without anyone noticing.

• Correctness verification: catching bugs, style violations, and departures from design principles
• Knowledge propagation: the process by which someone other than the author comes to understand the logic
• Ownership transfer: from an individual's output to the team's — from "I wrote this" to "we approved this"
• Fixing accountability: the human record of who looked at the change and signed off, in case something goes wrong later
• Decision history: leaving a comment trail explaining why a given implementation was chosen

Until now, all of these dissolved into a single activity called "review." As long as review wasn't allowed to become a rubber stamp, it worked without anyone having to think about which function was in play.

AI Has Only Replaced a Small Slice of It

What AI is good at, and what it's actually replacing at scale, is almost entirely correctness verification. Syntax errors, known anti-patterns, style violations, consistency with existing code — these are questions with reasonably well-defined criteria, which is exactly where AI performs well.

The other four functions, though, sit on a different axis entirely — they aren't questions of "right or wrong." The validity of a design trade-off in particular — why this implementation over the alternatives — is a context-dependent judgment among options that could all have been reasonable, with no single correct answer. AI can read the code, but it has no access to the business requirements, organizational constraints, or history of past decisions behind it. That context is out of reach in principle.

In other words, what AI review mechanically covers is only a thin slice of what review used to do. Yet when the only flow in place is "respond to AI's comments and merge," the sharing of why this design quietly falls through the cracks. Structurally, that leaves the process half-finished.

When the Gaps in Coverage Line Up

What makes this worse is a blind spot on the engineering side. Unless people are explicit about which range AI is reviewing and which range humans are actually responsible for, the review ends up full of holes.

Safety engineering has a concept for this: the Swiss cheese model.

Multiple layers of defense work because each layer's holes sit in different places — stacked together, they block what any single layer would let through.

What seems to be happening now is the opposite. The assumption "AI is probably catching this" and the assumption "the human reviewer is probably looking at the big picture" end up pointing at the exact same gap. The holes don't cancel out — they line up, and the risk passes straight through.

What's particularly dangerous is that an AI's approval-shaped output delivers a kind of psychological relief. The fact that "AI approved it" can end up functioning as license to skip the verification that was always the human's job — asking why this implementation. A tool meant to raise quality ends up creating a quality risk of its own — that twist is exactly what this post's title refers to.

What Review Needs to Become

Now that AI has taken over correctness verification, the center of gravity for review should shift — from "double-checking quality" to "a human understanding what AI generated and flagged, taking responsibility for it, and turning it into the team's agreed decision." How humans judge AI's review comments is itself becoming the substance of review.

In practice, a useful first step is to stop leaving the division of labor between AI and humans implicit, and spell it out instead.

• What AI review covers: syntax checks, known vulnerability patterns, style compliance, consistency with existing code
• What requires a human: the validity of design decisions, weighing alternatives, alignment with business requirements, and owning the accountability that comes with "approved"

A few concrete ways to build this in:

1. Add a required field to the PR template asking "why this implementation" — one only a human can fill in
2. Visually separate AI comments from human comments in the review UI, so it's always clear which is which
3. Write down, as a team, an explicit agreement on what AI covers and what humans cover — and revisit it regularly

Closing

AI code review has meaningfully advanced both development speed and baseline quality. But that doesn't mean review itself has become unnecessary. If anything, the parts of review AI cannot replace — validating design decisions and owning the accountability behind them — now stand out more sharply than they used to, no longer hidden inside a function we used to lump together as one thing.

Recognizing the actual coverage each side provides, and running that division deliberately, is what turns AI review into a real quality advantage. Left ambiguous, the assumption that "it's fine, AI is reviewing it" may be the biggest quality risk of all.

Questions or feedback on this post are always welcome.

Contact: contact@katatan.com