Inside CodeRabbit's thousands-of-pull-requests month — and why AI code review has to be

redesigned as a pipeline, not bolted on as a tool.


Harjot Gill runs CodeRabbit, a company that sells AI code review. Last month, by his account, he personally shipped more than 5,000 pull requests — most of them written by agents, a large share of them janitorial: fixing flaky tests, repairing CI, refining caching, work the agents did during his commute and overnight. His AI spend runs at roughly $250,000 a month. His conclusion, in a long conversation with Gergely Orosz of The Pragmatic Engineer, is blunt: human code review — a person reading agent-written diffs line by line — is dead.

Discount the source appropriately before using any of this. Gill sells the category he is describing, and his numbers are his own: his public posts put his monthly pull request count at roughly 3,000 in one place and 5,000 in another, and none of it is independently audited. Orosz himself flags CodeRabbit as an outlier — the AI spend he hears about at typical companies is $200 to $2,000 per developer per month, not a quarter of a million behind one founder. Treat the specifics as a vendor's claims. Treat the structure underneath them as the part worth your attention, because the structure does not depend on Gill's arithmetic: when the cost of producing code collapses, review — the one stage that still runs on senior human attention — becomes the constraint. Organisations that keep review shaped like a gate will not notice it failing. It fails quietly, as rubber-stamping.

Review redesigned: agent pre-checks and an evidence bundle replace the all-human gate; human judgment is spent on the risky slice, and the merge gate opens on evidence.

Decision 1: Review capacity does not scale — plan for the queue, not the ideal

Gill saw the early version of this years ago. At FluxNinja, his previous company, in 2021, he found code review already becoming a bottleneck — and that was when GitHub Copilot had merely sped up the writing, with autocomplete. Agents are a different order of volume. A reviewer corps sized for human-paced production does not stretch to factory-paced production, because the input scales with compute and the reviewers do not.

The failure modes are worth naming, because none of them announces itself. First, latency: the queue grows, changes wait days for a first read, and engineers route around review or batch changes into larger, riskier units to amortise the wait. Second, theatre: approval continues, reading stops. The approve button still gets clicked, the audit trail still looks clean, and the gate's output — a human actually having understood the change — quietly goes to zero. Third, attrition: your strongest engineers become full-time gatekeepers for machine output, which is neither the work they are best at nor the work they will stay for. All three surface months later, as escaped defects and incident reviews, long after the decision that caused them has been forgotten.

Decision 2: Verification moves into the pipeline — agents check, machines prove

What replaces the gate in Gill's account is a stack of self-verification — he argues agents need to be "hill climbing," with ways to verify their own work. The layers he describes: formal models — his team used TLA+ on one product and found race conditions that code review had overlooked, and he is experimenting with Lean; local deterministic simulations that fuzz different interleavings of concurrent operations; emulated versions of real APIs, so agents test against the shape of the real thing; development environments agents spin up to confirm services actually run; and in production, staged rollouts behind feature flags, with an agent observing the rollout's behaviour and widening it only when things stay stable. All of this is CodeRabbit's practice as described in the episode — claims, not findings. But every layer is checkable against your own stack, which is the point.

The principle for leaders: every check that is mechanical, repeatable, and cheap belongs in the pipeline, run before human attention is spent. AI code review tooling — Gill's own product category included — is one layer of that stack, not the stack. Buying a reviewer bot while leaving the human gate untouched does not relieve the bottleneck. It gives you two.

Decision 3: Humans judge the risky slice — and a merge requires evidence

At CodeRabbit, per the episode, engineers no longer review individual pull requests. They review releases — bundles of changes — and spend their reading time on the planning and the final asset rather than the diff. Trust is made explicit and revocable: a "review exempt list" of engineers who may push without anyone reviewing their releases; when an outage happens, the team traces the change that caused it and the engineer earns a strike; enough strikes and the exemption is gone.

Generalised, the merge standard changes from a human saw this to this change carries its evidence: tests run and passed, simulation or formal checks where the domain warrants them, a stated risk classification, a rollback path. Human line-by-line reading is reserved for the slice where judgment is actually load-bearing — the plan, the invariants, the changes whose blast radius is wide or hard to reverse. Everything else merges on evidence. And evidence has one advantage a clicked approval never had: it is still there afterwards, auditable, when the incident review asks what the gate actually knew.

What to do Monday

  1. Measure the queue. Median time a change waits for its first human review — and your approval-without-comment rate. The second number is your quiet-failure gauge.
  2. Classify your checks. List everything reviewers currently do by eye that a machine could do deterministically. Move those into the pipeline first.
  3. Define the evidence bundle. Decide what a merge must attach — test results, risk class, rollback plan — before any human is asked to look at it.
  4. Shrink the human gate deliberately. Name the change classes that still require line-by-line reading, and let the rest merge on evidence — with exemption and strikes making trust explicit and revocable.

Bottom line

Gill's software factory is an extreme, run by a man selling the machinery, and its economics will not transfer to a company with a fixed customer base — Orosz makes that point himself. But the bottleneck Gill is describing is already present in milder form anywhere agents write code faster than senior engineers can read it. The decision is not whether to adopt AI code review. It is whether review gets redesigned — pre-checks in the pipeline, evidence at the gate, human judgment spent only where it is scarce and decisive — while you still get to choose the design, or whether it collapses on its own schedule and you find out from the incident review.


Sources: The Pragmatic Engineer, "Software factories and code reviews with Harjot Gill" (Nov 18, 2026) — Gergely Orosz's published written summary of his podcast conversation with CodeRabbit founder and CEO Harjot Gill. All figures for pull request volume, AI spend, and per-pull-request cost are Gill's own, as reported there; they vary across his public posts and are treated in this brief as claims, not findings. The "hill climbing" verification layers, release-level review, and the review-exempt list are described as CodeRabbit's practice in the same account. Orosz's counterpoints — typical AI spend of $200–$2,000 per developer per month, and CodeRabbit's outlier status — are from the same piece.