AI Code Review: Where It Helps and Where Humans Still Decide

An ai code review means a model reads a diff before a person does and flags bugs, missing tests, security smells, and style issues. It handles routine checks well and does poorly on design, product intent, and anything that can break production. Treat the model as a first reader, and leave the merge with an engineer who knows the system.

Teams add the pass because reviewers keep typing the same comments: an unhandled null, a test that never ran the new branch, a secret in a log line. A model gets through those in seconds. The reviewer still decides whether the change is the right one.

What AI code review is good at

An ai powered code review is a pattern matcher over public code and your prompt. It reads the diff, sometimes the surrounding files, and returns comments. It doesn't ship the change or own the outcome. It pays off on local checks that tired humans skim past.

Defects that show up in the diff

• Null and error paths that the happy path ignores
• Off-by-one loops, wrong units, and swapped arguments
• Copy-paste that updates one call site and misses the twin
• Tests that assert the old behavior, or never call the new code
• Resource leaks, like a file, a lock, or a transaction left open on one branch
• Log lines that print tokens, emails, or full request bodies

For each one, open the line, accept or dismiss the comment, and move on.

House style and review checklists

Style guides and "did you update the migration?" checklists barely change from week to week. A model follows a written list more reliably than a reviewer who is switching between three services. Put the test command, the naming rules, and the comments you consider out of scope in the instructions.

Security smells, not a security audit

Models spot known bad shapes: string-built SQL, eval on request data, disabled TLS checks, overly broad CORS, secrets in fixtures. That isn't a pentest. Use them to catch the obvious miss, and keep a real review for auth, payments, and anything that crosses a trust boundary. Deterministic rules belong in code scanning rather than in a free-form comment, and GitHub's documentation on code scanning covers that setup.

Where humans still decide

The expensive mistakes are rarely a missed semicolon. More often the code correctly implements the wrong behavior.

Design and blast radius

A model can tell you a function is long. It can't reliably tell you that this helper will be copied into four services, or that the new table locks the orders row during checkout. Those questions go to a person: what else calls this, and what happens if it fails halfway?

Product intent

Ticket text and code often disagree. "Users can export their data" might mean a CSV of their own rows, or a dump of the whole tenant. A diff can satisfy the words and still build the wrong product. A reviewer who has read the ticket will catch that, while a model that only sees the diff will praise a complete export.

Risk and the merge bar

Someone has to say "this can merge" and answer for the rollback. Models sound just as sure when they're wrong as when they're right, so if the bot can approve, the merge decision sits with a system that never gets paged. Keep approval with a named person. Let the bot block only on checks you would also put in CI, such as formatting, banned APIs, or missing tests for a touched module.

If comments go up and the change fail rate doesn't go down, the bot is adding noise. Track fail rate and time to restore for a month before you add more automated gates. The DORA research program defines both measures.

How to run it without lowering the bar

The bot comments, a human resolves those comments, and CI still runs the tests. A green bot is not a sign-off, and nobody should read it as one.

Write the instructions the way you brief a new teammate

Give the reviewer a bounded job:

1. Read the diff and the linked ticket.
2. Flag bugs, security smells, and missing tests.
3. Do not restyle unrelated lines.
4. Do not approve. Post questions.
5. When you are unsure, say what you would need to see.

"Be thorough" gets you essays. Specific instructions get comments a person can accept or reject in under a minute.

Separate suggestion from gate

A suggestion is a comment the author can dismiss. A gate fails the check until a rule passes. Deterministic rules go in CI (types, lint, unit tests, secret scanning), and judgment goes in comments. If the model can fail the build on a free-text opinion, authors learn to reword the diff until the comment goes away, which is not the same as fixing the issue.

Keep a human on risky paths

Require a person, not only a bot, when the change touches authentication, payments, migrations that lock large tables, exports of personal data, or a dependency you haven't pinned before. Everywhere else, one human plus the bot is enough. Putting a second human on a copy change is how queues grow.

A worked example

A six-line change adds a discount code to checkout. The author updates the price function and adds one test for a valid code.

The model points out that an empty string returns a zero price, and that the test never covers an expired code. Both notes are local and correct, so the author adds a guard and a second test.

Then a human asks something else. The ticket says codes are single-use per account, but the diff only checks that the code exists and hasn't expired. Nothing records the redemption. The model didn't flag this because the code does exactly what it says. The gap is a requirement the code never implements. The author adds a redemption row and a unique constraint.

If you read only the bot, you ship a discount that can be reused forever. If you wait only for a human, the empty-string bug can still land, because people skim small diffs. Run both.

How ai code review tools fit the workflow

Most ai code review tools live on the pull request: GitHub and GitLab bots, IDE extensions, and reviewers built into coding agents. The main difference between them is how much they can see.

A diff-only bot is fast and blind to callers. A bot with repository access can check whether a renamed function still has other references. A reviewer that can see the ticket can compare the diff to the request, which would have caught the gap in the discount example. More context means longer runs and comments on untouched code, so cap the files it may read and ban drive-by refactors.

A code review ai should answer one question. Pick the tool that fits that question:

Question	Better fit
Does this line look wrong?	Diff-only reviewer or IDE hint
Did we miss a call site?	Reviewer with repo read access
Does this match the ticket?	Agent that can see the ticket and the diff
Does the test suite pass?	CI, not a model
Is this safe to merge?	Named human reviewer

Where coding agents fit

An agent that opens a pull request shouldn't be the only reviewer of its own work. Run a second pass that ignores the agent's summary, or have a person read the ticket and the diff first. AI coding agents stay useful as long as that second look stays in place.

If the agent is Claude Code or Codex, give it a repository, a ticket, and a short rule file instead of a raw diff. The path from issue to pull request still ends with a human merge, and the agent's comments count as suggestions, not approval.

An MCP server lets a reviewer read a file, list call sites, or fetch the ticket without pasting the repo into the prompt. Falrow exposes this as a catalogue of MCP tools, so agents can plan from the git repository under the team's rules, with version-checked writes. Any MCP client can keep the same split: the bot reads, and a person merges.

What to measure

Ignore "comments generated." The useful question is whether review got cheaper without getting sloppier.

• Time from pull request open to first useful comment. The bot should bring this down.
• Share of bot comments the author accepts. A very low share means the instructions are noisy.
• Change fail rate on changes the bot reviewed. If it gets worse, narrow the bot's scope.

Track these for four weeks on one repo, with one person responsible for deleting bad instructions.

Review is also a planning problem. A Friday diff with no ticket won't get a careful pass. AI project management means scoped tickets exist before anyone writes code, so the intent sits next to the diff. A roadmap counted from real tickets shows whether delays come from staffing or from batch size. Falrow is one workspace for that: messages, calls, and Sentry errors become tickets, and agents plan from the repo. To start, a spreadsheet of the three measures above is enough.

Ways this goes wrong

Turn off bot approval on day one. Once a bot can sign off, authors stop reading.

Review the instructions every month and delete rules that CI already enforces. Your own rule file should win over habits copied from public repositories. Give the bot read access to the repo and the ticket, and nothing more. Let authors mark a comment "won't fix" with a one-line reason, then sample those reasons to find which rules to delete.

FAQ

Is AI code review accurate enough to replace a reviewer?

No. It catches local bugs and checklist misses, and it is often wrong in a confident voice. Keep a human on the merge for auth, data, and payments, and use the model to shorten the time to a first useful comment.

What should a review comment from a model include?

A file, a line or small range, what looks wrong, and why it matters. The best comments state the failing case, such as "empty string returns a zero price." "Consider refactoring" wastes the author's time. If the model is unsure, it should say what evidence is missing.

How is an AI powered code review different from lint and CI?

Lint and CI enforce rules that always have the same answer. A model comments on questions that depend on context: does this match the ticket, is this error handled the way the rest of the service handles errors, is a test missing for a branch the type checker can't see? Those comments are suggestions. Failing the build stays with deterministic checks.

Which changes should skip the automated review pass?

Generated code, lockfiles, and formatting diffs add noise and little signal. Skip the bot on a spike branch you plan to throw away. Don't skip it on small production diffs, because that's where empty-string bugs hide and where a model is cheapest to run.

Write the instructions down, keep approval with a person, and measure fail rate before you widen what the bot is allowed to say.