How to review an AI-generated pull request
I start an AI-generated PR review with the contract. What was supposed to change? Then I follow that behavior through the production code and read the tests after it.
An agent can update a type, parser, two callers, and a test in one pass. All five files agree. That still leaves a question: do they agree on the right thing?
If you want a model's second pass, use this code review prompt to supply the contract and ask for evidence you can verify. Keep the approval decision with the person reading the code.
A quick review path:
- Establish the intended behavior before reading the summary.
- Follow a value across production code and tests.
- Check unchanged boundaries and extra changes.
- Leave questions that name the missing evidence.
On this page
Start with intent, not the PR summary
Suppose you ask an agent to rename timeout to timeoutMs. It updates the references. CI passes. The description says “updated callers.”
You could approve this as a mechanical rename. But did you want a new name, a conversion from seconds to milliseconds, or a compatibility period that accepts both fields?
Those are three different changes. The patch can look consistent for any of them.
I'd write down the expected behavior before reading the implementation. For an author checking agent-written work, this is also a useful point to compare the patch with the original request.
Follow the value across files
Start with the type and parser. Does the old name still work? If both names are present, which wins? If neither is present, what default reaches the caller?
Then read both callers. A variable called timeoutMs can still contain seconds. Renaming it doesn't multiply anything by 1,000.
Check the failure paths and output too. Search the repository for consumers the PR didn't change. The agent's summary won't necessarily mention an unchanged caller that now has the wrong contract.
Turn the request into observable examples
Before opening more files, write two or three examples in plain language. For the timeout migration, an old client sending five seconds might still need a five-second timeout. A new client sending 5,000 milliseconds should reach the same duration. An omitted value should use the agreed default. These examples make the requirement precise enough to compare with code.
Choose examples that expose different interpretations. If both the old and new contract accept the number 1,000, testing only that number can hide a conversion mistake. Include the name and unit of the input, the expected internal value, and the externally visible result. You now have a trace to follow through the parser and outbound request.
If you cannot write the expected result, stop treating the patch as a straightforward rename. Ask the author or product owner to choose the behavior. Neither the agent's confidence nor a clean typecheck can supply a missing compatibility decision.
Inspect the boundaries the patch did not change
Start with one real entry point: a request handler, command-line option, configuration loader, or public function. Follow the input until it reaches an effect. Types inside the changed files may agree while an older configuration file or external caller still uses the previous representation.
Search for the old field name and for functions that receive the value. Read unchanged callers where necessary. Check whether a default is applied before or after conversion, and whether validation rejects zero, negative values, or unsupported combinations. The important question is what a real input does, not whether every reference was renamed.
Look at outputs as well. Error messages and logs can retain the old unit and mislead the next person diagnosing a failure. A public response may expose the changed field to another service. When a dependency sits outside the repository, ask for an explicit compatibility assumption and evidence from the author rather than assuming repository search covered it.
Read production code before tests
A test that passes 1000 and expects 1000 proves that the value survived. It doesn't establish a unit conversion or backward compatibility.
Pick an input that separates the possible behaviors. Five seconds. The old field name. No field at all.
Then ask whether the assertion would catch the mistake you're considering. A test name can say “supports timeouts” while proving much less.
I want to understand the implementation before I read the test's explanation of it.
Check whether the test could disagree with the implementation
A test loses value when its expected result is computed through the same helper as the production path. If both use the same incorrect conversion, they can agree while the behavior is wrong. Inspect the expected value and ask whether it comes from the requirement or merely mirrors the implementation.
Prefer a small example with a result you can explain independently. For a five-second input, you should be able to state the expected outbound duration without running the conversion helper yourself. For compatibility, send the old request shape through the actual parser. For a rejected input, verify the observable error or absence of a side effect, rather than only a private helper's return value.
Run the focused repository check after reading the test. Record the command and result accurately. If you cannot execute it because a service or credential is missing, say so and identify which behavior remains unverified. Reading an assertion and observing it pass are different pieces of evidence.
Review the agent's extra changes on their own merits
Agents sometimes tidy adjacent code while implementing the request. Inventory those edits: dependency changes, new helpers, renamed files, altered error handling, and removed tests. Ask how each supports the stated behavior. A cleanup can be sensible and still increase the amount of code you must understand before approving.
Pay particular attention to deleted assertions. A shorter test suite might remove duplication, or it might remove the only compatibility example. Compare the old behavior being checked with the new evidence. Do not infer the purpose of a deleted test solely from its replacement's title.
When an unrelated change creates a separate decision, ask to move it to another PR. If it must stay, request an explanation of the dependency and review it explicitly. The goal is to leave no behavior approved accidentally because it appeared beside the change you intended to read.
Use review units to keep related hunks together
Guided Review groups real diff hunks into ordered review units with commentary. Production changes and tests appear as separate units. On a github.com PR, the Chrome extension puts that walkthrough in an overlay. Before a PR exists, there's the local CLI workflow.
Yes, I'm describing my own tool. The part I care about is getting the contract and its callers in front of me without reconstructing the reading order from file names.
The model supplies structure and commentary. Displayed code comes from the parsed diff; invalid hunk references are dropped. That check can reject an invented reference. It can't tell me whether the timeout should be five seconds.
Leave a question someone can answer
“This caller still passes seconds. Should the value be converted, or is the new name wrong?”
That's a review comment with a decision attached. “Please check the timeout logic” leaves the author to repeat the investigation.
The extension can draft line comments and submit Comment, Approve, or Request Changes. You choose the outcome. Guided Review doesn't produce an automated findings verdict or act as a merge gate.
For a three-line patch whose behavior is obvious, I'd just read the normal diff.
For this patch, I'd want an answer about the units before approving.
After the author answers, check the updated diff rather than approving the explanation alone. Confirm that the chosen contract appears in production code, that a meaningful test distinguishes it from the original mistake, and that any rollout constraint is recorded in the PR. That gives the next reviewer a durable account of why the change is acceptable.
FAQ
Should I review AI-generated code differently from human-written code?
The review standard is the same, but start by recovering intent. Agent-written changes can be broad and internally consistent while still implementing the wrong contract.
Do passing tests make an AI-generated pull request safe to approve?
No. Tests show that selected examples pass. Read the production path first, then check whether the tests distinguish the intended behavior from a plausible mistake.
Does Guided Review decide whether to approve the pull request?
No. It groups real diff hunks into an ordered walkthrough and adds commentary. You read the code and choose Comment, Approve, or Request Changes.