A code reviewer that has to cite its sources

On the core banking system I work on, a merge request is rarely just code. It carries a Jira ticket with acceptance criteria written by a banking analyst, a Confluence page describing the entities involved, an OpenAPI contract for the service it talks to, and a thread of discussion on GitLab. A reviewer who reads only the diff reviews the wrong thing. The question is never “is this TypeScript correct” but “does this do what the ticket says, against the contract the backend actually exposes”.

That reading takes time, and we have more merge requests than people willing to do it properly. So I built a reviewer that does the reading first.

What it retrieves

For each merge request, the agent pulls four things: the acceptance criteria from the linked Jira issue; the entity specifications from Confluence for the domain objects the diff touches; the diff itself and the existing discussion from GitLab; and the live Swagger contract of the services the code calls. That last one matters more than it sounds. Half the defects on a multi-team monorepo come from a client written against a contract that changed last week, and the reviewer who catches that is the one who looked at today’s contract, not the one in the ticket.

Only then does it read the code.

What it writes

Two kinds of comments. Inline ones, anchored to a line, for things a reviewer would point at: a field the contract does not return, a criterion from the ticket that has no code path, a request made without the idempotency key the service requires. And one general comment at the top that says what the merge request claims to do, what it actually does, and where the two differ.

Every comment says where its evidence came from. “Acceptance criterion 3 in TEFO-1482 requires X; nothing in this diff handles X.” “The /parties/{id} contract as of today has no segment field; line 41 reads one.” A reviewer can click through and check. That is the rule the whole thing is built on: a comment without a source is an opinion, and the team does not need more opinions on a merge request.

What it does not do

It does not approve. It does not merge. It does not rewrite code. It does not comment on style, because we have a linter for that and a model arguing about semicolons is a waste of everyone’s attention. Its scope is bounded on purpose: requirements, contracts, and the gap between them. A human still reads the merge request; the agent’s job is to make sure the human starts with the context already assembled and the obvious mismatches already flagged.

What changed

Reviewer load dropped, which was the point, but that is not the change I value most. The change is in what reviewers now spend their time on. When the mechanical checks are done before a person opens the diff, the person’s comments become about design: whether this belongs in the service layer, whether the accounting side effect is in the right transaction, whether the analyst’s criterion is even the right criterion. That is the conversation a review should be.

The second change was on the analysts’ side. When a ticket’s acceptance criteria are vague, the agent’s general comment says so, in front of everyone. Criteria got sharper within a month, because nobody enjoys being quoted.

Why this design and not a bigger one

Everything about a model inside a regulated product comes back to one question: can someone check it. Provenance, bounded scope, a human in the loop. A reviewer that cites Jira, Confluence, GitLab and Swagger is checkable line by line. A reviewer that says “this looks fine” is not, and in a bank, “looks fine” is not a category that exists.

I would build it the same way again, and I would start the same way: not with the model, but with the list of documents a good reviewer reads before they read the code.