Why Two Neural Networks Work on a Document Instead of One — and the Second Is Built to Argue with the First

A model cannot check itself: ask it "are you sure?" and it says yes. A look at why two roles work on the same document, why the second one has a different job rather than running the same pass twice, and the mistake we made when the Reviewer started reviewing the Auditor.

Suppose an AI has gone through a specification and produced a finding: the area in the column doesn't reconcile with the geometry on the plan, and the discrepancy is tenfold.

A good finding. What remains is working out whether it's true.

The most natural thing that comes to mind is to ask the same model: "Are you sure?" We tried it. The model said it was sure. We rephrased — "check yourself again." The model checked and confirmed. The trouble is that it confirms an invention with exactly the same composure: the network that named a non-existent client sixty-one times in a row answered questions about its own confidence just as calmly.

This article is about why the thing that checks a model's answer can't be the model itself — and how the second role, whose job is to argue with the first, is put together.

One model versus a pair of roles: self-checking returns the same answer, an independent check asks a different question

"Are you sure?" isn't a check. It's a request to repeat the answer, and the model honestly obliges.

Why a model can't check itself

The reason isn't laziness, and it isn't the quality of the model. It's structural.

The answer a model has already given becomes part of its context. When you ask it to re-check, it isn't re-reading the document — it's re-reading its own conclusion and looking for support. This isn't malice: it's how text continuation works. A statement already written above is the strongest available signal about what to write next.

Two consequences follow, and neither is pleasant.

First: the errors correlate. Run the same model on the same task twice and you don't get two independent attempts, you get two very similar ones. What it failed to make out the first time it will most likely fail to make out the second, for the same reason it missed it initially: the lettering is too small, the table fell awkwardly, the sheet it needed wasn't in the run. Repeating a pass cures a random slip and does nothing for a systematic one — and it's the systematic ones that are expensive.

Second: asking about confidence yields no information. The number a model produces in response to "rate your confidence" isn't a measurement. It's one more generated token, like all the others. It sounds convincing and bears almost no relation to whether the answer is correct.

Only one conclusion follows from those two points: a second pass is worth anything only if it has a different job. Not "do the same thing again", but "do what the first one didn't do".

The second agent isn't "another pass" — it's different work

That's how the pair of roles came about.

The first role is the Auditor. Its job is to find. It opens sheets, writes out values, recomputes quantities, reconciles the table against the geometry, and states findings with an address: the sheet, the line, and the figures that produce a different number.

The second role is the Reviewer. Its job is to doubt. Not "look at the document again with the same eyes", but put questions to the assembled material that the Auditor never put to itself:

  • Is this really a discrepancy — or two numbers about different things? Area to the axes and area in the clear differ legitimately, and experience shows that most false findings are born on pairs exactly like that.
  • Is the basis actually given? If a finding rests on a value that isn't in the document, it isn't a finding, it's a hypothesis.
  • What has not been checked here? The Auditor found a discrepancy on one item — did it go through the remaining items of the same table, or stop at the first?
  • Is anything on the list the same thing recorded twice in different words?

The key difference from "check yourself again" is that the Reviewer isn't trying to reproduce the same result. It works with someone else's result as material — and that's the only mode in which "are you sure?" stops being a rhetorical question.

The mistake we made: the Reviewer started reviewing the Auditor

The first working version of the pair looked sensible: "here are your colleague's findings, assess them." The result was unexpected.

The Reviewer began writing remarks about the Auditor's work rather than about the document. "The justification is not set out in sufficient detail", "the counting methodology is not disclosed", "it is recommended that the wording be clarified." All of it competent, polite and entirely useless: the client needs a list of defects in the documentation, and what they got was a review of our internal kitchen.

The cause is simple, and once again it isn't about model quality but about how the task was framed. If the subject under review is someone else's work, the remarks will be about the work. The model followed the instruction literally; the instruction was wrong.

The fix is a single sentence: the subject of any finding remains the documentation, not the actions of another role. The Reviewer may withdraw a finding, may demand proof, may add one of its own — but everything it writes has to be a statement about the document being checked, not about whoever checked it.

The declared subject decides everything: while the Auditor's work was under review the model wrote a critique of methodology; once the document became the subject, findings followed

Same role, same model. All that changed was the declared subject of the remark — and with it the entire result.

The lesson runs wider than multi-agent systems. When an AI behaves strangely, the first question isn't "what's wrong with the model" but "what exactly did I ask it for". In the overwhelming majority of cases the model does precisely what's written, including the parts you wrote without thinking.

The argument is bounded — and it is asymmetric

Two roles that can argue raise two new questions: when to stop, and whom to believe.

When to stop. The argument is finite by construction: the number of rounds is capped in advance. The pair usually converges within two or three rounds — disagreements are either settled by evidence or turned into separate findings. If they don't converge, that isn't a malfunction, it's a result: disagreement between two roles is a signal that the question needs a human. There'll be a separate article on how an AI ought to stop and ask.

Whom to believe. Here we had to admit something uncomfortable: the two possible errors don't cost the same.

If the Reviewer leaves a false finding in, a human will see it: the finding has an address, checking takes a minute, and the price of the error is a minute of irritation.

If the Reviewer withdraws a true finding, it never comes back. It isn't in the report, nobody knows it existed, and it surfaces on the building site.

The errors are asymmetric, so the bar has to be asymmetric too. Keeping a finding requires nothing. Withdrawing one requires naming a reason, and the reason has to be checkable: "the value is corrected on sheet 14", "this duplicates the finding above", "the requirement belongs to a different section". Words like "no longer relevant" or "clarified", with nothing to point at, are not reasons.

One admission deserves its own line: what pulls at the Reviewer hardest is neither confirming nor refuting, but declaring the check impossible. "There is insufficient data for a conclusion" is a universally safe answer that looks careful and professional. Sometimes it's true: documentation genuinely can leave nothing to check against, and that is a defect in itself. But as a habit it eats the analysis alive — which is why declaring something underivable is allowed only after showing that the operands were looked for and not found.

The two errors cost differently: a false finding left in costs a minute of checking, a true one withdrawn surfaces on the building site

It's this asymmetry that makes withdrawing a finding harder than keeping one — and that is deliberate.

What we're not claiming

Two models can be wrong in the same way. A pair removes the correlation partly, not completely: the roles have different jobs but a shared nature. If the lettering physically can't be read, neither the first nor the second will read it. The pair protects against carelessness and hasty conclusions — not against a document that was submitted badly. Which is why the preparation checklist isn't going anywhere.

A pair costs more. A second agent means additional passes over the same material, and you pay for what gets read, not for the answer. An analysis with an opponent costs several times what a solo one does. That's a deliberate trade: an analysis you can't trust is cheaper and useless.

The Reviewer is not a guarantee. It cuts down the share of noise and forces checks to be carried through. It doesn't turn a conclusion into truth, and it doesn't change the fact that a human signs off on the project.

What to do about it

If you're choosing a tool that "checks documents with a neural network", two questions save a lot of time.

First: who checks the answer? If the answer is "we ask the model to re-check itself", you already know what that's worth. Self-checking returns the same answer in the same tone of voice.

Second: what happens on disagreement? The answer "that doesn't happen with us" is worse than "the discrepancy goes to a human". A system in which two checks never diverge is a system where the second check does nothing.

To see what an analysis looks like when every finding carries an address and a basis, take a look at FIBRA's open demo report.

The pair's working loop: findings, objections, proof or withdrawal with a named reason, then convergence or handover to a human

The argument is finite, and disagreement after the last round isn't a breakdown — it's grounds for calling in a human.

In the next article: where that boundary runs — when an AI is obliged to stop and put a question to a human instead of picking an answer on its own.

For the technical audience: a couple of clarifications

  • "Different jobs" here doesn't mean different models: the roles can be played by one and the same model under different instructions. What has to be separated is the task, not the vendor. Different models do have an effect too, but a substantially smaller one than separating the task — and they cost noticeably more.
  • Capping the number of rounds isn't an optimization, it's a necessity: without it the pair collapses into endless refinement of wording, where every round costs money and adds no findings.
  • The requirement to name a reason for withdrawal is the one piece of mechanics worth remembering here: without it the reviewing role degenerates into a convenient way of shortening the list, and the quality of the analysis formally rises while actually falling.