Which extractions need a human to look at them?

Most teams try to build a single confidence threshold and end up either drowning in review or quietly accepting errors. Three buckets works better, because it separates "look at this document" from "confirm this one field".

Auto-accept: every deterministic check passes, the document type is routine, and no high-consequence field was flagged. Field verification: one or two specific fields are shown to a person with the source image beside them, and they either confirm or correct. Full review: the document is opened and worked by hand, because something structural failed — the arithmetic does not balance, the document type is unexpected, or the file is unreadable.

The middle bucket is where the leverage is. A reviewer confirming two highlighted fields takes seconds; a reviewer re-reading a whole document takes minutes.

In the Dokyumi response, confidence is a map of field names to values between 0 and 1, produced by the model alongside the data. Fields can be missing from that map entirely. Scores below the schema threshold — 0.8 by default — appear in validation.low_confidence_fields and set the overall status to review.

Read that as a signal, not a probability. A low score is a good reason to look; a high score is not proof of correctness. This is not a Dokyumi quirk, it is how model-reported confidence behaves generally, and any vendor claiming calibrated per-field probabilities should be asked how they measured it and on which document set.

These triggers do not care what the model thought. They are computed from the extracted values and your own data, they behave identically every time, and they catch confident errors — the ones confidence-based routing misses by definition.

• Arithmetic failure: line items do not sum, or subtotal plus tax does not equal the total.
• Referential failure: the vendor, member or account number does not exist in your records.
• Novelty: first document ever seen from this sender, or first of this document type this quarter.
• Change of banking or payment details versus the record you already hold.
• Out-of-range value: more than an order of magnitude from this counterparty’s trailing median.
• Missing required field: a required key came back null.
• Structural surprise: page count, currency, or language different from every previous document from that sender.

A review policy is only manageable if you measure it. Four numbers tell you everything you need, and all four come from your own logs rather than a vendor dashboard.

Auto-accept rate: the share of documents that cleared every gate. Review rate: the share that needed a person, split by trigger so you can see which rule generates the work. Correction rate within review: how often the reviewer actually changed something — if it is very low, the trigger is too sensitive. Escaped-error rate: errors found downstream, by a supplier complaint or a reconciliation, that your gates did not catch. That last one is the only measure of whether the policy is working, and it is the one nobody instruments until after the first bad month.