jorgecuadros-platform/docs/STATEMENT_OCR.md

Utility Statement OCR Capture (receipt capture)

Reads the stack of scanned utility bills the office pays every month, proposes the customer and the amount for each page, and posts the confirmed pages to the ledger as one batch against one check. Built 2026-08-01, live under /recibos.

This is the as-built record. The design and the reasoning behind it are RECEIPT_CAPTURE_SPEC.md §2, which also carries the measured results and the four spec corrections the real scans forced. Read that for why; read this for what is there.

The job it replaces

Staff receive 300+ pages per month per service provider — CFE electricity, CESPT water, Telnor phone, gas, municipal predial, federal zone — and key each one into the ledger by hand, as a charge against the customer whose property the bill belongs to. The whole stack is paid with one office check, so the capture is naturally a batch.

Auto-capture is a mode of the existing capture screen, not a separate feature: it is the same daily job with a scanner instead of a keyboard, and both modes post through the same ledger path.

What ships

Piece	Path
API module	apps/api/src/statements/ (service, controller, DTOs, matcher, parsers)
OCR seam	apps/api/src/statements/ocr/ (interface + Tesseract), bound in apps/api/src/ocr/ocr.module.ts
Tables	statement_batches, statement_documents (20260731235721_statement_ocr_intake)
Web	components/Captura.tsx (tab shell), StatementIntake.tsx (upload + batch list), /recibos/:id (review queue)
Abilities	statement:ingest, statement:review — both STAFF

STAFF is deliberate: the review step is what makes machine capture safe at that tier, since nothing reaches the ledger unconfirmed.

The screen

Captura is one screen with two ways in:

• /estado-cuenta/lote → manual tab (ManualCheckCapture, key each receipt against one check by hand)
• /recibos → automática (OCR) tab (StatementIntake, upload the scans)
• /recibos/:id → the batch review queue, page image beside the extracted fields

Both end in the same place — charges on customers' ledgers posted against one check — so they are modes of one screen. Staff pick by what is on the desk that morning. Either URL renders the same component, so old bookmarks land on the right tab.

Pipeline

upload PDFs (+ service kind) → store source → render pages → text layer? → parse → match → review → confirm → ledger

A batch is one service kind. The uploader labels it (ELECTRIC, WATER, TELEPHONE, GAS, PROPERTY_TAX, FEDERAL_ZONE) and that label is enforced: if the parser reads a page as a different provider, the page is rejected as mis-sorted rather than matched. Posting a phone bill as a water charge is the failure being prevented.

Processing is not awaited. 300 pages of OCR is minutes of CPU, far past any HTTP timeout, so POST /statements/batches returns the batch id immediately and the client polls. That is also what lets the review queue show partial progress.

One page = one document. Statements arrive bundled, one customer per page — Telnor's own Pág 3 de 6 is its internal pagination, not the office's scan — so every rendered page becomes its own StatementDocument and the parser runs per page. (The policy OCR feature inverts this; see "Sibling feature" below.)

One unreadable page must not abandon the other 299. A page that throws becomes a single OCR_FAILED row and the loop continues.

Both the source PDFs (statement/{batchId}/source-N.pdf) and every rendered page image (page-N.png) are stored. The source is the artifact the office received and the only way to re-run a corrected parser over the original; the page image is what the reviewer looks at, because "what the parser read" is only checkable against a picture of the paper.

The OCR seam

OcrProvider (ocr/ocr.provider.ts) is the swap point. Four methods: available(), renderPages(), recognize(), textPages().

Everything above it works in terms of page text and word boxes, so the engine is replaceable without touching the parsers, the matcher or the schema. The shipped implementation is self-hosted Tesseract, and that choice is evidence-based rather than assumed — see the spec's measured results. A managed API (Textract, Document Intelligence, Document AI) fits behind the same interface with no schema change; at 300+ pages/month/company it would carry real recurring cost for accuracy that is not the bottleneck.

The binding lives in apps/api/src/ocr/ocr.module.ts, extracted out of StatementsModule when POLICY_OCR.md needed the same seam. StatementsModule imports it and binds nothing itself, so the engine decision is one line in one file for both features.

Text layer first, OCR as the fallback

Not every statement is a scan. The gas company sends born-digital CFDI invoices whose text layer is already exact and already positioned. textPages() reads it (pdftotext -bbox-layout, same poppler package as pdftoppm) and OCR runs only where there is none.

Rasterising a born-digital page and re-recognising it can only lose information — one sample turned MEDIDOR: VM01014426 into ar (LTR): 014420 — while costing about a minute of CPU for the privilege. Positions come back in the same pixel space recognize() uses, so the parsers' geometric helpers work unchanged on either source. When the text layer is used the document's notes say so verbatim: "texto leído del PDF original, sin OCR".

Word boxes, not just text

OcrPage carries words[] with pixel boxes because several real layouts are tables: the CESPT "RECIBO" prints No. DE CUENTA as a column header with the value in the row beneath it, which line-oriented text cannot associate. Parsers fall back to geometry for exactly those fields.

Parsers

Eight providers, dispatched by a BRAND table checked before a LAYOUT table:

Provider	Service kind
CFE	ELECTRIC
CESPT	WATER
TELNOR	TELEPHONE
GAS TIJUANA	GAS
PREDIAL TIJUANA / PREDIAL ROSARITO / PREDIAL ENSENADA	PROPERTY_TAX
ZONA FEDERAL TIJUANA	FEDERAL_ZONE

Three predial parsers rather than one because Tijuana, Rosarito and Ensenada issue three completely different documents — same tax, nothing else in common.

Rules that are load-bearing and easy to break:

• Brand before layout, and never interleaved. Scanned logos read badly (a CESPT header came back as E BAJA ES PAGO / EALIFORNIA), which is why the layout fallback exists — but every brand rule runs first, because a Telnor page contains words a CFE structural rule would otherwise claim.
• Tijuana bills predial and zona federal from the same treasury. Same header, same address, same ATB-541201 RFC, so every predial discriminator matches a zona federal page too. The words only that layout prints are Marítimo Terrestre, so its rule is asked ahead of all three predial ones. Order matters here in a way that is invisible from the code shape.
• Parse amounts by separator position. A real Telnor bill OCR'd as $ 649,00; stripping commas as thousands separators makes that $64,900.
• A misread $ is the dangerous failure, not a missing one. An Ensenada receipt for $2,203.00 OCR'd as 82,203.00 — the sign read as an 8, which would post a charge 37× too large and look entirely ordinary in the ledger. Every predial amount therefore requires a literal $; a page that cannot produce one reports no amount and goes to review.
• Digit-confusion repair only on fields known to be digits (O→0, S→5, B→8, …), never on free text.
• The clave catastral is not [A-Z]{2}[0-9]{6}. Position three is a letter in fifteen of the 932 stored claves (MMB01041, CGH52121). Digitising the whole tail maps that B to an 8 and yields a key matching no property.
• Barcodes beat printed labels. Where a provider prints a payment barcode it is preferred and the two are cross-checked; disagreement sets crossChecked: false and forces review, because which of the two was misread is a judgement call.

Matching

StatementMatcherService. Two rules govern everything:

Match on one scoped field, never fuzzily across all identifiers. Each service kind has exactly one column its statements print, and only that column is consulted. A blanket search over accountNumber/meterNumber/route would let a water account number collide with an unrelated phone number, and the mis-post would look perfectly ordinary in the ledger.

Never match on the customer name. A CESPT receipt for account 5365218 prints ARNAIZ ROSAS ELSA AURORA; the office's book, corroborated by the clave, has CATT, RANDY. The name on a utility bill is the registrant, not the current owner. Names are shown to the reviewer and are never an input.

scopedRefField — which column each kind actually prints

Kind	Column	Why
ELECTRIC, WATER, TELEPHONE, CABLE	accountNumber	the legacy column holds the printed number
GAS	meterNumber	the number lived in free-text notes; accountNumber was never populated
PROPERTY_TAX	meterNumber	accountNumber holds DATMEX.predial, an office file number that is neither unique nor printed anywhere
FEDERAL_ZONE	meterNumber	accountNumber holds DATMEX.zfed, which is a peso amount, not a reference

scopedRefField is exported because three places must agree on the answer: the lookup, the blank-service fill on review, and the write-back on confirm. When they disagree a reference gets learned into a column nothing searches, and the same page returns to the review queue every month forever.

The FEDERAL_ZONE case is the sharpest instance of a trap this codebase hits repeatedly (see also policies.total): a legacy column whose name promises an identifier and whose contents are something else. Three of its 77 values carry cents and one is negative. Worse than never matching — because every row already has a value, the [field]: null guards on learning and on the blank-service fill would never fire either.

The clave catastral is a rescue on some layouts and the primary key on others

CESPT bills print the clave as well as an account number, so it rescues a page whose account number did not OCR — which happened on real samples. There it stays a hint.

On Rosarito and Ensenada predial the receipt prints nothing else, so a unique clave hit is a real match and auto-matches. Tijuana predial prints no clave at all; its only identifier is an 8-digit municipal account carried in a 32-digit payment barcode (account(8) + DDMMYY + amount(9) + folio(9)) that the legacy database never held, so those pages start cold and are taught by the first confirm.

Multiple hits are always surfaced, never auto-picked — duplicate account numbers do occur in the legacy data, and the office's own DUPLICADOS report existed for a reason.

Confirm: what gets written

confirmBatch posts through BillingService.createBatch — the same method the manual Editor screen uses — rather than writing Transaction rows directly, so OCR-sourced and hand-keyed receipts share one write path, one validation path and one audit trail.

• source: "OCR" and a per-line captureRef of the document id feed the duplicate-post guard, so a batch confirmed twice cannot double-charge.
• items[i] is positionally parallel to lines[i] (a documented seam guarantee), so the created rows zip straight back onto the documents that produced them via postedTransactionId.
• The sign is applied here. Charges are negative in this ledger; the parser reads the printed positive figure, and -Math.abs() is applied at the single point where a statement becomes a ledger row.
• A missing amount blocks the confirm with the offending page numbers, rather than silently posting zero.

Learning: the cold start is a one-time cost

After posting, learnAccountRefs writes each confirmed reference back onto the PropertyService that matched — only where the field was null. Never overwrites a number already on file, which would let one misread page rewrite good reference data.

This is what turns gas (whose numbers the migration never populated) and Tijuana predial (whose municipal account the legacy database never held) from a permanent review queue into a one-time cost: next month's statement for the same account matches on its own.

Discarding

Refused once any page is POSTED — those pages already wrote ledger rows against a check, and a "discarded" label on the batch would leave the charges unexplained. Reject the remaining pages individually instead.

API surface

Method	Route	Ability
GET	/statements/status (is OCR + storage available)	authenticated
GET	/statements/batches, /batches/:id, /batches/:id/documents	authenticated
GET	/statements/documents/:id/page (streams the page image)	authenticated
POST	/statements/batches (upload + service kind)	statement:ingest
PATCH	/statements/documents/:id (correct a field or the match)	statement:review
POST	/statements/documents/:id/reject	statement:review
POST	/statements/batches/:id/discard	statement:review
POST	/statements/batches/:id/confirm	statement:review

Requirements

Object storage (S3_ENDPOINT + credentials) for the scans, and tesseract-ocr / tesseract-ocr-data-spa / poppler-utils in the API image. Both are checked at upload rather than at the first write — a missing dependency should be a 400 on the request, not a FAILED batch minutes later. GET /statements/status reports both and the upload card hides itself unless both hold.

Tests

• parsers/statement-parser.spec.ts — detectProvider, normalizeCadastralKey, and one suite per newer parser (parsePredialTijuana, parsePredialRosarito, parsePredialEnsenada, parseGas, parseZonaFederal). Every fixture is a verbatim OCR excerpt from a real receipt, including the ones that bit: the 82,203.00 Ensenada dollar sign, the three-letter clave, the CESPT logo garbage.
• ocr/tesseract.provider.spec.ts — parseBboxLayout, the pdftotext -bbox-layout reader that produces the text-layer OcrPage (positions included, which is what lets the geometric helpers work on born-digital input).

Note the CFE / CESPT / Telnor parsers themselves have no unit suite — they predate the gas/predial extension and were verified against the 46-page corpus end to end rather than in isolation. Worth closing if they are touched.

Not built

• Handwritten folder numbers. Staff pencil a customer number on each bill (9, 405); Tesseract read 405 as 205. Handwriting is a review hint at best and is deliberately not an input to matching.
• Re-running a corrected parser over a stored batch. The source PDFs are kept precisely so this is possible, but nothing exposes it yet.
• Providers beyond the eight above. Adding one is a BRAND entry, an optional LAYOUT entry, and a parser function.

Sibling feature

POLICY_OCR.md — the same pipeline reading carrier policy PDFs into Policy rows, built out of this one. It reuses the seam, the provider-detection ordering, the digit-confusion map and the amount-by-separator rule.

One assumption does not carry over. Here a page is a document, because statements arrive one customer per page. A policy PDF is one document across several pages, so that feature concatenates the pages and parses once per file. If you are porting a change between the two, that is the difference to check first.