feat(statements): OCR intake for scanned utility bills
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m41s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m18s

Staff key 300+ utility statements per company per month by hand. This adds
the ingest -> split -> OCR -> match -> review pipeline that proposes customer
and amount per page instead (RECEIPT_CAPTURE_SPEC §2), posting through the
existing BillingService.createBatch seam with source=OCR and a per-document
captureRef so machine and hand capture share one write path and audit trail.

Everything was designed against 10 real scanned statements (46 pages of CFE,
CESPT and Telnor bills) rather than from the sample-free spec. The scans have
no text layer at all — they are camera images — so OCR is mandatory, and they
arrive bundled one customer per page. Measured on those pages the parser
identifies the provider 46/46 and reads an account reference 43/46; against
the dev database that is 39/46 (85%) exact auto-match, 40/46 identified, with
the rest genuine review cases. That closes the OCR-provider question in favour
of self-hosted Tesseract: it clears the bar for a queue where a human confirms
every row, and OcrProvider keeps a managed API a one-line swap.

The samples corrected three things the spec had wrong or unknown:

- Clave catastral is NOT predial. DATMEX.clave (934 rows) is what CESPT and
  predial bills print; DATMEX.predial, which PROPERTY_TAX.accountNumber holds,
  has 663 distinct values across 1135 rows and appears on no statement. The
  clave now lives on Property.cadastralKey as the matcher's secondary key;
  predial is left untouched. This had been blocking predial matching.
- Gas was recoverable: 160 of 334 DATMEX.gas values are real account numbers
  (the rest are ESTACIONARIO/CILINDRO descriptors), now in GAS.meterNumber.
- Phone is one billed line per property (534/18/1 across phone1/2/3), so the
  new TELEPHONE ServiceKind backfills from phone1 only, not three rows.

Matching is scoped to one column per service kind and never reads the customer
name — a CESPT receipt prints ARNAIZ ROSAS ELSA AURORA for an account this
office holds under CATT, RANDY, because the printed name is the registrant,
not the current owner. Where a provider prints a payment barcode it beats the
printed label (one CFE label OCR'd a digit too many while its barcode was
correct) and the two cross-check, with disagreement forcing review.

Confirming a document whose service had no reference writes it back, so gas
and any other cold start is a one-time cost rather than a permanent queue.

Verified end to end against the live dev API and MinIO: real scans uploaded
over HTTP, matched, confirmed against a check, and the resulting rows checked
in MySQL (negative amounts, captureSource=OCR, concept derived from the batch
kind, captureRef linking back to each page). Re-confirming a posted batch is
refused. Test data was removed afterwards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-01 00:42:35 -07:00
co-authored by Claude Opus 5
parent 121952fdc1
commit 4d5008b545
26 changed files with 3077 additions and 19 deletions
+270
View File
@@ -0,0 +1,270 @@
"use client";
import { useCallback, useEffect, useState } from "react";
import Link from "next/link";
import { AppShell } from "@/components/AppShell";
import {
getStatementStatus,
listStatementBatches,
uploadStatementBatch,
} from "@/lib/api";
import { useCan } from "@/lib/abilities";
import { formatDate, SERVICE_KIND_LABELS, serviceKindLabel } from "@/lib/labels";
import type { ServiceKind, StatementBatch, StatementBatchStatus } from "@/lib/types";
/**
* Statement OCR intake (docs/RECEIPT_CAPTURE_SPEC.md §2).
*
* Each utility company mails 300+ paper bills a month, one per customer, which
* staff otherwise key in by hand through the "Captura" screen. Here they scan
* the stack, and the machine proposes customer + amount for every page; a
* human still confirms before anything reaches the ledger.
*
* One batch = one service kind, because the matcher is scoped per kind: a
* water account number and a phone number are compared against different
* columns, and mixing them in one upload is how a bill gets posted to the
* wrong customer.
*/
/** The kinds the parsers actually recognise today. */
const SUPPORTED: ServiceKind[] = ["ELECTRIC", "WATER", "TELEPHONE"];
/** Uploadable, but every page will land in review until a parser learns it. */
const OTHER_KINDS: ServiceKind[] = ["GAS", "PROPERTY_TAX", "FEDERAL_ZONE", "CABLE"];
const STATUS_LABEL: Record<StatementBatchStatus, string> = {
UPLOADED: "Recibido",
PROCESSING: "Procesando…",
READY_FOR_REVIEW: "Listo para revisar",
COMPLETED: "Registrado",
FAILED: "Falló",
};
export default function RecibosPage() {
return (
<AppShell>
<Recibos />
</AppShell>
);
}
function Recibos() {
const canIngest = useCan("statement:ingest");
const [batches, setBatches] = useState<StatementBatch[]>([]);
const [ocrAvailable, setOcrAvailable] = useState<boolean | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const load = useCallback(async () => {
try {
const [list, status] = await Promise.all([
listStatementBatches(),
getStatementStatus(),
]);
setBatches(list.items);
setOcrAvailable(status.ocrAvailable);
setError(null);
} catch (e) {
setError((e as Error)?.message ?? "No se pudieron cargar los lotes.");
} finally {
setLoading(false);
}
}, []);
useEffect(() => {
void load();
}, [load]);
// A batch of 300 pages takes minutes to OCR, so the list refreshes itself
// while anything is still working rather than making staff reload.
const working = batches.some(
(b) => b.status === "PROCESSING" || b.status === "UPLOADED",
);
useEffect(() => {
if (!working) return;
const t = setInterval(() => void load(), 4000);
return () => clearInterval(t);
}, [working, load]);
return (
<div className="stack">
<header className="page-head">
<div>
<h1 className="page-title">Recibos (OCR)</h1>
<p className="page-sub">
Escanea los recibos del mes y el sistema propone cliente e importe
para cada página. Nada se registra sin tu confirmación.
</p>
</div>
</header>
{ocrAvailable === false && (
<div className="state-box state-error">
Este servidor no tiene OCR instalado, así que no se pueden procesar
recibos. El resto del sistema funciona con normalidad.
</div>
)}
{canIngest && ocrAvailable && <UploadCard onDone={load} />}
{error && <div className="state-box state-error">{error}</div>}
<section className="card" style={{ padding: 16 }}>
<h2 className="section-title" style={{ marginTop: 0 }}>
Lotes
</h2>
{loading ? (
<div className="state-box">Cargando</div>
) : batches.length === 0 ? (
<div className="state-box">Todavía no hay lotes de recibos.</div>
) : (
<div className="tx-scroll">
<table className="tx-table">
<thead>
<tr>
<th>Fecha</th>
<th>Servicio</th>
<th>Referencia</th>
<th>Estado</th>
<th className="num">Páginas</th>
<th>Subido por</th>
<th />
</tr>
</thead>
<tbody>
{batches.map((b) => (
<tr key={b.id}>
<td style={{ whiteSpace: "nowrap" }}>{formatDate(b.createdAt)}</td>
<td>{serviceKindLabel(b.serviceKind)}</td>
<td>{b.label || "—"}</td>
<td>
<StatusTag status={b.status} />
{b.error && (
<div className="page-sub" style={{ marginTop: 4 }}>
{b.error}
</div>
)}
</td>
<td className="num">{b._count?.documents ?? 0}</td>
<td>{b.uploadedBy?.name ?? "—"}</td>
<td>
<Link className="btn btn-ghost" href={`/recibos/${b.id}`}>
Revisar
</Link>
</td>
</tr>
))}
</tbody>
</table>
</div>
)}
</section>
</div>
);
}
function StatusTag({ status }: { status: StatementBatchStatus }) {
return <span className="tag">{STATUS_LABEL[status] ?? status}</span>;
}
function UploadCard({ onDone }: { onDone: () => void }) {
const [files, setFiles] = useState<File[]>([]);
const [serviceKind, setServiceKind] = useState<ServiceKind>("ELECTRIC");
const [label, setLabel] = useState("");
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
async function submit() {
if (!files.length) return;
setBusy(true);
setError(null);
try {
await uploadStatementBatch(files, serviceKind, label.trim() || undefined);
setFiles([]);
setLabel("");
onDone();
} catch (e) {
setError((e as Error)?.message ?? "No se pudo subir el lote.");
} finally {
setBusy(false);
}
}
const unsupported = !SUPPORTED.includes(serviceKind);
return (
<section className="card" style={{ padding: 16 }}>
<h2 className="section-title" style={{ marginTop: 0 }}>
Subir recibos escaneados
</h2>
<div className="inline-form" style={{ flexWrap: "wrap", gap: 12 }}>
<label>
<span className="page-sub">Servicio</span>
<select
className="input"
value={serviceKind}
onChange={(e) => setServiceKind(e.target.value as ServiceKind)}
>
<optgroup label="Con lectura automática">
{SUPPORTED.map((k) => (
<option key={k} value={k}>
{SERVICE_KIND_LABELS[k] ?? k}
</option>
))}
</optgroup>
<optgroup label="Sin lectura automática (revisión manual)">
{OTHER_KINDS.map((k) => (
<option key={k} value={k}>
{SERVICE_KIND_LABELS[k] ?? k}
</option>
))}
</optgroup>
</select>
</label>
<label>
<span className="page-sub">Referencia (opcional)</span>
<input
className="input"
placeholder="ej. CFE julio 2026"
value={label}
onChange={(e) => setLabel(e.target.value)}
/>
</label>
<label>
<span className="page-sub">Archivos PDF</span>
<input
type="file"
className="input"
accept="application/pdf"
multiple
onChange={(e) => setFiles(Array.from(e.target.files ?? []))}
/>
</label>
<button
type="button"
className="btn btn-primary"
disabled={!files.length || busy}
onClick={submit}
>
{busy ? "Subiendo…" : `Procesar ${files.length || ""}`.trim()}
</button>
</div>
{unsupported && (
<div className="state-box" style={{ marginTop: 12 }}>
Todavía no hay lectura automática para{" "}
{SERVICE_KIND_LABELS[serviceKind] ?? serviceKind}: cada página quedará
para revisión manual. Al confirmarlas se guarda el número de cuenta,
así que los recibos del mes siguiente se reconocerán solos.
</div>
)}
<p className="page-sub" style={{ marginTop: 12 }}>
Un lote es de un solo servicio. Cada página del PDF se trata como un
recibo distinto, salvo que el proveedor imprima varias hojas por cliente.
</p>
</section>
);
}