Adds four parsers to the statement intake — GAS TIJUANA plus one per municipality, because Tijuana, Rosarito and Ensenada issue three completely different predial documents — and a text-layer fast path for the born-digital invoices the gas company sends. Measured against a new corpus of 14 documents / 29 pages: provider read on 29/29, amount on 26/29, and 21/29 auto-matched against the dev database (22/29 identified). The eight review cases are all legitimate. Five things the corpus forced: - Not every statement is a scan. The gas invoices are born-digital CFDIs whose text layer is exact; rasterising them only loses information (one sample turned `MEDIDOR: VM01014426` into `ar (LTR): 014420`). The new `OcrProvider.textPages` reads the embedded layer via `pdftotext -bbox-layout` — same poppler package as `pdftoppm`, so no new dependency — and OCR stays the fallback for real scans. Poppler's own `<line>` grouping follows text flow rather than the page, so words are regrouped by vertical position; without that, a two-column header leaves every label separated from the value printed beside it. - The clave catastral is not two letters and six digits. Position three is a letter in 15 of the 932 stored claves, and digitising the whole tail mapped a real `MMB01041` to a nonexistent `MM801041`. - Tijuana predial prints no clave at all. Its only identifier is an 8-digit municipal account carried in a 32-digit payment barcode, which the legacy database never held, so it goes in `meterNumber` alongside gas — `accountNumber` holds `DATMEX.predial`, which is not a per-property key and must not be overwritten. Those pages start cold and are taught by the first confirm. - On Rosarito and Ensenada the clave is the primary key, not a fallback: those receipts print nothing else, so a unique hit auto-matches. On a utility bill that merely happens to print one it stays a review hint. - A misread `$` is the dangerous failure. An Ensenada receipt for $2,203.00 OCR'd as `82,203.00`, which would post a charge 37x too large and look ordinary in the ledger. Predial amounts now require a literal `$` and a page that cannot produce one goes to review. The scoped match field is now one exported function rather than three copies of `kind === "GAS" ? ... : ...`, since the lookup, the blank-service fill and the confirm write-back have to agree or a reference gets learned into a column nothing searches. First tests in this package: 23 specs over the parsers and the text-layer reader, every fixture a verbatim OCR excerpt from a real receipt. Adds the jest config they need and a build tsconfig so they stay out of dist. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
279 lines
9.2 KiB
TypeScript
279 lines
9.2 KiB
TypeScript
"use client";
|
|
|
|
import { useCallback, useEffect, useState } from "react";
|
|
import Link from "next/link";
|
|
import {
|
|
getStatementStatus,
|
|
listStatementBatches,
|
|
uploadStatementBatch,
|
|
} from "@/lib/api";
|
|
import { useCan } from "@/lib/abilities";
|
|
import { formatDate, SERVICE_KIND_LABELS, serviceKindLabel } from "@/lib/labels";
|
|
import type { ServiceKind, StatementBatch, StatementBatchStatus } from "@/lib/types";
|
|
|
|
/**
|
|
* Automatic capture — statement OCR intake (docs/RECEIPT_CAPTURE_SPEC.md §2).
|
|
*
|
|
* Each utility company mails 300+ paper bills a month, one per customer, which
|
|
* staff otherwise key in by hand on the manual tab of the same "Captura"
|
|
* screen. Here they scan the stack instead, and the machine proposes customer +
|
|
* amount for every page; a human still confirms before anything reaches the
|
|
* ledger. Same daily job, same ledger path — only the input differs, which is
|
|
* why it lives as a mode of Captura rather than a screen of its own.
|
|
*
|
|
* One batch = one service kind, because the matcher is scoped per kind: a
|
|
* water account number and a phone number are compared against different
|
|
* columns, and mixing them in one upload is how a bill gets posted to the
|
|
* wrong customer.
|
|
*/
|
|
|
|
/** The kinds the parsers actually recognise today. */
|
|
const SUPPORTED: ServiceKind[] = [
|
|
"ELECTRIC",
|
|
"WATER",
|
|
"TELEPHONE",
|
|
"GAS",
|
|
"PROPERTY_TAX",
|
|
];
|
|
/** Uploadable, but every page will land in review until a parser learns it. */
|
|
const OTHER_KINDS: ServiceKind[] = ["FEDERAL_ZONE", "CABLE"];
|
|
|
|
const STATUS_LABEL: Record<StatementBatchStatus, string> = {
|
|
UPLOADED: "Recibido",
|
|
PROCESSING: "Procesando…",
|
|
READY_FOR_REVIEW: "Listo para revisar",
|
|
COMPLETED: "Registrado",
|
|
FAILED: "Falló",
|
|
};
|
|
|
|
export function StatementIntake() {
|
|
const canIngest = useCan("statement:ingest");
|
|
const [batches, setBatches] = useState<StatementBatch[]>([]);
|
|
const [ocrAvailable, setOcrAvailable] = useState<boolean | null>(null);
|
|
const [storageAvailable, setStorageAvailable] = useState<boolean | null>(null);
|
|
const [loading, setLoading] = useState(true);
|
|
const [error, setError] = useState<string | null>(null);
|
|
|
|
const load = useCallback(async () => {
|
|
try {
|
|
const [list, status] = await Promise.all([
|
|
listStatementBatches(),
|
|
getStatementStatus(),
|
|
]);
|
|
setBatches(list.items);
|
|
setOcrAvailable(status.ocrAvailable);
|
|
setStorageAvailable(status.storageAvailable);
|
|
setError(null);
|
|
} catch (e) {
|
|
setError((e as Error)?.message ?? "No se pudieron cargar los lotes.");
|
|
} finally {
|
|
setLoading(false);
|
|
}
|
|
}, []);
|
|
|
|
useEffect(() => {
|
|
void load();
|
|
}, [load]);
|
|
|
|
// A batch of 300 pages takes minutes to OCR, so the list refreshes itself
|
|
// while anything is still working rather than making staff reload.
|
|
const working = batches.some(
|
|
(b) => b.status === "PROCESSING" || b.status === "UPLOADED",
|
|
);
|
|
useEffect(() => {
|
|
if (!working) return;
|
|
const t = setInterval(() => void load(), 4000);
|
|
return () => clearInterval(t);
|
|
}, [working, load]);
|
|
|
|
const ready = ocrAvailable === true && storageAvailable === true;
|
|
|
|
return (
|
|
<div className="stack">
|
|
{ocrAvailable === false && (
|
|
<div className="state-box state-error">
|
|
Este servidor no tiene OCR instalado, así que no se pueden leer recibos
|
|
escaneados. Usa la captura manual; el resto del sistema funciona con
|
|
normalidad.
|
|
</div>
|
|
)}
|
|
|
|
{storageAvailable === false && (
|
|
<div className="state-box state-error">
|
|
Este servidor no tiene configurado el almacenamiento de documentos, así
|
|
que no hay dónde guardar los recibos escaneados. Usa la captura manual
|
|
mientras se configura.
|
|
</div>
|
|
)}
|
|
|
|
{canIngest && ready && <UploadCard onDone={load} />}
|
|
|
|
{error && <div className="state-box state-error">{error}</div>}
|
|
|
|
<section className="card" style={{ padding: 16 }}>
|
|
<h2 className="section-title" style={{ marginTop: 0 }}>
|
|
Lotes
|
|
</h2>
|
|
{loading ? (
|
|
<div className="state-box">Cargando…</div>
|
|
) : batches.length === 0 ? (
|
|
<div className="state-box">Todavía no hay lotes de recibos.</div>
|
|
) : (
|
|
<div className="tx-scroll">
|
|
<table className="tx-table">
|
|
<thead>
|
|
<tr>
|
|
<th>Fecha</th>
|
|
<th>Servicio</th>
|
|
<th>Referencia</th>
|
|
<th>Estado</th>
|
|
<th className="num">Páginas</th>
|
|
<th>Subido por</th>
|
|
<th />
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
{batches.map((b) => (
|
|
<tr key={b.id}>
|
|
<td style={{ whiteSpace: "nowrap" }}>{formatDate(b.createdAt)}</td>
|
|
<td>{serviceKindLabel(b.serviceKind)}</td>
|
|
<td>{b.label || "—"}</td>
|
|
<td>
|
|
<StatusTag status={b.status} />
|
|
{b.error && (
|
|
<div className="page-sub" style={{ marginTop: 4 }}>
|
|
{b.error}
|
|
</div>
|
|
)}
|
|
</td>
|
|
<td className="num">{b._count?.documents ?? 0}</td>
|
|
<td>{b.uploadedBy?.name ?? "—"}</td>
|
|
<td>
|
|
<Link className="btn btn-ghost" href={`/recibos/${b.id}`}>
|
|
Revisar
|
|
</Link>
|
|
</td>
|
|
</tr>
|
|
))}
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
)}
|
|
</section>
|
|
</div>
|
|
);
|
|
}
|
|
|
|
function StatusTag({ status }: { status: StatementBatchStatus }) {
|
|
return <span className="tag">{STATUS_LABEL[status] ?? status}</span>;
|
|
}
|
|
|
|
function UploadCard({ onDone }: { onDone: () => void }) {
|
|
const [files, setFiles] = useState<File[]>([]);
|
|
const [serviceKind, setServiceKind] = useState<ServiceKind>("ELECTRIC");
|
|
const [label, setLabel] = useState("");
|
|
const [busy, setBusy] = useState(false);
|
|
const [error, setError] = useState<string | null>(null);
|
|
|
|
async function submit() {
|
|
if (!files.length) return;
|
|
setBusy(true);
|
|
setError(null);
|
|
try {
|
|
await uploadStatementBatch(files, serviceKind, label.trim() || undefined);
|
|
setFiles([]);
|
|
setLabel("");
|
|
onDone();
|
|
} catch (e) {
|
|
setError((e as Error)?.message ?? "No se pudo subir el lote.");
|
|
} finally {
|
|
setBusy(false);
|
|
}
|
|
}
|
|
|
|
const unsupported = !SUPPORTED.includes(serviceKind);
|
|
|
|
return (
|
|
<section className="card" style={{ padding: 16 }}>
|
|
<h2 className="section-title" style={{ marginTop: 0 }}>
|
|
Subir recibos escaneados
|
|
</h2>
|
|
<div className="inline-form" style={{ flexWrap: "wrap", gap: 12 }}>
|
|
<label>
|
|
<span className="page-sub">Servicio</span>
|
|
<select
|
|
className="input"
|
|
value={serviceKind}
|
|
onChange={(e) => setServiceKind(e.target.value as ServiceKind)}
|
|
>
|
|
<optgroup label="Con lectura automática">
|
|
{SUPPORTED.map((k) => (
|
|
<option key={k} value={k}>
|
|
{SERVICE_KIND_LABELS[k] ?? k}
|
|
</option>
|
|
))}
|
|
</optgroup>
|
|
<optgroup label="Sin lectura automática (revisión manual)">
|
|
{OTHER_KINDS.map((k) => (
|
|
<option key={k} value={k}>
|
|
{SERVICE_KIND_LABELS[k] ?? k}
|
|
</option>
|
|
))}
|
|
</optgroup>
|
|
</select>
|
|
</label>
|
|
|
|
<label>
|
|
<span className="page-sub">Referencia (opcional)</span>
|
|
<input
|
|
className="input"
|
|
placeholder="ej. CFE julio 2026"
|
|
value={label}
|
|
onChange={(e) => setLabel(e.target.value)}
|
|
/>
|
|
</label>
|
|
|
|
<label>
|
|
<span className="page-sub">Archivos PDF</span>
|
|
<input
|
|
type="file"
|
|
className="input"
|
|
accept="application/pdf"
|
|
multiple
|
|
onChange={(e) => setFiles(Array.from(e.target.files ?? []))}
|
|
/>
|
|
</label>
|
|
|
|
<button
|
|
type="button"
|
|
className="btn btn-primary"
|
|
disabled={!files.length || busy}
|
|
onClick={submit}
|
|
>
|
|
{busy ? "Subiendo…" : `Procesar ${files.length || ""}`.trim()}
|
|
</button>
|
|
</div>
|
|
|
|
{error && (
|
|
<div className="state-box state-error" style={{ marginTop: 12 }}>
|
|
{error}
|
|
</div>
|
|
)}
|
|
|
|
{unsupported && (
|
|
<div className="state-box" style={{ marginTop: 12 }}>
|
|
Todavía no hay lectura automática para{" "}
|
|
{SERVICE_KIND_LABELS[serviceKind] ?? serviceKind}: cada página quedará
|
|
para revisión manual. Al confirmarlas se guarda el número de cuenta,
|
|
así que los recibos del mes siguiente sí se reconocerán solos.
|
|
</div>
|
|
)}
|
|
|
|
<p className="page-sub" style={{ marginTop: 12 }}>
|
|
Un lote es de un solo servicio. Cada página del PDF se trata como un
|
|
recibo distinto, salvo que el proveedor imprima varias hojas por cliente.
|
|
</p>
|
|
</section>
|
|
);
|
|
}
|