feat(recibos): OCR capture for gas butano and municipal predial
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m50s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m8s

Adds four parsers to the statement intake — GAS TIJUANA plus one per
municipality, because Tijuana, Rosarito and Ensenada issue three
completely different predial documents — and a text-layer fast path for
the born-digital invoices the gas company sends.

Measured against a new corpus of 14 documents / 29 pages: provider read
on 29/29, amount on 26/29, and 21/29 auto-matched against the dev
database (22/29 identified). The eight review cases are all legitimate.

Five things the corpus forced:

- Not every statement is a scan. The gas invoices are born-digital CFDIs
  whose text layer is exact; rasterising them only loses information (one
  sample turned `MEDIDOR: VM01014426` into `ar (LTR): 014420`). The new
  `OcrProvider.textPages` reads the embedded layer via `pdftotext
  -bbox-layout` — same poppler package as `pdftoppm`, so no new
  dependency — and OCR stays the fallback for real scans. Poppler's own
  `<line>` grouping follows text flow rather than the page, so words are
  regrouped by vertical position; without that, a two-column header
  leaves every label separated from the value printed beside it.

- The clave catastral is not two letters and six digits. Position three
  is a letter in 15 of the 932 stored claves, and digitising the whole
  tail mapped a real `MMB01041` to a nonexistent `MM801041`.

- Tijuana predial prints no clave at all. Its only identifier is an
  8-digit municipal account carried in a 32-digit payment barcode, which
  the legacy database never held, so it goes in `meterNumber` alongside
  gas — `accountNumber` holds `DATMEX.predial`, which is not a
  per-property key and must not be overwritten. Those pages start cold
  and are taught by the first confirm.

- On Rosarito and Ensenada the clave is the primary key, not a fallback:
  those receipts print nothing else, so a unique hit auto-matches. On a
  utility bill that merely happens to print one it stays a review hint.

- A misread `$` is the dangerous failure. An Ensenada receipt for
  $2,203.00 OCR'd as `82,203.00`, which would post a charge 37x too large
  and look ordinary in the ledger. Predial amounts now require a literal
  `$` and a page that cannot produce one goes to review.

The scoped match field is now one exported function rather than three
copies of `kind === "GAS" ? ... : ...`, since the lookup, the
blank-service fill and the confirm write-back have to agree or a
reference gets learned into a column nothing searches.

First tests in this package: 23 specs over the parsers and the text-layer
reader, every fixture a verbatim OCR excerpt from a real receipt. Adds
the jest config they need and a build tsconfig so they stay out of dist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-01 12:52:20 -07:00
co-authored by Claude Opus 5
parent 216309190c
commit d6501f1d74
12 changed files with 940 additions and 58 deletions
@@ -32,30 +32,46 @@ export interface MatchResult {
* person. Names are displayed for the reviewer to sanity-check, and are never
* an input to matching.
*/
/**
* Which `PropertyService` column a given kind's statements actually print.
*
* Exported because the same answer governs three places that must agree: the
* lookup here, the blank-service fill on review, and the write-back on confirm.
* When they disagree, a reference gets learned into a column nothing searches,
* and the same page returns to the review queue every month forever.
*
* `meterNumber` is doing double duty for the two kinds whose printed reference
* DATMEX never held in `accountNumber`:
* - GAS, where the number lived in free-text notes, and
* - PROPERTY_TAX, where `accountNumber` holds DATMEX.predial — a 3-4 digit
* office file number that is neither unique nor printed on any statement.
* The Tijuana municipal receipt prints an 8-digit account and no clave
* catastral at all, so it needs a column of its own; overwriting the legacy
* predial numbers to make room would destroy the only link back to the
* original records.
*/
export function scopedRefField(
kind: ServiceKind,
): "accountNumber" | "meterNumber" | null {
switch (kind) {
case "ELECTRIC": // CFE "NO. DE SERVICIO" -> DATMEX.rpu
case "WATER": // CESPT "Cuenta" / "No. DE CUENTA" -> DATMEX.agua
case "TELEPHONE": // Telnor "Teléfono" (LADA stripped) -> DATMEX.telefono
case "FEDERAL_ZONE":
case "CABLE":
return "accountNumber";
case "GAS": // bajagas "Cuenta" -> recovered from notes into meterNumber
case "PROPERTY_TAX": // Tijuana's 8-digit municipal account
return "meterNumber";
default:
return null;
}
}
@Injectable()
export class StatementMatcherService {
constructor(private readonly prisma: PrismaService) {}
/** Which PropertyService column a given kind's statements actually print. */
private fieldFor(kind: ServiceKind): "accountNumber" | "meterNumber" | null {
switch (kind) {
case "ELECTRIC": // CFE "NO. DE SERVICIO" -> DATMEX.rpu
case "WATER": // CESPT "Cuenta" / "No. DE CUENTA" -> DATMEX.agua
case "TELEPHONE": // Telnor "Teléfono" (LADA stripped) -> DATMEX.telefono
case "FEDERAL_ZONE":
case "CABLE":
return "accountNumber";
case "GAS": // no account column in DATMEX; the number lived in notes
return "meterNumber";
// PROPERTY_TAX deliberately has no scoped column: what its
// accountNumber holds is DATMEX.predial, which is neither unique nor
// printed on any statement. Predial bills match on the clave catastral
// alone — see matchByCadastralKey.
default:
return null;
}
}
async match(parsed: ParsedStatement, expectedKind: ServiceKind): Promise<MatchResult> {
const kind = parsed.serviceKind ?? expectedKind;
@@ -68,33 +84,39 @@ export class StatementMatcherService {
);
}
const field = this.fieldFor(kind);
const field = scopedRefField(kind);
if (field && parsed.accountRef) {
const hit = await this.byServiceField(kind, field, parsed.accountRef);
if (hit) return hit;
}
// Secondary key. The clave catastral is printed on CESPT bills as well as
// predial ones, so it rescues a page whose account number did not OCR —
// which happened on real samples, where the clave read cleanly and the
// account number did not.
// The clave catastral is printed on CESPT bills as well as predial ones, so
// it rescues a page whose account number did not OCR — which happened on
// real samples, where the clave read cleanly and the account number did
// not. On the Rosarito and Ensenada predial layouts it is not a rescue at
// all but the only identifier the receipt carries, so a unique hit there is
// as good as any account-number match and is treated as one.
if (parsed.cadastralKey) {
const hit = await this.byCadastralKey(kind, parsed.cadastralKey);
const primary = kind === "PROPERTY_TAX" && !parsed.accountRef;
const hit = await this.byCadastralKey(kind, parsed.cadastralKey, primary);
if (hit) return hit;
}
if (!field && !parsed.cadastralKey) {
return this.unmatched(`no hay campo de búsqueda definido para ${kind}`);
}
if (!parsed.accountRef && !parsed.cadastralKey) {
return this.unmatched(
kind === "PROPERTY_TAX"
? "el predial sólo se puede identificar por clave catastral y no se leyó ninguna"
: `no hay campo de búsqueda definido para ${kind}`,
? "no se leyó ni la clave catastral ni la cuenta municipal"
: "no se pudo leer la referencia de la cuenta",
);
}
return this.unmatched(
parsed.accountRef
? `no se encontró ningún servicio de ${kind} con la referencia ${parsed.accountRef}`
: "no se pudo leer la referencia de la cuenta",
: `no se encontró ninguna propiedad con la clave catastral ${parsed.cadastralKey}`,
);
}
@@ -145,6 +167,8 @@ export class StatementMatcherService {
private async byCadastralKey(
kind: ServiceKind,
key: string,
/** True when the clave is the identifier the statement was issued against. */
primary: boolean,
): Promise<MatchResult | null> {
const props = await this.prisma.property.findMany({
where: { cadastralKey: key },
@@ -174,15 +198,21 @@ export class StatementMatcherService {
};
}
// The clave identifies the property with certainty, but it is a *secondary*
// key: it was not the number the statement was issued against. Left for
// review so the confirm also teaches the matcher the account number, rather
// than the same page needing the fallback again next month.
// When the clave is the *secondary* key — a utility bill that also happens
// to print it — the page is left for review, because the clave was not the
// number the statement was issued against and confirming is what teaches
// the matcher the account number for next month. When it is the primary key
// (Rosarito and Ensenada predial, which print nothing else), a unique hit
// is a real match and there is no second number to learn.
return {
propertyServiceId: candidates[0].propertyServiceId ?? null,
customerId: candidates[0].customerId,
note: `identificado por clave catastral ${key}; confirme para registrar también el número de cuenta`,
confident: false,
note: primary
? `coincidencia exacta por clave catastral ${key}`
: `identificado por clave catastral ${key}; confirme para registrar también el número de cuenta`,
// A clave with no service row of the right kind behind it still needs a
// human: there is nothing to attach the posting to.
confident: primary && candidates[0].propertyServiceId != null,
candidates,
};
}