Adds the ZONA FEDERAL TIJUANA parser to the statement intake, measured against 8 pages of real "Zona Federal Marítimo Terrestre" receipts — the federal maritime-zone occupancy fee the municipality bills on beachfront lots. Provider read on 8/8, amount on 8/8 (each verified against the paper), concession clave on 6/8, period on 8/8, deadline on 2/8. Four things the corpus forced: - Tijuana bills predial and zona federal from the same treasury: same header, same Paseo del Centenario address, same ATB-541201 RFC. Every predial discriminator matches a zona federal page too, so whichever rule is asked first wins it. The only words exclusive to this layout are "Marítimo Terrestre", so its brand rule is asked ahead of all three predial ones — and its structural rule, anchored on the stub's "Derechos de ocupación", ahead of theirs. - FEDERAL_ZONE.accountNumber is an amount, not a reference. It holds DATMEX.zfed, whose 77 values include 246.06, 2369.09, 22653.94 and a negative -1679, while the concession claves these receipts are keyed by appear nowhere in the database. Matching on that column could never hit — and because every row already has a value, the `[field]: null` guards on learnAccountRefs and on the review blank-service fill would never fire either, so every page would return to the queue every bimester forever. The clave moves to meterNumber, joining gas and Tijuana predial, and the first confirm teaches the match. - The payable figure is not the printed subtotal. The municipality rounds to whole pesos and prints the difference on its own "Ajuste Ley Hacienda Mpal" line (-$0.05 against a 591.05 subtotal, $0.21 against 2,872.79). The "Total a pagar" box carrying the rounded figure sits on a grey fill and OCR'd on 1 of 8 pages; the SubTotal row read on 8 of 8. So the amount is the rounded subtotal, cross-checked against the printed box wherever it survives — where it did, it agreed. - The clave is 2 digits, a letter and 3 digits (12-T -012), not the cadastral shape, and the letter is kept as printed: toDigits maps D to 0, which turns a real 14-D -014 into 140014. It is printed twice, which rescued a page whose heading was struck through by the office's own highlighter — the failure mode behind both missing claves. Deriving the deadline from the bimester is deliberately not attempted: it is the 17th of the month after the bimester closes on a current bill, but four of these eight are late (a $1,000 Multa) and print a recalculated date, so a derived date would be wrong on exactly the pages a human most wants to see. Re-ran the earlier corpora (25 pages: predial Tijuana/Rosarito/Ensenada, CFE, CESPT, Telnor) through detection to confirm the new rules steal nothing — all 25 still read as their original provider, including the five Tijuana predial pages that share the RFC. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
237 lines
9.4 KiB
TypeScript
237 lines
9.4 KiB
TypeScript
import { Injectable } from "@nestjs/common";
|
|
import type { ServiceKind } from "@jorgecuadros/database";
|
|
import { PrismaService } from "../prisma/prisma.service";
|
|
import type { ParsedStatement } from "./parsers/statement-parser";
|
|
|
|
export interface MatchResult {
|
|
propertyServiceId: string | null;
|
|
customerId: string | null;
|
|
/** Why it landed here — shown in the review queue verbatim. */
|
|
note: string;
|
|
/** True only for an unambiguous hit on the scoped field. */
|
|
confident: boolean;
|
|
/** Populated when more than one service claims the same number. */
|
|
candidates: { propertyServiceId: string; customerId: string; customerName: string }[];
|
|
}
|
|
|
|
/**
|
|
* Resolves a parsed statement to the customer who should be billed for it.
|
|
*
|
|
* Two rules govern everything here.
|
|
*
|
|
* **Match on one scoped field, never fuzzily across all identifiers.** Each
|
|
* service kind has exactly one column its statements print, and only that
|
|
* column is consulted. A blanket search over accountNumber/meterNumber/route
|
|
* would let a water account number collide with an unrelated phone number, and
|
|
* the resulting mis-post would look perfectly ordinary in the ledger.
|
|
*
|
|
* **Never match on the customer name.** The name on a utility bill is the
|
|
* account's registrant, which drifts from the current owner and is often years
|
|
* stale — one sample CESPT receipt is printed to "ARNAIZ ROSAS ELSA AURORA"
|
|
* for an account this office holds under "CATT, RANDY", who is not the same
|
|
* person. Names are displayed for the reviewer to sanity-check, and are never
|
|
* an input to matching.
|
|
*/
|
|
/**
|
|
* Which `PropertyService` column a given kind's statements actually print.
|
|
*
|
|
* Exported because the same answer governs three places that must agree: the
|
|
* lookup here, the blank-service fill on review, and the write-back on confirm.
|
|
* When they disagree, a reference gets learned into a column nothing searches,
|
|
* and the same page returns to the review queue every month forever.
|
|
*
|
|
* `meterNumber` is doing double duty for the three kinds whose printed
|
|
* reference DATMEX never held in `accountNumber`:
|
|
* - GAS, where the number lived in free-text notes,
|
|
* - PROPERTY_TAX, where `accountNumber` holds DATMEX.predial — a 3-4 digit
|
|
* office file number that is neither unique nor printed on any statement.
|
|
* The Tijuana municipal receipt prints an 8-digit account and no clave
|
|
* catastral at all, so it needs a column of its own; overwriting the legacy
|
|
* predial numbers to make room would destroy the only link back to the
|
|
* original records, and
|
|
* - FEDERAL_ZONE, where `accountNumber` holds DATMEX.zfed, which is not a
|
|
* reference of any kind but a peso amount: 3 of its 77 values carry cents
|
|
* (`246.06`, `2369.09`, `22653.94`) and one is negative. Searching it for
|
|
* the concession clave the receipt prints would never hit, and — worse —
|
|
* because every row already has a value, the `[field]: null` guards in
|
|
* `learnAccountRefs` and the blank-service fill would never fire either, so
|
|
* the same page would return to the review queue every bimester forever.
|
|
*/
|
|
export function scopedRefField(
|
|
kind: ServiceKind,
|
|
): "accountNumber" | "meterNumber" | null {
|
|
switch (kind) {
|
|
case "ELECTRIC": // CFE "NO. DE SERVICIO" -> DATMEX.rpu
|
|
case "WATER": // CESPT "Cuenta" / "No. DE CUENTA" -> DATMEX.agua
|
|
case "TELEPHONE": // Telnor "Teléfono" (LADA stripped) -> DATMEX.telefono
|
|
case "CABLE":
|
|
return "accountNumber";
|
|
case "GAS": // bajagas "Cuenta" -> recovered from notes into meterNumber
|
|
case "PROPERTY_TAX": // Tijuana's 8-digit municipal account
|
|
case "FEDERAL_ZONE": // ZOFEMAT concession clave, e.g. `12T012`
|
|
return "meterNumber";
|
|
default:
|
|
return null;
|
|
}
|
|
}
|
|
|
|
@Injectable()
|
|
export class StatementMatcherService {
|
|
constructor(private readonly prisma: PrismaService) {}
|
|
|
|
async match(parsed: ParsedStatement, expectedKind: ServiceKind): Promise<MatchResult> {
|
|
const kind = parsed.serviceKind ?? expectedKind;
|
|
|
|
// The uploader labels a batch with one service kind. If the parser reads a
|
|
// page as a different provider, that is a mis-sorted page, not a match —
|
|
// posting it would book a phone bill as a water charge.
|
|
if (parsed.serviceKind && parsed.serviceKind !== expectedKind) {
|
|
return this.unmatched(
|
|
`la página parece de ${parsed.provider} (${parsed.serviceKind}) pero el lote es de ${expectedKind}`,
|
|
);
|
|
}
|
|
|
|
const field = scopedRefField(kind);
|
|
|
|
if (field && parsed.accountRef) {
|
|
const hit = await this.byServiceField(kind, field, parsed.accountRef);
|
|
if (hit) return hit;
|
|
}
|
|
|
|
// The clave catastral is printed on CESPT bills as well as predial ones, so
|
|
// it rescues a page whose account number did not OCR — which happened on
|
|
// real samples, where the clave read cleanly and the account number did
|
|
// not. On the Rosarito and Ensenada predial layouts it is not a rescue at
|
|
// all but the only identifier the receipt carries, so a unique hit there is
|
|
// as good as any account-number match and is treated as one.
|
|
if (parsed.cadastralKey) {
|
|
const primary = kind === "PROPERTY_TAX" && !parsed.accountRef;
|
|
const hit = await this.byCadastralKey(kind, parsed.cadastralKey, primary);
|
|
if (hit) return hit;
|
|
}
|
|
|
|
if (!field && !parsed.cadastralKey) {
|
|
return this.unmatched(`no hay campo de búsqueda definido para ${kind}`);
|
|
}
|
|
if (!parsed.accountRef && !parsed.cadastralKey) {
|
|
return this.unmatched(
|
|
kind === "PROPERTY_TAX"
|
|
? "no se leyó ni la clave catastral ni la cuenta municipal"
|
|
: "no se pudo leer la referencia de la cuenta",
|
|
);
|
|
}
|
|
return this.unmatched(
|
|
parsed.accountRef
|
|
? `no se encontró ningún servicio de ${kind} con la referencia ${parsed.accountRef}`
|
|
: `no se encontró ninguna propiedad con la clave catastral ${parsed.cadastralKey}`,
|
|
);
|
|
}
|
|
|
|
private async byServiceField(
|
|
kind: ServiceKind,
|
|
field: "accountNumber" | "meterNumber",
|
|
ref: string,
|
|
): Promise<MatchResult | null> {
|
|
const rows = await this.prisma.propertyService.findMany({
|
|
where: { kind, [field]: ref },
|
|
select: {
|
|
id: true,
|
|
property: {
|
|
select: { customerId: true, customer: { select: { name: true } } },
|
|
},
|
|
},
|
|
});
|
|
if (rows.length === 0) return null;
|
|
|
|
const candidates = rows.map((r) => ({
|
|
propertyServiceId: r.id,
|
|
customerId: r.property.customerId,
|
|
customerName: r.property.customer.name,
|
|
}));
|
|
|
|
// Duplicate account numbers do occur in the legacy data (the office's own
|
|
// DUPLICADOS report existed for a reason), so every candidate is surfaced
|
|
// for the reviewer to choose rather than one being picked arbitrarily.
|
|
if (rows.length > 1) {
|
|
return {
|
|
propertyServiceId: null,
|
|
customerId: null,
|
|
note: `${rows.length} servicios comparten la referencia ${ref}`,
|
|
confident: false,
|
|
candidates,
|
|
};
|
|
}
|
|
|
|
return {
|
|
propertyServiceId: candidates[0].propertyServiceId,
|
|
customerId: candidates[0].customerId,
|
|
note: `coincidencia exacta por ${field === "accountNumber" ? "número de cuenta" : "medidor"} ${ref}`,
|
|
confident: true,
|
|
candidates,
|
|
};
|
|
}
|
|
|
|
private async byCadastralKey(
|
|
kind: ServiceKind,
|
|
key: string,
|
|
/** True when the clave is the identifier the statement was issued against. */
|
|
primary: boolean,
|
|
): Promise<MatchResult | null> {
|
|
const props = await this.prisma.property.findMany({
|
|
where: { cadastralKey: key },
|
|
select: {
|
|
customerId: true,
|
|
customer: { select: { name: true } },
|
|
services: { where: { kind }, select: { id: true } },
|
|
},
|
|
});
|
|
if (props.length === 0) return null;
|
|
|
|
const candidates = props.flatMap((p) =>
|
|
(p.services.length ? p.services.map((s) => s.id) : [null]).map((sid) => ({
|
|
propertyServiceId: sid as string,
|
|
customerId: p.customerId,
|
|
customerName: p.customer.name,
|
|
})),
|
|
);
|
|
|
|
if (candidates.length > 1) {
|
|
return {
|
|
propertyServiceId: null,
|
|
customerId: null,
|
|
note: `${candidates.length} propiedades comparten la clave catastral ${key}`,
|
|
confident: false,
|
|
candidates,
|
|
};
|
|
}
|
|
|
|
// When the clave is the *secondary* key — a utility bill that also happens
|
|
// to print it — the page is left for review, because the clave was not the
|
|
// number the statement was issued against and confirming is what teaches
|
|
// the matcher the account number for next month. When it is the primary key
|
|
// (Rosarito and Ensenada predial, which print nothing else), a unique hit
|
|
// is a real match and there is no second number to learn.
|
|
return {
|
|
propertyServiceId: candidates[0].propertyServiceId ?? null,
|
|
customerId: candidates[0].customerId,
|
|
note: primary
|
|
? `coincidencia exacta por clave catastral ${key}`
|
|
: `identificado por clave catastral ${key}; confirme para registrar también el número de cuenta`,
|
|
// A clave with no service row of the right kind behind it still needs a
|
|
// human: there is nothing to attach the posting to.
|
|
confident: primary && candidates[0].propertyServiceId != null,
|
|
candidates,
|
|
};
|
|
}
|
|
|
|
private unmatched(note: string): MatchResult {
|
|
return {
|
|
propertyServiceId: null,
|
|
customerId: null,
|
|
note,
|
|
confident: false,
|
|
candidates: [],
|
|
};
|
|
}
|
|
}
|