rmancinas/jorgecuadros-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(policy-ocr): read A.N.A. Seguros' two policy faces
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m39s
Details
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m21s
Details

Browse Source 
A.N.A. is the Rosarito office's tourist auto book and the second carrier
the policy OCR pipeline reads. It ships two unrelated faces, and the split
is different from GMX's: GMX ships two documents about one policy, A.N.A.
ships two products.

  AUTOMOBILE (SPECIAL POLICY FOR TOURISTS)  insures a car; vehicle table,
                                            9 numbered sections, one
                                            LIMIT OF LIABILITY column
  DRIVER'S POLICY (the office: "licencia")  insures up to 5 named drivers;
                                            no vehicle at all, 6 unnumbered
                                            sections in a different order,
                                            SUM INSURED + PREMIUM columns

The four automobile products the office sells (amplia / responsabilidad
civil, annual / by-the-day) are the same layout with different numbers, so
they get one parser rather than four.

These are born-digital portal PDFs, so pdftotext -layout returns exact
columns and the driver's-policy parser uses that: its two value columns
print the same shape (100,000.00 usd. / 18.70 usd.) with no per-row label,
so horizontal position is the only thing separating them. The split comes
from the header's own offsets, not a constant, because they shift between
products; when it can't be read every amount is reported as a sum insured
and the reviewer is told, rather than half the premiums being filed as
coverage limits.

Three things the layout will punish a naive read for:

- Each PDF prints its face two or three times (ORIGINAL, AGENT COPY, then
  a receipt and three travel cards) and the pipeline concatenates every
  page before parsing. The coverage walk is bounded to the first copy and
  the driver list to the first POLICY HOLDER block. Unbounded, the licencia
  returns the same person three times, which reads as a three-driver policy
  rather than as a bug.
- The money row is read positionally off its header. An unused DISCOUNT
  prints as a bare "-", so "find the six amounts" shifts every value one
  column left on a discounted policy.
- Two five-digit numbers sit in the header band and only one is the agent
  clave; the agent's street address is "BENITO JUAREZ 25 No.50 INT 38",
  three lines above the No. cell holding the policy number.

Sections 6-8 print a PREMIUM where the others print a limit, so
ParsedCoverage gains an optional `premium` (GMX never fills it) and the
review table a column: $40 is what legal aid cost, not a $40 liability
limit. Exclusions follow the GMX rule and go in the risk label with a null
amount -- which matters more here, since a responsabilidad-civil policy
prints 0.00 for material damage and the two are identical on the page.

Also in this change:

- coveragePeriodDays is parsed and written. A.N.A. sells 3- and 4-day
  policies; Policy.coveragePeriodDays defaults to 365, so a weekend policy
  left at the default sits in the renewals window a year out. Derived from
  the dates, cross-checked against the printed DAYS cell, disagreement
  noted not resolved.
- Vehicles and named drivers are parsed, shown read-only in review, and
  written as Vehicle / InsuredDriver rows on confirm, skipping any already
  on the policy (VIN then plate; licence then name). The case that forces
  the skip is confirming a renewal onto an existing policy. Nothing is ever
  updated or deleted -- a changed plate lands as a second row for a human.
- Batch.provider is set from what the parsers actually claimed instead of
  being hardcoded "GMX", so a mixed upload is labelled as mixed and the
  header can never contradict its own documents. PolicyDocument.documentType
  follows the same rule (was hardcoded GMX_POLICY).
- matchNote becomes TEXT. It was VARCHAR(191) and the note trail was sliced
  to 190 chars, which cut the tail notes -- the "could not read X" ones.
- The policy detail page renders an array coveragesJson as a table. Both
  shapes have always been possible there, but the object renderer was the
  only one, so an OCR-confirmed policy showed a row per array index labelled
  "0", "1", "2" with [object Object] as the value. ANA makes that routine.

GMX is untouched behaviourally; its two parsers now spread a shared empty
base instead of listing every null field. 29 new parser cases against
verbatim pdftotext output of three real ANA PDFs, 53 in the suite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
rmancinas
2026-08-15 01:09:16 -07:00
co-authored by Claude Opus 5
parent 14c4d44acb
commit d645ba51d3
15 changed files with 1990 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/api/src/policy-ocr/policy-ocr.service.ts
+155 -10
View File
@@ -10,7 +10,7 @@ import { PrismaService } from "../prisma/prisma.service";
import { StorageService } from "../storage/storage.service";
import type { UploadedFileLike } from "../storage/upload-file";
import { OCR_PROVIDER, type OcrPage, type OcrProvider } from "../statements/ocr/ocr.provider";
import { parsePolicy } from "./parsers/policy-parser";
import { parsePolicy, type ParsedDriver, type ParsedVehicle } from "./parsers/policy-parser";
import { PolicyMatcherService } from "./policy-matcher.service";
import type {
ConfirmPolicyBatchDto,
@@ -69,8 +69,11 @@ export class PolicyOcrService {
);
}
// The provider is not asked of the uploader and not assumed: `process`
// sets it from what the parsers actually claimed, so the batch label can
// never contradict its own documents. Until then it says so.
const batch = await this.prisma.policyOcrBatch.create({
data: { provider: "GMX", uploadedById, label, fileCount: files.length },
data: { provider: "por detectar", uploadedById, label, fileCount: files.length },
});
const copies = files.map((f) => ({ buffer: f.buffer, name: f.originalname }));
@@ -112,6 +115,7 @@ export class PolicyOcrService {
let fileOrdinal = 0;
let globalPageOrdinal = 0;
const providersSeen = new Set<string>();
for (const file of files) {
fileOrdinal += 1;
const sourceKey = `policy-ocr/${batchId}/source-${fileOrdinal}.pdf`;
@@ -155,6 +159,7 @@ export class PolicyOcrService {
if (parsed.provider === "") {
throw new Error("no se reconoció el proveedor");
}
providersSeen.add(parsed.provider);
const match = await this.matcher.match(parsed);
const notes = [...parsed.notes, match.note].filter(Boolean);
// Confident when exactly one Policy carries the printed number —
@@ -193,12 +198,19 @@ export class PolicyOcrService {
? (parsed.coverages as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
extractedPremiumPayment: parsed.premiumPayment,
extractedCoveragePeriodDays: parsed.coveragePeriodDays,
extractedVehiclesJson: parsed.vehicles.length
? (parsed.vehicles as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
extractedDriversJson: parsed.drivers.length
? (parsed.drivers as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
matchedPolicyId: match.policyId,
matchedCustomerId: match.customerId,
matchCandidates: match.candidates.length
? (match.candidates as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
matchNote: notes.join("; ").slice(0, 190),
matchNote: notes.join("; "),
},
});
} catch (err) {
@@ -211,7 +223,7 @@ export class PolicyOcrService {
pageNumber: fileOrdinal,
storageKey: sourceKey,
status: "OCR_FAILED",
matchNote: (err as Error).message.slice(0, 190),
matchNote: (err as Error).message,
},
});
}
@@ -219,7 +231,13 @@ export class PolicyOcrService {
await this.prisma.policyOcrBatch.update({
where: { id: batchId },
data: { status: "READY_FOR_REVIEW" },
data: {
status: "READY_FOR_REVIEW",
// Whatever the parsers claimed. A mixed upload is labelled as mixed
// rather than as whichever provider happened to come first — the
// review header is the only place staff see what they dropped in.
provider: [...providersSeen].sort().join(" + ") || "desconocido",
},
});
}
@@ -349,6 +367,7 @@ export class PolicyOcrService {
? (dto.coveragesJson as Prisma.InputJsonValue)
: undefined,
extractedPremiumPayment: dto.premiumPayment ?? undefined,
extractedCoveragePeriodDays: dto.coveragePeriodDays ?? undefined,
matchedPolicyId,
matchedCustomerId,
status: dto.forceConfirm ? "CONFIRMED" : "MATCHED",
@@ -474,14 +493,18 @@ export class PolicyOcrService {
policyId = created.id;
}
// 2. Attach the source PDF as a PolicyDocument. `doc.storageKey`
// 2. Vehicles and named drivers, for the providers whose face carries
// them (ANA's automobile and driver's policies; never GMX Hogar).
await this.applyVehiclesAndDrivers(doc, policyId);
// 3. Attach the source PDF as a PolicyDocument. `doc.storageKey`
// already points at the exact upload (`policy-ocr/{batchId}/source-N.pdf`)
// so the attach is just a stream copy into the policy's namespace —
// the previous per-page "which file did this page come from" walk is
// gone because one PDF = one doc now.
await this.attachSourcePdf(doc.storageKey, policyId);
await this.attachSourcePdf(doc.storageKey, policyId, doc.provider);
// 3. Optionally post the premium to the ledger. Only when staff
// 4. Optionally post the premium to the ledger. Only when staff
// explicitly asked (`postPremium` true) and netPremium parses — without
// that gate a missing premium would silently book $0.
let postedTransactionId: string | null = null;
@@ -539,13 +562,104 @@ export class PolicyOcrService {
};
}
/**
* Write the parsed `Vehicle` and `InsuredDriver` rows onto the policy.
*
* Both inserts are skipped when an equivalent row is already on the policy.
* The reason is `confirmBatch` applying to an EXISTING policy: the office
* uploads a renewal for a car already on file, and a blind insert would
* leave the customer with the same VIN listed twice with no way to tell
* which row the renewal belongs to. Matching is on the identifier the
* document actually prints — the VIN for a vehicle (falling back to the
* plate, since ANA's TRAILER/TOWING slots have no VIN), the licence number
* for a driver (falling back to the name).
*
* Nothing is ever updated or deleted here. A vehicle whose plate changed
* lands as a second row for a human to reconcile, which is the safe half
* of the mistake: an over-write would destroy the only record of what was
* insured last term.
*/
private async applyVehiclesAndDrivers(
doc: { extractedVehiclesJson: Prisma.JsonValue | null; extractedDriversJson: Prisma.JsonValue | null },
policyId: string,
): Promise<void> {
const vehicles = asArray<ParsedVehicle>(doc.extractedVehiclesJson);
const drivers = asArray<ParsedDriver>(doc.extractedDriversJson);
if (vehicles.length === 0 && drivers.length === 0) return;
const policy = await this.prisma.policy.findUnique({
where: { id: policyId },
select: { customerId: true },
});
if (!policy) return;
if (vehicles.length) {
const existing = await this.prisma.vehicle.findMany({
where: { policyId },
select: { vinNumber: true, licensePlate: true },
});
const seen = new Set(
existing.flatMap((v) =>
[v.vinNumber, v.licensePlate].filter((k): k is string => !!k).map(norm),
),
);
for (const v of vehicles) {
const key = norm(v.vinNumber ?? v.licensePlate ?? "");
if (!key || seen.has(key)) continue;
seen.add(key);
await this.prisma.vehicle.create({
data: {
policyId,
customerId: policy.customerId,
make: v.make,
// ANA prints one BODY cell, not separate model/body columns, so
// it lands on `bodyType`; `model` stays null rather than being
// guessed out of the same string.
bodyType: v.bodyType,
modelYear: v.modelYear,
vinNumber: v.vinNumber,
licensePlate: v.licensePlate,
// "VEHICLE" / "TRAILER" / "TOWING" — the printed slot, which is
// the difference between the insured car and the trailer behind
// it and has no column of its own.
notes: v.item && v.item !== "VEHICLE" ? v.item : null,
},
});
}
}
if (drivers.length) {
const existing = await this.prisma.insuredDriver.findMany({
where: { policyId },
select: { licenseNumber: true, fullName: true },
});
const seen = new Set(
existing.flatMap((d) =>
[d.licenseNumber, d.fullName].filter((k): k is string => !!k).map(norm),
),
);
for (const d of drivers) {
const key = norm(d.licenseNumber ?? d.fullName ?? "");
if (!key || seen.has(key)) continue;
seen.add(key);
await this.prisma.insuredDriver.create({
data: { policyId, fullName: d.fullName, licenseNumber: d.licenseNumber },
});
}
}
}
/**
* Stream the source PDF (`sourceKey`, set by `process` on the doc row)
* into the policy's storage namespace and create a `PolicyDocument`
* pointer. Trivial now that the doc row holds the exact source key —
* the old per-page "which file did this page come from" walk is gone.
*/
private async attachSourcePdf(sourceKey: string, policyId: string): Promise<void> {
private async attachSourcePdf(
sourceKey: string,
policyId: string,
provider: string | null,
): Promise<void> {
const got = await this.storage.getStream(sourceKey);
const chunks: Buffer[] = [];
for await (const c of got.stream) chunks.push(c as Buffer);
@@ -556,7 +670,10 @@ export class PolicyOcrService {
await this.prisma.policyDocument.create({
data: {
policyId,
documentType: "GMX_POLICY",
// Named after whichever parser claimed the page. Was hardcoded
// `GMX_POLICY`, which mislabelled every ANA upload as a GMX
// document in the policy's file list.
documentType: `${provider ?? "OCR"}_POLICY`,
storageKey: newKey,
},
});
@@ -611,6 +728,7 @@ function buildPolicyUpdateFromDoc(
extractedTotal: Prisma.Decimal | null;
extractedCoveragesJson: Prisma.JsonValue | null;
extractedPremiumPayment: string | null;
extractedCoveragePeriodDays: number | null;
},
): Prisma.PolicyUpdateInput {
const numOrUndef = (a: number | undefined, b: Prisma.Decimal | null): Prisma.Decimal | undefined => {
@@ -635,6 +753,12 @@ function buildPolicyUpdateFromDoc(
policyFrom: dateOrUndef(item.policyFrom, doc.extractedPolicyFrom),
policyTo: dateOrUndef(item.policyTo, doc.extractedPolicyTo),
policyDate: dateOrUndef(item.policyDate, doc.extractedPolicyDate),
// Left undefined when the document didn't print a term, so the schema
// default (365) stands for GMX. ANA's by-the-day policies DO print one,
// and the default would otherwise turn a 4-day tourist policy into an
// annual one on the renewals screen.
coveragePeriodDays:
item.coveragePeriodDays ?? doc.extractedCoveragePeriodDays ?? undefined,
currency: strOrUndef(item.currency, doc.extractedCurrency) as Currency | undefined,
netPremium: numOrUndef(item.netPremium, doc.extractedNetPremium),
policyFee: numOrUndef(item.policyFee, doc.extractedPolicyFee),
@@ -683,6 +807,7 @@ function buildPolicyCreateFromDoc(
extractedTotal: Prisma.Decimal | null;
extractedCoveragesJson: Prisma.JsonValue | null;
extractedPremiumPayment: string | null;
extractedCoveragePeriodDays: number | null;
},
customerId: string,
): Prisma.PolicyUncheckedCreateInput {
@@ -716,6 +841,12 @@ function buildPolicyCreateFromDoc(
policyFrom: dateOrUndef(item.policyFrom, doc.extractedPolicyFrom),
policyTo: dateOrUndef(item.policyTo, doc.extractedPolicyTo),
policyDate: dateOrUndef(item.policyDate, doc.extractedPolicyDate),
// Left undefined when the document didn't print a term, so the schema
// default (365) stands for GMX. ANA's by-the-day policies DO print one,
// and the default would otherwise turn a 4-day tourist policy into an
// annual one on the renewals screen.
coveragePeriodDays:
item.coveragePeriodDays ?? doc.extractedCoveragePeriodDays ?? undefined,
currency: strOrUndef(item.currency, doc.extractedCurrency) as Currency | undefined,
netPremium: numOrUndef(item.netPremium, doc.extractedNetPremium),
policyFee: numOrUndef(item.policyFee, doc.extractedPolicyFee),
@@ -764,4 +895,18 @@ function strOrUndefDb(a: string | undefined, b: string | null): string | undefin
if (a != null && a !== "") return a;
if (b != null && b !== "") return b;
return undefined;
}
/** A JSON column the parser wrote as an array, read back as one. Anything
* else (null, DbNull, a legacy object shape) is an empty list rather than a
* crash — these columns are only ever populated by the parser, so a
* surprise shape means old data, not a caller to reject. */
function asArray<T>(value: Prisma.JsonValue | null): T[] {
return Array.isArray(value) ? (value as unknown as T[]) : [];
}
/** Compare identifiers the way a person would: case- and space-insensitive.
* VINs and plates are printed inconsistently ("8BPX206" vs "8BPX 206"). */
function norm(s: string): string {
return s.replace(/\s+/g, "").toUpperCase();
}