feat(policy-ocr): read A.N.A. Seguros' two policy faces
A.N.A. is the Rosarito office's tourist auto book and the second carrier
the policy OCR pipeline reads. It ships two unrelated faces, and the split
is different from GMX's: GMX ships two documents about one policy, A.N.A.
ships two products.
AUTOMOBILE (SPECIAL POLICY FOR TOURISTS) insures a car; vehicle table,
9 numbered sections, one
LIMIT OF LIABILITY column
DRIVER'S POLICY (the office: "licencia") insures up to 5 named drivers;
no vehicle at all, 6 unnumbered
sections in a different order,
SUM INSURED + PREMIUM columns
The four automobile products the office sells (amplia / responsabilidad
civil, annual / by-the-day) are the same layout with different numbers, so
they get one parser rather than four.
These are born-digital portal PDFs, so pdftotext -layout returns exact
columns and the driver's-policy parser uses that: its two value columns
print the same shape (100,000.00 usd. / 18.70 usd.) with no per-row label,
so horizontal position is the only thing separating them. The split comes
from the header's own offsets, not a constant, because they shift between
products; when it can't be read every amount is reported as a sum insured
and the reviewer is told, rather than half the premiums being filed as
coverage limits.
Three things the layout will punish a naive read for:
- Each PDF prints its face two or three times (ORIGINAL, AGENT COPY, then
a receipt and three travel cards) and the pipeline concatenates every
page before parsing. The coverage walk is bounded to the first copy and
the driver list to the first POLICY HOLDER block. Unbounded, the licencia
returns the same person three times, which reads as a three-driver policy
rather than as a bug.
- The money row is read positionally off its header. An unused DISCOUNT
prints as a bare "-", so "find the six amounts" shifts every value one
column left on a discounted policy.
- Two five-digit numbers sit in the header band and only one is the agent
clave; the agent's street address is "BENITO JUAREZ 25 No.50 INT 38",
three lines above the No. cell holding the policy number.
Sections 6-8 print a PREMIUM where the others print a limit, so
ParsedCoverage gains an optional `premium` (GMX never fills it) and the
review table a column: $40 is what legal aid cost, not a $40 liability
limit. Exclusions follow the GMX rule and go in the risk label with a null
amount -- which matters more here, since a responsabilidad-civil policy
prints 0.00 for material damage and the two are identical on the page.
Also in this change:
- coveragePeriodDays is parsed and written. A.N.A. sells 3- and 4-day
policies; Policy.coveragePeriodDays defaults to 365, so a weekend policy
left at the default sits in the renewals window a year out. Derived from
the dates, cross-checked against the printed DAYS cell, disagreement
noted not resolved.
- Vehicles and named drivers are parsed, shown read-only in review, and
written as Vehicle / InsuredDriver rows on confirm, skipping any already
on the policy (VIN then plate; licence then name). The case that forces
the skip is confirming a renewal onto an existing policy. Nothing is ever
updated or deleted -- a changed plate lands as a second row for a human.
- Batch.provider is set from what the parsers actually claimed instead of
being hardcoded "GMX", so a mixed upload is labelled as mixed and the
header can never contradict its own documents. PolicyDocument.documentType
follows the same rule (was hardcoded GMX_POLICY).
- matchNote becomes TEXT. It was VARCHAR(191) and the note trail was sliced
to 190 chars, which cut the tail notes -- the "could not read X" ones.
- The policy detail page renders an array coveragesJson as a table. Both
shapes have always been possible there, but the object renderer was the
only one, so an OCR-confirmed policy showed a row per array index labelled
"0", "1", "2" with [object Object] as the value. ANA makes that routine.
GMX is untouched behaviourally; its two parsers now spread a shared empty
base instead of listing every null field. 29 new parser cases against
verbatim pdftotext output of three real ANA PDFs, 53 in the suite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+9
-4
@@ -165,12 +165,17 @@ Each of these is a known, deliberate stopping point rather than a bug.
|
||||
- No `SKIPPED_NO_EMAIL` worklist (see 1.9).
|
||||
|
||||
**Policy OCR** — [`POLICY_OCR.md`](POLICY_OCR.md)
|
||||
- **GMX only.** The dispatcher is a `[provider, pattern]` table plus a parser
|
||||
map, so a second carrier is one function and two entries — but no other
|
||||
layout has been seen, and guessing produces a parser nobody can verify.
|
||||
- **GMX and A.N.A. only.** The dispatcher is a `[provider, pattern]` table plus
|
||||
a parser map, so a third carrier is one function and two entries — but no
|
||||
other layout has been seen, and guessing produces a parser nobody can verify.
|
||||
- **The `recibo` PDF is unread.** The GMX certificate carries no premium at
|
||||
all; reading the separate receipt and pairing it to its certificate is what
|
||||
would let `postPremium` stop being a manual tick.
|
||||
would let `postPremium` stop being a manual tick. A.N.A. prints its premium
|
||||
on the face, so this is a GMX-only gap.
|
||||
- **No `policyTypeId` from OCR.** A.N.A.'s two faces are distinguishable in the
|
||||
parser (automobile vs driver's policy) and the platform has a `PolicyType`
|
||||
discriminator, but confirm never sets one — partly because the `policy_types`
|
||||
rows are themselves incomplete (INSURANCE §live defects).
|
||||
- **No versioning.** A re-issued policy arrives as a new certificate with the
|
||||
same number and confirm updates the existing row. Nothing records that this
|
||||
is the 2027 issue of that policy.
|
||||
|
||||
+129
-12
@@ -34,7 +34,7 @@ was **reused, not copied**.
|
||||
|---|---|
|
||||
| API module | `apps/api/src/policy-ocr/` (service, controller, DTOs, matcher, parser) |
|
||||
| Shared OCR seam | `apps/api/src/ocr/ocr.module.ts` |
|
||||
| Tables | `policy_ocr_batches`, `policy_ocr_documents` (`20260801000000_policy_ocr_intake`) |
|
||||
| Tables | `policy_ocr_batches`, `policy_ocr_documents` (`20260801000000_policy_ocr_intake`, extended by `20260815120000_policy_ocr_ana`) |
|
||||
| Web | `components/PolicyCaptura.tsx` (tab shell), `PolicyOcrIntake.tsx` (upload), `PolicyOcrReview.tsx` (review queue) |
|
||||
| Abilities | `policy:ingest`, `policy:ocr-review` — both **STAFF** |
|
||||
|
||||
@@ -194,6 +194,94 @@ amount, never as `0`: a coverage insured for zero and an excluded coverage are
|
||||
the same number and very different facts, and `ParsedCoverage` has no field
|
||||
for the distinction.
|
||||
|
||||
## A.N.A. ships two unrelated faces too
|
||||
|
||||
`A.N.A. Compañía de Seguros` is the Rosarito office's tourist auto book. Same
|
||||
split as GMX, different reason: GMX ships two *documents about one policy*,
|
||||
A.N.A. ships two *products*.
|
||||
|
||||
| | **AUTOMOBILE** (`SPECIAL POLICY FOR TOURISTS`) | **DRIVER´S POLICY** (the office says *licencia*) |
|
||||
|---|---|---|
|
||||
| Insures | a specific car | up to five named drivers, whatever they drive |
|
||||
| Vehicle table | `ITEM / YEAR / MAKE / BODY / SERIAL No. / PLATES` | **none** |
|
||||
| Insured | one `INSURED` cell | numbered `POLICY HOLDER` list |
|
||||
| Value columns | one (`LIMIT OF LIABILITY`) | two (`SUM INSURED`, `PREMIUM`) |
|
||||
| Sections | 9, numbered | 6, unnumbered, **in a different order** |
|
||||
| Parser | `parseAnaAutomobile` | `parseAnaDriverPolicy` |
|
||||
|
||||
Selected by `isAnaDriverPolicy` on the title band.
|
||||
|
||||
The **four automobile products** the office sells — amplia and responsabilidad
|
||||
civil, each annual or by-the-day — are the **same layout with different
|
||||
numbers**. "Amplia" prints a vehicle value and `COVERED` on sections 1–2;
|
||||
"resp. civil" prints `0.00` and `EXCLUDED`. That is data, not a layout, so
|
||||
there is one parser rather than four.
|
||||
|
||||
Things worth knowing before touching the ANA parsers:
|
||||
|
||||
- **These are born-digital portal PDFs**, so `pdftotext -layout` returns exact
|
||||
glyphs and exact columns. The driver's policy parser uses that: `SUM
|
||||
INSURED` and `PREMIUM` print the same shape (`100,000.00 usd.` /
|
||||
`18.70 usd.`) with no per-row label, so **horizontal position is the only
|
||||
thing that separates them**. The split is computed from the header's own
|
||||
column offsets rather than hardcoded, because they shift between products.
|
||||
If a scan ever arrives without column fidelity, every amount is reported as
|
||||
a sum insured and the reviewer is told the split failed.
|
||||
- **The money row is read positionally, not by finding six amounts.** An
|
||||
unused `DISCOUNT` prints as a bare `-`, so an "amounts in order" reading
|
||||
shifts every value one column left on a discounted policy. The parser
|
||||
requires exactly six whitespace-separated cells or reports the row unread.
|
||||
- **Each PDF prints its face two or three times** (ORIGINAL, AGENT COPY, then
|
||||
a summary receipt and three travel ID cards), and the pipeline concatenates
|
||||
every page before parsing. The coverage walk is bounded to the first copy
|
||||
and the driver list to the first `POLICY HOLDER` block. Unbounded, the
|
||||
licencia returns the same person three times — which reads as a
|
||||
three-driver policy, not as a bug, so nothing downstream would catch it.
|
||||
- **Two five-digit numbers sit in the header band** and only one is the agent
|
||||
clave: the other is the agent's own postal code
|
||||
(`ROSARITO, BAJA CALIFORNIA 22710`). Likewise the agent's street address
|
||||
reads `BENITO JUAREZ 25 No.50 INT 38`, three lines above the `No.` cell that
|
||||
holds the policy number — hence the two-space floor after `No.`.
|
||||
- **Sections 6–8 print a PREMIUM where the others print a limit.** $40 is what
|
||||
legal aid *cost*, not a $40 liability limit, so it lands on
|
||||
`ParsedCoverage.premium` (a field GMX never fills) and gets its own column
|
||||
on the review screen. Adding the two together would be meaningless.
|
||||
- **`coveragePeriodDays` matters here and nowhere else.** A.N.A. sells 3- and
|
||||
4-day policies. `Policy.coveragePeriodDays` defaults to 365, so a weekend
|
||||
policy left at the default sits in the renewals window a year out. The term
|
||||
is derived from the two dates and cross-checked against the printed `DAYS`
|
||||
cell; a disagreement is noted rather than resolved.
|
||||
|
||||
Exclusions follow the GMX rule — recorded in the risk label
|
||||
(`MATERIAL DAMAGE — VEHICLE: EXCLUDED`) with a null amount, never as `0`. It
|
||||
matters more here: a responsabilidad-civil policy prints `0.00` for material
|
||||
damage, so the two are visually identical on the page.
|
||||
|
||||
### Vehicles and drivers
|
||||
|
||||
A.N.A. is the first provider whose face carries either, so confirm now writes
|
||||
`Vehicle` and `InsuredDriver` rows alongside the `Policy`
|
||||
(`applyVehiclesAndDrivers`). The parsed values are stored on the document as
|
||||
`extractedVehiclesJson` / `extractedDriversJson` and shown read-only in the
|
||||
review queue, so a misread VIN is catchable before it is applied.
|
||||
|
||||
Both inserts skip a row that already exists on the policy, matched on the
|
||||
identifier the document prints — VIN then plate for a vehicle (A.N.A.'s
|
||||
TRAILER and TOWING slots have no VIN), licence number then name for a driver.
|
||||
The case that forces this is confirming a **renewal** onto an existing policy:
|
||||
a blind insert leaves the customer with the same VIN listed twice and no way
|
||||
to tell which row the renewal belongs to.
|
||||
|
||||
Nothing is ever updated or deleted there. A vehicle whose plate changed lands
|
||||
as a second row for a human to reconcile — the safe half of the mistake, since
|
||||
an overwrite would destroy the only record of what was insured last term.
|
||||
|
||||
The vehicle table is parsed **by token role, not by column offset**, because
|
||||
`BODY` is the cell that wraps: `PACIFICA` is one token and `GENESIS SEDAN` is
|
||||
two, so a fixed token count reads the VIN out of the wrong slot on the second.
|
||||
The 17-character VIN is the anchor and `BODY` is whatever sits between the
|
||||
make and it.
|
||||
|
||||
## What the parser reads, and the field it cannot
|
||||
|
||||
`ParsedPolicy` fields are all nullable on purpose: each carrier prints a
|
||||
@@ -206,6 +294,12 @@ insured, broker (→ `Policy.agentName`), legal address, ZIP, `policyFrom` /
|
||||
per-coverage table (risk, insured amount, deductible, loss participation)
|
||||
preserved verbatim.
|
||||
|
||||
Read from an A.N.A. face: all of the above except additional insured and
|
||||
broker parens, **plus** the premium (A.N.A. prints it — see below), the policy
|
||||
fee, the total, the term in days, the vehicle table, and the named drivers
|
||||
with their US licence numbers. The tax and the agent clave have no column in
|
||||
the schema and ride in the notes.
|
||||
|
||||
> **The GMX certificate carries no premium.** Not "sometimes missing" — the
|
||||
> document does not have the figure. It lives on GMX's **separate `recibo`
|
||||
> PDF**. The parser leaves `netPremium` / `policyFee` / `brokerFee` / `total`
|
||||
@@ -218,6 +312,10 @@ This is also why confirm never overwrites an existing `Policy.netPremium`
|
||||
with null: the certificate not carrying a premium is not evidence that the
|
||||
premium is gone.
|
||||
|
||||
A.N.A.'s faces do print one — the `DISCOUNT / PREMIUM / POLICY FEE / TAX /
|
||||
LOCAL TAX / TOTAL` row is on the same page — so an ANA document reaches the
|
||||
review queue with `netPremium` populated and `postPremium` already ticked.
|
||||
|
||||
Deductible and loss participation are stored as **strings** (`"5%"`, `"20%"`,
|
||||
`"USD 1,000"`) — they are printed as a mix of percentages, currency amounts
|
||||
and free text, and normalising them would lose the distinction.
|
||||
@@ -229,9 +327,14 @@ Per confirmed document, in order:
|
||||
1. **The `Policy` row** — updated if a policy was matched, created under the
|
||||
picked customer if not. Only non-null `extracted*` fields are written; null
|
||||
never overwrites existing data.
|
||||
2. **A `PolicyDocument`** — the source PDF is streamed into the policy's
|
||||
storage namespace and attached, so the paperwork stays with the policy.
|
||||
3. **Optionally a `Transaction`** — `INSURANCE` domain, negative amount
|
||||
2. **`Vehicle` and `InsuredDriver` rows** — for the providers whose face
|
||||
carries them (A.N.A.; never GMX Hogar), skipping any that already exist on
|
||||
the policy. See *Vehicles and drivers* above.
|
||||
3. **A `PolicyDocument`** — the source PDF is streamed into the policy's
|
||||
storage namespace and attached, so the paperwork stays with the policy. Its
|
||||
`documentType` is named after whichever parser claimed the page
|
||||
(`ANA_POLICY`, `GMX_POLICY`).
|
||||
4. **Optionally a `Transaction`** — `INSURANCE` domain, negative amount
|
||||
(a charge), `captureSource: "OCR"`, `captureRef` = the document id.
|
||||
|
||||
The ledger write is **opt-in twice over**: staff must tick `postPremium`
|
||||
@@ -271,10 +374,11 @@ feature is disabled.
|
||||
|
||||
## Tests
|
||||
|
||||
`apps/api/src/policy-ocr/parsers/policy-parser.spec.ts` — 24 cases against
|
||||
verbatim text extracted from two real documents, indentation and blank lines
|
||||
`apps/api/src/policy-ocr/parsers/policy-parser.spec.ts` — 53 cases against
|
||||
verbatim text extracted from five real documents, indentation and blank lines
|
||||
included (the column positions are what the parser reads, so a cleaned-up
|
||||
fixture would test nothing).
|
||||
fixture would test nothing — and on A.N.A.'s driver's policy the offsets are
|
||||
literally the only thing separating two columns).
|
||||
|
||||
From `HC_Folio_000767_Traduccion.pdf` (caratula): provider detection from the
|
||||
wordmark and from the footer URL, the header fields, every coverage row off
|
||||
@@ -294,7 +398,20 @@ sublimit block whose amount sits after both a blank line and a page break,
|
||||
the excluded earthquake coverage, and the hydrometeorological deductible and
|
||||
coinsurance pulled from their own per-zone block.
|
||||
|
||||
Four of those are regression tests for ways the parser can silently attach
|
||||
From the three A.N.A. PDFs: brand detection (and that GMX's layout rules
|
||||
cannot claim an ANA page), the header band, DD MM YYYY read out of three
|
||||
separate column cells, the six money cells with `DISCOUNT` printed as a bare
|
||||
`-`, the vehicle row with a one-word and a two-word `BODY` cell, the empty
|
||||
TRAILER/TOWING slots, the agent street number and postal code that must *not*
|
||||
be read as the policy number and clave, the declared values labelled by item
|
||||
slot, the `$500.00` inside the deductible sentence that is not a sum insured,
|
||||
the per-person/per-accident split in both of its printed forms, an add-on's
|
||||
figure recorded as a premium, section 9's parenthesised limit, the by-the-day
|
||||
term, the excluded sections, the column-position split on the driver's policy,
|
||||
its different section order, and — for both faces — that a doubled or tripled
|
||||
input yields one set of coverages and one driver rather than one per copy.
|
||||
|
||||
Four of the GMX cases are regression tests for ways the parser can silently attach
|
||||
the *wrong* value rather than none — a neighbouring coverage's prose read as
|
||||
a deductible, the page-level `DEDUCIBLES:` paragraph read as one, a coverage
|
||||
named after a wrapped prose tail, and one section's per-zone deductible
|
||||
@@ -303,10 +420,10 @@ the parser against the full ten-page document.
|
||||
|
||||
## Not built
|
||||
|
||||
- **Only GMX.** The dispatcher (`detectPolicyProvider`) is a table of
|
||||
`[provider, pattern]` pairs plus a `parsers` map, so adding ANA or Qualitas
|
||||
is a parser function and two entries — but no other carrier's layout has
|
||||
been seen yet, and guessing at one produces a parser nobody can verify.
|
||||
- **GMX and A.N.A. only.** The dispatcher (`detectPolicyProvider`) is a table
|
||||
of `[provider, pattern]` pairs plus a `parsers` map, so adding Qualitas is a
|
||||
parser function and two entries — but no other carrier's layout has been
|
||||
seen yet, and guessing at one produces a parser nobody can verify.
|
||||
- **The `recibo` PDF.** Reading the premium off GMX's separate receipt
|
||||
document, and pairing it to the certificate it belongs to, is the obvious
|
||||
next piece. It is what would let `postPremium` stop being a manual tick.
|
||||
|
||||
Reference in New Issue
Block a user