The parser has been reading A.N.A.'s `TAX` cell since the ANA layout landed,
but `ParsedPolicy` had nowhere to put it, so the figure only ever reached a
review note and every OCR-confirmed policy was written with `tax` null — even
though the paper states it.
`TAX` now flows parser -> `extractedTax` -> `Policy.tax`, alongside the premium
fields beside it. GMX stays null: its certificate carries no premium at all, so
there is no tax on it either.
The other two cells stay out, for reasons worth keeping:
- `LOCAL TAX` is a separate levy with no destination column, and summing it
into `tax` would produce an IVA that no longer divides back to a rate —
which is the whole reason to store the figure. It reads 0.00 on every
policy seen so far; a non-zero one now raises a note saying the total will
not reconcile, instead of quietly inflating the IVA.
- `DISCOUNT` has no column and prints as a bare "-" when unused, which is
what makes the row positional rather than "find six amounts".
`taxRate` is left null by confirm. A.N.A. prints the amount, not the rate, and
back-dividing it would mint a rate the document never stated; the capture form
resolves one from the line of business instead.
The review screen gains derecho de póliza next to the new IVA field. It was
already parsed and already written on confirm, but never shown — and an IVA
with no fee beside it leaves the reviewer unable to see why premium + fee + tax
equals the printed total.
The spec's assertion moved off the note and onto the field, plus a check that
the row reconciles: 298.61 + 30.00 at 8% is 26.29, totalling 354.90. That
agreement is what proves the positional mapping landed on the right cells
rather than merely on six numbers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
576 lines
31 KiB
Markdown
576 lines
31 KiB
Markdown
# Insurance Policy OCR Capture
|
||
|
||
Reads an insurance policy PDF the office downloads from a carrier portal,
|
||
proposes the `Policy` row it should become, and lets staff confirm. Built
|
||
2026-08-01 (`5e9cb12`), live under `/polizas/captura`.
|
||
|
||
## Why this exists — it was not planned
|
||
|
||
This feature is **not in any spec**. It came out of building the utility
|
||
statement OCR intake in [`RECEIPT_CAPTURE_SPEC.md`](RECEIPT_CAPTURE_SPEC.md)
|
||
§2: once there was a working render → OCR → parse → match → review pipeline
|
||
for CFE/CESPT/Telnor receipts, it was obvious the same shape applies to the
|
||
*other* stack of paper this office keys in by hand every week — the carrier
|
||
policy PDFs behind every `Policy` row.
|
||
|
||
The two are the same job with a different document on the scanner. Keeping
|
||
that recognition cheap is the whole point of how it was built: the pipeline
|
||
was **reused, not copied**.
|
||
|
||
- `OcrModule` (`apps/api/src/ocr/ocr.module.ts`) was extracted out of
|
||
`StatementsModule` in this same commit, purely so `PolicyOcrModule` can
|
||
inject `OCR_PROVIDER` without dragging in the statement pipeline.
|
||
`StatementsModule` now imports it and binds nothing itself. That extraction
|
||
was **blocking**: without it the policy module could not resolve the
|
||
provider at all.
|
||
- The engine stays Tesseract behind the same swappable seam, so a managed
|
||
extraction API remains a one-line change in one file for both features.
|
||
- The intake screen is a *mode of* the existing policy-creation screen, the
|
||
same way OCR receipt capture is a mode of Captura — not a new menu entry.
|
||
|
||
## What ships
|
||
|
||
| Piece | Path |
|
||
|---|---|
|
||
| API module | `apps/api/src/policy-ocr/` (service, controller, DTOs, matcher, parser) |
|
||
| Shared OCR seam | `apps/api/src/ocr/ocr.module.ts` |
|
||
| Tables | `policy_ocr_batches`, `policy_ocr_documents` (`20260801000000_policy_ocr_intake`, extended by `20260815120000_policy_ocr_ana` and `20260815160000_policy_type_repair`) |
|
||
| Web | `components/PolicyCaptura.tsx` (tab shell), `PolicyOcrIntake.tsx` (upload), `PolicyOcrReview.tsx` (review queue) |
|
||
| Abilities | `policy:ingest`, `policy:ocr-review` — both **STAFF** |
|
||
|
||
Abilities are STAFF for the same reason statement OCR is: nothing reaches the
|
||
books unconfirmed, and the review step is what makes machine capture safe at
|
||
that tier.
|
||
|
||
## The screen
|
||
|
||
`PolicyCaptura` is one screen with two ways in, mirroring `Captura.tsx`:
|
||
|
||
- `/polizas/nuevo` → **manual** tab (`PolicyForm`, every field by hand)
|
||
- `/polizas/captura` → **automática** tab (`PolicyOcrIntake`, drop a PDF)
|
||
- `/polizas/captura/[id]` → the batch review queue
|
||
|
||
Both modes end at the same place — a `Policy` row on a customer's file — so
|
||
they are modes of one screen rather than two menu entries. Either URL renders
|
||
the same component, so the tab toggle works from either entry point and old
|
||
bookmarks land on the right tab.
|
||
|
||
## Pipeline
|
||
|
||
```
|
||
upload PDF → store source → render pages → text layer? → parse → match → review → confirm
|
||
```
|
||
|
||
1. **Store the source.** `policy-ocr/{batchId}/source-N.pdf`, before anything
|
||
else touches it.
|
||
2. **Render + read.** Every page is rendered to
|
||
`policy-ocr/{batchId}/page-M.png`. Text-layer wins when the PDF has one
|
||
(cheap, exact); the rendered image is OCR'd only when it does not — the
|
||
same precedence rule as the statement pipeline. Carrier-portal PDFs are
|
||
usually born-digital, so most of the time no OCR runs at all.
|
||
3. **Parse.** Provider detected by brand signal first
|
||
(`GMX`, `Grupo Mexicano de Seguros`, `gmx.com.mx`,
|
||
`JUNTOS EL RIESGO ES MENOR`), layout patterns only as fallback — the same
|
||
ordering rule the statement parser needed.
|
||
4. **Match.** Against `Policy.policyNumber`.
|
||
5. **Review + confirm.** Nothing is written to `Policy` until a human
|
||
confirms.
|
||
|
||
### One PDF = one policy
|
||
|
||
This is the sharpest difference from statement OCR, and it inverts that
|
||
feature's core assumption.
|
||
|
||
Utility statements arrive **bundled, one customer per page** — so there, one
|
||
page is one document and the parser runs per page. A policy PDF is the
|
||
opposite: the GMX certificate is a 2-page document where page 1 carries the
|
||
contract header and page 2 carries the per-coverage table (and the PVL
|
||
especificación runs to ten), and **every page describes the same policy**. So
|
||
the pipeline concatenates every page's text
|
||
(`\n\n` between pages, which also keeps `ocrRawText` readable for debugging)
|
||
and runs the parser and the matcher exactly **once per file**.
|
||
|
||
Consequences worth knowing before touching this code:
|
||
|
||
- `PolicyOcrDocument.pageNumber` is repurposed as the **file ordinal within
|
||
the batch** (1, 2, 3…), not a page index. The
|
||
`(batchId, pageNumber)` unique constraint still holds, and one batch still
|
||
carries many policies — one per uploaded file.
|
||
- Parser regexes are anchored across the whole concatenated text (`^From$`,
|
||
`^Currency\s+…`), which is why the page-boundary blank line matters.
|
||
- `ocrConfidence` on the row is the **mean** across the file's pages.
|
||
- A file that fails to parse produces exactly one `OCR_FAILED` row — the right
|
||
granularity, and the page PNGs stay on disk for a re-run after a parser fix.
|
||
|
||
### `storageKey` is the source PDF, not a page image
|
||
|
||
`PolicyOcrDocument.storageKey` points at `source-N.pdf`. The review screen
|
||
embeds that file directly, so the reviewer looks at the **exact artifact the
|
||
office received** and gets the browser's native PDF scrolling, zoom and text
|
||
selection for free. Rendered PNGs are still written for future re-OCR or an
|
||
image-based audit, but nothing points at them as the document's identity.
|
||
|
||
(The statement side does the opposite — there `storageKey` is the page image,
|
||
because a page *is* the document.)
|
||
|
||
## Matching: policy number only, never the insured name
|
||
|
||
`PolicyMatcherService` matches on `Policy.policyNumber` and nothing else.
|
||
|
||
The certificate's "Insured" line is the account's registrant, which drifts
|
||
from the customer the office actually holds the file under — the same finding
|
||
the statement matcher is built around (a CESPT receipt reading
|
||
`ARNAIZ ROSAS ELSA AURORA` for a customer this office holds as `CATT, RANDY`).
|
||
Names are shown to the reviewer as a sanity check and never feed matching.
|
||
|
||
| Rows on `policyNumber` | Result |
|
||
|---|---|
|
||
| exactly 1 | `MATCHED`, confident — the only unambiguous hit |
|
||
| 0 | new policy: review offers a customer picker, confirm **creates** the row |
|
||
| >1 | surfaced as candidates, human picks |
|
||
|
||
More than one hit is never auto-resolved. Duplicate policy numbers across
|
||
customers do occur (one group policy bound by two related parties), and
|
||
picking arbitrarily would silently book the wrong coverage against the wrong
|
||
person.
|
||
|
||
### Name suggestions on the zero-hit path
|
||
|
||
When the policy number finds nothing — the new-policy case, where a human has
|
||
to pick a customer anyway — `name-matcher.ts` ranks the customer book against
|
||
the printed insured name and the review screen offers the top three as
|
||
one-click buttons above the picker. They are written to
|
||
`policy_ocr_documents.customerSuggestions`, deliberately **not** to
|
||
`matchCandidates`, so a name hint can never be read as a policy-number hit.
|
||
Nothing sets `matchedCustomerId`; the rule above is unchanged.
|
||
|
||
The problem is only ordering: the office books customers surname-first
|
||
(`WAGONER, PAMELA`) and carriers print them given-name-first
|
||
(`PAMELA DENISE WAGONER`), so a string compare never hits while a **token-set**
|
||
compare does. Names are normalized (accents folded, so OCR's `MUNOZ` reaches
|
||
the book's `MUÑOZ`; initials, `DE`/`LA`/`Y`, `JR`, `S.A. DE C.V.` and any token
|
||
containing a digit dropped — ANA prints the phone hard against the name as
|
||
`Ph.3102001538`). Two tiers:
|
||
|
||
| Tier | Rule |
|
||
|---|---|
|
||
| `EXACT` | identical token sets, any order |
|
||
| `PARTIAL` | one set contains the other, ≥2 shared tokens, **and** the surname is present |
|
||
|
||
Both thresholds come from measuring the real book (1536 customers):
|
||
|
||
- 1487 distinct token sets, so `EXACT` cross-person collisions are ~0
|
||
- loosen to surname + first given name and 131 customers (8.5%) collide —
|
||
the book holds `MCWILLIAMS, BRIAN MICHAEL` *and* `MCWILLIAMS, BRIAN`
|
||
- 185 surnames are shared by 524 customers, so one token is never evidence;
|
||
hence the ≥2 floor and the explicit surname requirement, which is what stops
|
||
`JERRY MARILYN` reaching `ESTRADA, JERRY & MARILYN` on given names alone
|
||
|
||
Replaying every book row as a carrier would print it (given-name-first, joint
|
||
spouse dropped): 97.9% top-ranked correct, 0.9% no suggestion, 1.2% a
|
||
different row — and all but two of those are the same human on a duplicate or
|
||
variant row (`MOLNAR, JANOS` vs `MOLNAR, JANOS`, `IBARRA, ISMAEL &`). The
|
||
two genuine wrong-person cases are `CUADROS, JORGE JR` against three
|
||
`CUADROS, JORGE H.`, and they appear as a tie in the list rather than as a
|
||
single answer.
|
||
|
||
A blob is refused outright (>8 tokens or >80 characters): GMX's especificación
|
||
has no field labels and the parser has been seen handing its whole first page
|
||
over as `insuredName`, which would find a surname somewhere in the prose.
|
||
`(SIN NOMBRE)` — 14 rows the migration left — is skipped on both sides.
|
||
|
||
**Not used for utility statements.** There the registrant genuinely is not the
|
||
customer (the `CATT, RANDY` finding above), so the same trick would be wrong,
|
||
not merely noisy.
|
||
|
||
## GMX ships two unrelated documents for the same policy
|
||
|
||
The office downloads both from the same portal, and either can land in a
|
||
batch. They share only the brand and the policy number, so `parseGmx` is a
|
||
two-line dispatcher over two real parsers — both returning `provider: "GMX"`,
|
||
because the matcher keys on the policy number alone and must not care which
|
||
artifact was uploaded.
|
||
|
||
| | **Caratula** (`…_Traduccion.pdf`) | **Especificación** (`…-CondicionesParticulares.pdf`) |
|
||
|---|---|---|
|
||
| Language | English (free translation) | Spanish |
|
||
| Shape | boxed header table + 4-column coverage table | 10 pages of prose, no tables at all |
|
||
| Header fields | Policy / Insured / Broker / Term / From / To / Currency | insured name, risk location, property description |
|
||
| Dates, broker, currency field | yes | **none printed** |
|
||
| Coverages | one row per risk | section heading + `Límite Máximo de Responsabilidad:` |
|
||
| Parser | `parseGmxCaratula` | `parseGmxEspecificacion` |
|
||
|
||
Selected by `isEspecificacion` on the PVL page header
|
||
(`ESPECIFICACIÓN QUE SE ADHIERE`, `PVL Hogar`, `Nombre del asegurado`).
|
||
|
||
Three things about the especificación are worth knowing before touching it:
|
||
|
||
- **The policy number's group widths differ between the two.** The caratula
|
||
reads `007-037-07005947-0000-02` and the especificación
|
||
`07-037-07006957-00000-01` — 2 digits in the first group, 5 in the fourth.
|
||
The original parser pinned the widths, so it read one family and returned
|
||
null on the other. `POLICY_NUMBER_SHAPE` now matches the shape, and since
|
||
the especificación prints the number on all ten page headers, the ten
|
||
readings cross-check each other (disagreement is noted, not resolved — the
|
||
same rule the zona federal parser applies to its clave).
|
||
- **Coverages are found by anchoring on the limit label and walking backwards
|
||
for the heading.** There is no row shape to match. A heading is a short line
|
||
*preceded by a blank line* — that last condition is the whole trick, since
|
||
length alone cannot tell a heading from the wrapped tail of the paragraph
|
||
above it (`efectuados.`, `Y CADA PÉRDIDA.`), and without it coverages get
|
||
named after the last word of the preceding prose.
|
||
- **Vigencia, agente and prima are absent by design**, not unread. The parser
|
||
says so in a note, so a reviewer seeing three empty fields does not read it
|
||
as a broken parse.
|
||
|
||
**These three are keyed in by hand** — confirmed 2026-08-14 with Luz, who
|
||
handles GMX policies at the office. The review screen already has editable
|
||
inputs for all three, and `postPremium` enables off the *typed* premium, so
|
||
a hand-entered prima posts to the ledger exactly like a parsed one. No code
|
||
change was needed to support this; it is a process decision, recorded here
|
||
because the parser's own note now instructs the reviewer accordingly.
|
||
|
||
> **A blank vigencia is silently permanent.** `Policy.policyTo` is nullable
|
||
> and the renewals window query filters `policyTo: { gte, lte }`
|
||
> (`renewals.service.ts`), so a policy confirmed without one **never matches
|
||
> and never gets a renewal notice** — no error, no warning, and nothing
|
||
> later notices. This is why the parser's note names the consequence instead
|
||
> of just listing the missing fields.
|
||
|
||
An excluded catastrophic risk is recorded as excluded **in the risk label**
|
||
(`Terremoto o erupción volcánica — Sección Edificio: EXCLUIDO`) with a null
|
||
amount, never as `0`: a coverage insured for zero and an excluded coverage are
|
||
the same number and very different facts, and `ParsedCoverage` has no field
|
||
for the distinction.
|
||
|
||
## A.N.A. ships two unrelated faces too
|
||
|
||
`A.N.A. Compañía de Seguros` is the Rosarito office's tourist auto book. Same
|
||
split as GMX, different reason: GMX ships two *documents about one policy*,
|
||
A.N.A. ships two *products*.
|
||
|
||
| | **AUTOMOBILE** (`SPECIAL POLICY FOR TOURISTS`) | **DRIVER´S POLICY** (the office says *licencia*) |
|
||
|---|---|---|
|
||
| Insures | a specific car | up to five named drivers, whatever they drive |
|
||
| Vehicle table | `ITEM / YEAR / MAKE / BODY / SERIAL No. / PLATES` | **none** |
|
||
| Insured | one `INSURED` cell | numbered `POLICY HOLDER` list |
|
||
| Value columns | one (`LIMIT OF LIABILITY`) | two (`SUM INSURED`, `PREMIUM`) |
|
||
| Sections | 9, numbered | 6, unnumbered, **in a different order** |
|
||
| Parser | `parseAnaAutomobile` | `parseAnaDriverPolicy` |
|
||
|
||
Selected by `isAnaDriverPolicy` on the title band.
|
||
|
||
The **four automobile products** the office sells — amplia and responsabilidad
|
||
civil, each annual or by-the-day — are the **same layout with different
|
||
numbers**. "Amplia" prints a vehicle value and `COVERED` on sections 1–2;
|
||
"resp. civil" prints `0.00` and `EXCLUDED`. That is data, not a layout, so
|
||
there is one parser rather than four.
|
||
|
||
Things worth knowing before touching the ANA parsers:
|
||
|
||
- **These are born-digital portal PDFs**, so `pdftotext -layout` returns exact
|
||
glyphs and exact columns. The driver's policy parser uses that: `SUM
|
||
INSURED` and `PREMIUM` print the same shape (`100,000.00 usd.` /
|
||
`18.70 usd.`) with no per-row label, so **horizontal position is the only
|
||
thing that separates them**. The split is computed from the header's own
|
||
column offsets rather than hardcoded, because they shift between products.
|
||
If a scan ever arrives without column fidelity, every amount is reported as
|
||
a sum insured and the reviewer is told the split failed.
|
||
- **The money row is read positionally, not by finding six amounts.** An
|
||
unused `DISCOUNT` prints as a bare `-`, so an "amounts in order" reading
|
||
shifts every value one column left on a discounted policy. The parser
|
||
requires exactly six whitespace-separated cells or reports the row unread.
|
||
- **Each PDF prints its face two or three times** (ORIGINAL, AGENT COPY, then
|
||
a summary receipt and three travel ID cards), and the pipeline concatenates
|
||
every page before parsing. The coverage walk is bounded to the first copy
|
||
and the driver list to the first `POLICY HOLDER` block. Unbounded, the
|
||
licencia returns the same person three times — which reads as a
|
||
three-driver policy, not as a bug, so nothing downstream would catch it.
|
||
- **Two five-digit numbers sit in the header band** and only one is the agent
|
||
clave: the other is the agent's own postal code
|
||
(`ROSARITO, BAJA CALIFORNIA 22710`). Likewise the agent's street address
|
||
reads `BENITO JUAREZ 25 No.50 INT 38`, three lines above the `No.` cell that
|
||
holds the policy number — hence the two-space floor after `No.`.
|
||
- **Sections 6–8 print a PREMIUM where the others print a limit.** $40 is what
|
||
legal aid *cost*, not a $40 liability limit, so it lands on
|
||
`ParsedCoverage.premium` (a field GMX never fills) and gets its own column
|
||
on the review screen. Adding the two together would be meaningless.
|
||
- **`coveragePeriodDays` matters here and nowhere else.** A.N.A. sells 3- and
|
||
4-day policies. `Policy.coveragePeriodDays` defaults to 365, so a weekend
|
||
policy left at the default sits in the renewals window a year out. The term
|
||
is derived from the two dates and cross-checked against the printed `DAYS`
|
||
cell; a disagreement is noted rather than resolved.
|
||
|
||
Exclusions follow the GMX rule — recorded in the risk label
|
||
(`MATERIAL DAMAGE — VEHICLE: EXCLUDED`) with a null amount, never as `0`. It
|
||
matters more here: a responsabilidad-civil policy prints `0.00` for material
|
||
damage, so the two are visually identical on the page.
|
||
|
||
### Vehicles and drivers
|
||
|
||
A.N.A. is the first provider whose face carries either, so confirm now writes
|
||
`Vehicle` and `InsuredDriver` rows alongside the `Policy`
|
||
(`applyVehiclesAndDrivers`). The parsed values are stored on the document as
|
||
`extractedVehiclesJson` / `extractedDriversJson` and shown read-only in the
|
||
review queue, so a misread VIN is catchable before it is applied.
|
||
|
||
Both inserts skip a row that already exists on the policy, matched on the
|
||
identifier the document prints — VIN then plate for a vehicle (A.N.A.'s
|
||
TRAILER and TOWING slots have no VIN), licence number then name for a driver.
|
||
The case that forces this is confirming a **renewal** onto an existing policy:
|
||
a blind insert leaves the customer with the same VIN listed twice and no way
|
||
to tell which row the renewal belongs to.
|
||
|
||
Nothing is ever updated or deleted there. A vehicle whose plate changed lands
|
||
as a second row for a human to reconcile — the safe half of the mistake, since
|
||
an overwrite would destroy the only record of what was insured last term.
|
||
|
||
The vehicle table is parsed **by token role, not by column offset**, because
|
||
`BODY` is the cell that wraps: `PACIFICA` is one token and `GENESIS SEDAN` is
|
||
two, so a fixed token count reads the VIN out of the wrong slot on the second.
|
||
The 17-character VIN is the anchor and `BODY` is whatever sits between the
|
||
make and it.
|
||
|
||
## What the parser reads, and the field it cannot
|
||
|
||
`ParsedPolicy` fields are all nullable on purpose: each carrier prints a
|
||
different subset, and the matcher and review queue both work better with
|
||
"field was read" vs "field was not" than with a guess.
|
||
|
||
Read from the GMX certificate: policy number, insured name, additional
|
||
insured, broker (→ `Policy.agentName`), legal address, ZIP, `policyFrom` /
|
||
`policyTo` / `policyDate`, currency, premium-payment cadence, and the full
|
||
per-coverage table (risk, insured amount, deductible, loss participation)
|
||
preserved verbatim.
|
||
|
||
Read from an A.N.A. face: all of the above except additional insured and
|
||
broker parens, **plus** the premium (A.N.A. prints it — see below), the policy
|
||
fee, the total, the term in days, the vehicle table, and the named drivers
|
||
with their US licence numbers. The tax and the agent clave have no column in
|
||
the schema and ride in the notes.
|
||
|
||
> **The GMX certificate carries no premium.** Not "sometimes missing" — the
|
||
> document does not have the figure. It lives on GMX's **separate `recibo`
|
||
> PDF**. The parser leaves `netPremium` / `policyFee` / `brokerFee` / `total`
|
||
> null and pushes a note onto the row —
|
||
> *"esta página no trae prima; revisar el recibo de GMX por separado"* — so
|
||
> the reviewer sees why the field is empty rather than assuming a read
|
||
> failure.
|
||
|
||
This is also why confirm never overwrites an existing `Policy.netPremium`
|
||
with null: the certificate not carrying a premium is not evidence that the
|
||
premium is gone.
|
||
|
||
A.N.A.'s faces do print one — the `DISCOUNT / PREMIUM / POLICY FEE / TAX /
|
||
LOCAL TAX / TOTAL` row is on the same page — so an ANA document reaches the
|
||
review queue with `netPremium` populated and `postPremium` already ticked.
|
||
|
||
Four of those six cells are stored: `PREMIUM` → `netPremium`, `POLICY FEE` →
|
||
`policyFee`, `TAX` → `tax` (`extractedTax` on the document, `Policy.tax` on
|
||
confirm), `TOTAL` → `total`. `DISCOUNT` and `LOCAL TAX` are reported as notes
|
||
instead:
|
||
|
||
- **`DISCOUNT`** has no column, and it prints as a bare `-` when unused, which
|
||
is what makes the row positional rather than "find six amounts".
|
||
- **`LOCAL TAX`** is a separate levy and is deliberately **not** summed into
|
||
`tax`. Folding it in would produce an IVA figure that no longer divides back
|
||
to a rate, which is the reason to store it at all. It reads 0.00 on every
|
||
A.N.A. policy seen so far; a non-zero one raises
|
||
*"impuesto local N no capturado"* and means `total` will not reconcile
|
||
against `netPremium + policyFee + tax`.
|
||
|
||
`Policy.taxRate` is left null by confirm. A.N.A. prints the IVA **amount**, not
|
||
the rate, and back-dividing one would mint a rate the document never stated —
|
||
the capture form resolves it from the line of business instead
|
||
(`PolicyType.taxRate`, see `apps/api/src/policies/premium.ts`). The figures do
|
||
agree: 298.61 + 30.00 taxed at 8% is 26.29, totalling 354.90, asserted in
|
||
`policy-parser.spec.ts`.
|
||
|
||
Deductible and loss participation are stored as **strings** (`"5%"`, `"20%"`,
|
||
`"USD 1,000"`) — they are printed as a mix of percentages, currency amounts
|
||
and free text, and normalising them would lose the distinction.
|
||
|
||
## Policy type and carrier
|
||
|
||
Confirm sets `Policy.policyTypeId` and `Policy.insuranceProviderId` from what
|
||
the parser read.
|
||
|
||
| document | `policyTypeName` |
|
||
|---|---|
|
||
| ANA `AUTOMOBILE` | `AUTO` |
|
||
| ANA `DRIVER´S POLICY` | `LICENCIAS` |
|
||
| GMX caratula **and** especificación | `MULT` |
|
||
|
||
The parser emits a **name**, never an id — it is a pure function over text and
|
||
must not reach for the database, so `resolveLookups()` in the service turns the
|
||
name into a foreign key. A renamed lookup row is then a data change rather than
|
||
a parser change.
|
||
|
||
**Resolve, never create.** A missing `policy_types` row means a human deleted
|
||
it, and silently recreating it would undo that with no record. The field stays
|
||
null and the reviewer adds the row through the lookups screen. An explicit
|
||
`policyTypeId` / `insuranceProviderId` on the confirm payload always wins.
|
||
|
||
Two judgement calls worth recording:
|
||
|
||
- **GMX is `MULT`, not `INCENDIO`.** The caratula's own header reads "Multiple
|
||
Policy / Home" and the especificación is "PVL Hogar" — one product, two
|
||
artifacts. `MULT` is the live row carrying 769 of them; `INCENDIO` is
|
||
fire-only and no policy in the book has ever used it.
|
||
- **The parser's provider code is not the carrier's row name.** The office's
|
||
book is filed under `ANA SEGUROS`, so `PROVIDER_ROW_NAME` maps `ANA` onto it.
|
||
A bare `ANA` row with 1 policy also existed and is merged away by
|
||
`20260815160000_policy_type_repair`.
|
||
|
||
> **Deleting a lookup row used to be silent data loss.** `policies.policyTypeId`,
|
||
> `policies.insuranceProviderId` and `claims.adjusterId` are all
|
||
> `ON DELETE SET NULL`, and the lookups screen deleted unconditionally — so the
|
||
> delete returned 200 and blanked the field on every row that used it. That is
|
||
> how `M_EMPR` vanished and left 5 policies with no ramo, found months later by
|
||
> querying. All three deletes now refuse while the row is in use, naming it and
|
||
> the count. See `assertLookupUnused` and BACKLOG §2.1.
|
||
|
||
## Confirm: what actually gets written
|
||
|
||
Per confirmed document, in order:
|
||
|
||
1. **The `Policy` row** — updated if a policy was matched, created under the
|
||
picked customer if not. Only non-null `extracted*` fields are written; null
|
||
never overwrites existing data. `policyTypeId` and `insuranceProviderId` are
|
||
resolved first (above) and left untouched when unresolvable, so an existing
|
||
policy never loses a type or carrier it already had.
|
||
2. **`Vehicle` and `InsuredDriver` rows** — for the providers whose face
|
||
carries them (A.N.A.; never GMX Hogar), skipping any that already exist on
|
||
the policy. See *Vehicles and drivers* above.
|
||
3. **A `PolicyDocument`** — the source PDF is streamed into the policy's
|
||
storage namespace and attached, so the paperwork stays with the policy. Its
|
||
`documentType` is named after whichever parser claimed the page
|
||
(`ANA_POLICY`, `GMX_POLICY`).
|
||
4. **Optionally a `Transaction`** — `INSURANCE` domain, negative amount
|
||
(a charge), `captureSource: "OCR"`, `captureRef` = the document id.
|
||
|
||
The ledger write is **opt-in twice over**: staff must tick `postPremium`
|
||
*and* a premium must have parsed to a positive number. Without that gate the
|
||
premium-less certificate above would silently book a $0 charge on every
|
||
confirm.
|
||
|
||
`createdPolicyId` and `postedTransactionId` are unique columns on the
|
||
document row, so a double-confirm cannot re-apply — and a `POSTED` document
|
||
is refused outright.
|
||
|
||
Discarding a batch is refused once any page is `POSTED`: a partly-applied
|
||
batch has already written `Policy` (and possibly `Transaction`) rows, and
|
||
hiding the paperwork behind a "discarded" label would leave those rows
|
||
unexplained. Reject the remaining pages individually instead.
|
||
|
||
## API surface
|
||
|
||
| Method | Route | Ability |
|
||
|---|---|---|
|
||
| `GET` | `/policy-ocr/status` (is OCR + storage available) | authenticated |
|
||
| `GET` | `/policy-ocr/batches`, `/batches/:id`, `/batches/:id/documents` | authenticated |
|
||
| `GET` | `/policy-ocr/documents/:id/page` (streams the source PDF) | authenticated |
|
||
| `POST` | `/policy-ocr/batches` (upload) | `policy:ingest` |
|
||
| `PATCH` | `/policy-ocr/documents/:id` (edit the extracted fields) | `policy:ocr-review` |
|
||
| `POST` | `/policy-ocr/documents/:id/reject` | `policy:ocr-review` |
|
||
| `POST` | `/policy-ocr/batches/:id/discard` | `policy:ocr-review` |
|
||
| `POST` | `/policy-ocr/batches/:id/confirm` | `policy:ocr-review` |
|
||
|
||
## Requirements
|
||
|
||
Same as statement OCR: object storage (`S3_ENDPOINT` + credentials) for the
|
||
source PDFs and page images, and `tesseract-ocr` / `tesseract-ocr-data-spa` /
|
||
`poppler-utils` in the API image. `GET /policy-ocr/status` reports both; if
|
||
either is missing the feature reports itself unavailable and only this
|
||
feature is disabled.
|
||
|
||
## Tests
|
||
|
||
`apps/api/src/policy-ocr/parsers/policy-parser.spec.ts` — 58 cases against
|
||
verbatim text extracted from five real documents, indentation and blank lines
|
||
included (the column positions are what the parser reads, so a cleaned-up
|
||
fixture would test nothing — and on A.N.A.'s driver's policy the offsets are
|
||
literally the only thing separating two columns).
|
||
|
||
From `HC_Folio_000767_Traduccion.pdf` (caratula): provider detection from the
|
||
wordmark and from the footer URL, the header fields, every coverage row off
|
||
the second page, the deductible/loss-participation strings, the
|
||
missing-premium note, the broker line with the agent-number parens absent,
|
||
and a page with no GMX signal at all (which must yield no provider rather
|
||
than a bad guess).
|
||
|
||
From `007_LGS-HGMX_07006957_01_0-CondicionesParticulares.pdf`
|
||
(especificación): the differently-grouped policy number, the risk location
|
||
read across its wrapped line, the empty `Asegurado Adicional` cell that must
|
||
not capture the next line, the absent-by-design fields, currency taken from
|
||
the USD limits rather than the M.N. sublimits in the body prose, a limit
|
||
split under `Edificio` / `Contenidos` sub-labels, a limit printed on the
|
||
label's own line, a deductible stated as a sentence *above* its limit, a
|
||
sublimit block whose amount sits after both a blank line and a page break,
|
||
the excluded earthquake coverage, and the hydrometeorological deductible and
|
||
coinsurance pulled from their own per-zone block.
|
||
|
||
Plus four cases in `apps/api/src/policies/lookup-delete-guard.spec.ts` pinning
|
||
the refusal that stops a lookup delete from silently blanking the rows that use
|
||
it, and four in the parser suite on the policy-type NAME each document yields.
|
||
|
||
From the three A.N.A. PDFs: brand detection (and that GMX's layout rules
|
||
cannot claim an ANA page), the header band, DD MM YYYY read out of three
|
||
separate column cells, the six money cells with `DISCOUNT` printed as a bare
|
||
`-`, the vehicle row with a one-word and a two-word `BODY` cell, the empty
|
||
TRAILER/TOWING slots, the agent street number and postal code that must *not*
|
||
be read as the policy number and clave, the declared values labelled by item
|
||
slot, the `$500.00` inside the deductible sentence that is not a sum insured,
|
||
the per-person/per-accident split in both of its printed forms, an add-on's
|
||
figure recorded as a premium, section 9's parenthesised limit, the by-the-day
|
||
term, the excluded sections, the column-position split on the driver's policy,
|
||
its different section order, and — for both faces — that a doubled or tripled
|
||
input yields one set of coverages and one driver rather than one per copy.
|
||
|
||
`apps/api/src/policy-ocr/name-matcher.spec.ts` — 21 cases on the customer name
|
||
suggestions, every fixture name lifted from the real book: the reversed name,
|
||
the printed middle name, the exact row outranking the row that merely contains
|
||
it, the joint account reached from one spouse (and refused when only given
|
||
names are printed), the Spanish double surname with the comma in either place,
|
||
the 54 rows with no comma at all, the `(SIN NOMBRE)` placeholder, a bare shared
|
||
surname, and the page-sized blob. Four more in
|
||
`policy-matcher.service.spec.ts` pin the wiring: suggestions on the zero-hit
|
||
and unreadable-number paths, no book read at all when the policy number hits,
|
||
and one book read across a batch.
|
||
|
||
Four of the GMX cases are regression tests for ways the parser can silently attach
|
||
the *wrong* value rather than none — a neighbouring coverage's prose read as
|
||
a deductible, the page-level `DEDUCIBLES:` paragraph read as one, a coverage
|
||
named after a wrapped prose tail, and one section's per-zone deductible
|
||
adopted by the coverage above it. Each was a real defect caught by running
|
||
the parser against the full ten-page document.
|
||
|
||
## Not built
|
||
|
||
- **GMX and A.N.A. only.** The dispatcher (`detectPolicyProvider`) is a table
|
||
of `[provider, pattern]` pairs plus a `parsers` map, so adding Qualitas is a
|
||
parser function and two entries — but no other carrier's layout has been
|
||
seen yet, and guessing at one produces a parser nobody can verify.
|
||
- **The `recibo` PDF.** Reading the premium off GMX's separate receipt
|
||
document, and pairing it to the certificate it belongs to, is the obvious
|
||
next piece. It is what would let `postPremium` stop being a manual tick.
|
||
- **Renewals from OCR.** A re-issued policy arrives as a new certificate with
|
||
the same number; confirm updates the existing row rather than versioning
|
||
it. Nothing tracks "this is the 2027 issue of that policy".
|
||
|
||
## Related
|
||
|
||
- [`STATEMENT_OCR.md`](STATEMENT_OCR.md) — the utility statement pipeline this
|
||
was lifted from, as built. [`RECEIPT_CAPTURE_SPEC.md`](RECEIPT_CAPTURE_SPEC.md)
|
||
§2 is its design and the measured evidence behind it. Between them they are
|
||
the origin of three rules the policy parser applies: detect the provider by brand before layout, only
|
||
ever apply the Tesseract digit-confusion map (`O→0`, `S→5`, `B→8`, …) to
|
||
fields known to be digits, and parse amounts by separator *position* rather
|
||
than assuming `,` is thousands.
|
||
|
||
Those last two are **duplicated on purpose**, not imported: the module is
|
||
kept self-contained, since sharing a helper would couple two unrelated
|
||
domains through it. If you fix a bug in one, check the other.
|
||
- [`INSURANCE_FEATURES_SPEC.md`](INSURANCE_FEATURES_SPEC.md) — the four
|
||
insurance features that *were* planned. This is not one of them.
|