The office books customers surname-first ("WAGONER, PAMELA") and carriers
print them given-name-first ("PAMELA DENISE WAGONER"), so the review screen
made staff retype a name the parser had already read. Comparing normalized
token sets makes the two orderings the same thing.
Only on the zero-hit path, where the policy number found nothing and a human
has to pick a customer anyway. The suggestions are written to a new
`customerSuggestions` column rather than `matchCandidates`, which the review
screen reads as policy-number hits, and they never set `matchedCustomerId` or
`confident` — matching on `Policy.policyNumber` is unchanged.
Two tiers, drawn where the real book has cliffs: EXACT (identical token sets)
and PARTIAL (containment, >=2 shared tokens, surname present). Of 1536
customers, 1487 have a distinct token set, so EXACT cross-person collisions
are ~0; loosen to surname + first given name and 131 (8.5%) collide, and 185
surnames are shared by 524 customers, which is why one token is never enough
and the surname must be printed explicitly. Replaying every book row as a
carrier would print it: 97.9% top-ranked correct, 1.2% a different row, all
but two of those the same human on a duplicate or variant row.
Normalization folds accents (OCR's MUNOZ reaches the book's MUÑOZ), drops
initials, Spanish particles, JR/S.A. DE C.V., and any token with a digit —
ANA prints the phone hard against the name as `Ph.3102001538`. Names over 8
tokens or 80 characters are refused outright, because GMX's especificación
has no field labels and the parser has handed its whole first page over as
`insuredName`.
Not used for utility statements: there the registrant genuinely is not the
customer, so the same trick would be wrong rather than noisy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
555 lines
30 KiB
Markdown
555 lines
30 KiB
Markdown
# Insurance Policy OCR Capture
|
||
|
||
Reads an insurance policy PDF the office downloads from a carrier portal,
|
||
proposes the `Policy` row it should become, and lets staff confirm. Built
|
||
2026-08-01 (`5e9cb12`), live under `/polizas/captura`.
|
||
|
||
## Why this exists — it was not planned
|
||
|
||
This feature is **not in any spec**. It came out of building the utility
|
||
statement OCR intake in [`RECEIPT_CAPTURE_SPEC.md`](RECEIPT_CAPTURE_SPEC.md)
|
||
§2: once there was a working render → OCR → parse → match → review pipeline
|
||
for CFE/CESPT/Telnor receipts, it was obvious the same shape applies to the
|
||
*other* stack of paper this office keys in by hand every week — the carrier
|
||
policy PDFs behind every `Policy` row.
|
||
|
||
The two are the same job with a different document on the scanner. Keeping
|
||
that recognition cheap is the whole point of how it was built: the pipeline
|
||
was **reused, not copied**.
|
||
|
||
- `OcrModule` (`apps/api/src/ocr/ocr.module.ts`) was extracted out of
|
||
`StatementsModule` in this same commit, purely so `PolicyOcrModule` can
|
||
inject `OCR_PROVIDER` without dragging in the statement pipeline.
|
||
`StatementsModule` now imports it and binds nothing itself. That extraction
|
||
was **blocking**: without it the policy module could not resolve the
|
||
provider at all.
|
||
- The engine stays Tesseract behind the same swappable seam, so a managed
|
||
extraction API remains a one-line change in one file for both features.
|
||
- The intake screen is a *mode of* the existing policy-creation screen, the
|
||
same way OCR receipt capture is a mode of Captura — not a new menu entry.
|
||
|
||
## What ships
|
||
|
||
| Piece | Path |
|
||
|---|---|
|
||
| API module | `apps/api/src/policy-ocr/` (service, controller, DTOs, matcher, parser) |
|
||
| Shared OCR seam | `apps/api/src/ocr/ocr.module.ts` |
|
||
| Tables | `policy_ocr_batches`, `policy_ocr_documents` (`20260801000000_policy_ocr_intake`, extended by `20260815120000_policy_ocr_ana` and `20260815160000_policy_type_repair`) |
|
||
| Web | `components/PolicyCaptura.tsx` (tab shell), `PolicyOcrIntake.tsx` (upload), `PolicyOcrReview.tsx` (review queue) |
|
||
| Abilities | `policy:ingest`, `policy:ocr-review` — both **STAFF** |
|
||
|
||
Abilities are STAFF for the same reason statement OCR is: nothing reaches the
|
||
books unconfirmed, and the review step is what makes machine capture safe at
|
||
that tier.
|
||
|
||
## The screen
|
||
|
||
`PolicyCaptura` is one screen with two ways in, mirroring `Captura.tsx`:
|
||
|
||
- `/polizas/nuevo` → **manual** tab (`PolicyForm`, every field by hand)
|
||
- `/polizas/captura` → **automática** tab (`PolicyOcrIntake`, drop a PDF)
|
||
- `/polizas/captura/[id]` → the batch review queue
|
||
|
||
Both modes end at the same place — a `Policy` row on a customer's file — so
|
||
they are modes of one screen rather than two menu entries. Either URL renders
|
||
the same component, so the tab toggle works from either entry point and old
|
||
bookmarks land on the right tab.
|
||
|
||
## Pipeline
|
||
|
||
```
|
||
upload PDF → store source → render pages → text layer? → parse → match → review → confirm
|
||
```
|
||
|
||
1. **Store the source.** `policy-ocr/{batchId}/source-N.pdf`, before anything
|
||
else touches it.
|
||
2. **Render + read.** Every page is rendered to
|
||
`policy-ocr/{batchId}/page-M.png`. Text-layer wins when the PDF has one
|
||
(cheap, exact); the rendered image is OCR'd only when it does not — the
|
||
same precedence rule as the statement pipeline. Carrier-portal PDFs are
|
||
usually born-digital, so most of the time no OCR runs at all.
|
||
3. **Parse.** Provider detected by brand signal first
|
||
(`GMX`, `Grupo Mexicano de Seguros`, `gmx.com.mx`,
|
||
`JUNTOS EL RIESGO ES MENOR`), layout patterns only as fallback — the same
|
||
ordering rule the statement parser needed.
|
||
4. **Match.** Against `Policy.policyNumber`.
|
||
5. **Review + confirm.** Nothing is written to `Policy` until a human
|
||
confirms.
|
||
|
||
### One PDF = one policy
|
||
|
||
This is the sharpest difference from statement OCR, and it inverts that
|
||
feature's core assumption.
|
||
|
||
Utility statements arrive **bundled, one customer per page** — so there, one
|
||
page is one document and the parser runs per page. A policy PDF is the
|
||
opposite: the GMX certificate is a 2-page document where page 1 carries the
|
||
contract header and page 2 carries the per-coverage table (and the PVL
|
||
especificación runs to ten), and **every page describes the same policy**. So
|
||
the pipeline concatenates every page's text
|
||
(`\n\n` between pages, which also keeps `ocrRawText` readable for debugging)
|
||
and runs the parser and the matcher exactly **once per file**.
|
||
|
||
Consequences worth knowing before touching this code:
|
||
|
||
- `PolicyOcrDocument.pageNumber` is repurposed as the **file ordinal within
|
||
the batch** (1, 2, 3…), not a page index. The
|
||
`(batchId, pageNumber)` unique constraint still holds, and one batch still
|
||
carries many policies — one per uploaded file.
|
||
- Parser regexes are anchored across the whole concatenated text (`^From$`,
|
||
`^Currency\s+…`), which is why the page-boundary blank line matters.
|
||
- `ocrConfidence` on the row is the **mean** across the file's pages.
|
||
- A file that fails to parse produces exactly one `OCR_FAILED` row — the right
|
||
granularity, and the page PNGs stay on disk for a re-run after a parser fix.
|
||
|
||
### `storageKey` is the source PDF, not a page image
|
||
|
||
`PolicyOcrDocument.storageKey` points at `source-N.pdf`. The review screen
|
||
embeds that file directly, so the reviewer looks at the **exact artifact the
|
||
office received** and gets the browser's native PDF scrolling, zoom and text
|
||
selection for free. Rendered PNGs are still written for future re-OCR or an
|
||
image-based audit, but nothing points at them as the document's identity.
|
||
|
||
(The statement side does the opposite — there `storageKey` is the page image,
|
||
because a page *is* the document.)
|
||
|
||
## Matching: policy number only, never the insured name
|
||
|
||
`PolicyMatcherService` matches on `Policy.policyNumber` and nothing else.
|
||
|
||
The certificate's "Insured" line is the account's registrant, which drifts
|
||
from the customer the office actually holds the file under — the same finding
|
||
the statement matcher is built around (a CESPT receipt reading
|
||
`ARNAIZ ROSAS ELSA AURORA` for a customer this office holds as `CATT, RANDY`).
|
||
Names are shown to the reviewer as a sanity check and never feed matching.
|
||
|
||
| Rows on `policyNumber` | Result |
|
||
|---|---|
|
||
| exactly 1 | `MATCHED`, confident — the only unambiguous hit |
|
||
| 0 | new policy: review offers a customer picker, confirm **creates** the row |
|
||
| >1 | surfaced as candidates, human picks |
|
||
|
||
More than one hit is never auto-resolved. Duplicate policy numbers across
|
||
customers do occur (one group policy bound by two related parties), and
|
||
picking arbitrarily would silently book the wrong coverage against the wrong
|
||
person.
|
||
|
||
### Name suggestions on the zero-hit path
|
||
|
||
When the policy number finds nothing — the new-policy case, where a human has
|
||
to pick a customer anyway — `name-matcher.ts` ranks the customer book against
|
||
the printed insured name and the review screen offers the top three as
|
||
one-click buttons above the picker. They are written to
|
||
`policy_ocr_documents.customerSuggestions`, deliberately **not** to
|
||
`matchCandidates`, so a name hint can never be read as a policy-number hit.
|
||
Nothing sets `matchedCustomerId`; the rule above is unchanged.
|
||
|
||
The problem is only ordering: the office books customers surname-first
|
||
(`WAGONER, PAMELA`) and carriers print them given-name-first
|
||
(`PAMELA DENISE WAGONER`), so a string compare never hits while a **token-set**
|
||
compare does. Names are normalized (accents folded, so OCR's `MUNOZ` reaches
|
||
the book's `MUÑOZ`; initials, `DE`/`LA`/`Y`, `JR`, `S.A. DE C.V.` and any token
|
||
containing a digit dropped — ANA prints the phone hard against the name as
|
||
`Ph.3102001538`). Two tiers:
|
||
|
||
| Tier | Rule |
|
||
|---|---|
|
||
| `EXACT` | identical token sets, any order |
|
||
| `PARTIAL` | one set contains the other, ≥2 shared tokens, **and** the surname is present |
|
||
|
||
Both thresholds come from measuring the real book (1536 customers):
|
||
|
||
- 1487 distinct token sets, so `EXACT` cross-person collisions are ~0
|
||
- loosen to surname + first given name and 131 customers (8.5%) collide —
|
||
the book holds `MCWILLIAMS, BRIAN MICHAEL` *and* `MCWILLIAMS, BRIAN`
|
||
- 185 surnames are shared by 524 customers, so one token is never evidence;
|
||
hence the ≥2 floor and the explicit surname requirement, which is what stops
|
||
`JERRY MARILYN` reaching `ESTRADA, JERRY & MARILYN` on given names alone
|
||
|
||
Replaying every book row as a carrier would print it (given-name-first, joint
|
||
spouse dropped): 97.9% top-ranked correct, 0.9% no suggestion, 1.2% a
|
||
different row — and all but two of those are the same human on a duplicate or
|
||
variant row (`MOLNAR, JANOS` vs `MOLNAR, JANOS`, `IBARRA, ISMAEL &`). The
|
||
two genuine wrong-person cases are `CUADROS, JORGE JR` against three
|
||
`CUADROS, JORGE H.`, and they appear as a tie in the list rather than as a
|
||
single answer.
|
||
|
||
A blob is refused outright (>8 tokens or >80 characters): GMX's especificación
|
||
has no field labels and the parser has been seen handing its whole first page
|
||
over as `insuredName`, which would find a surname somewhere in the prose.
|
||
`(SIN NOMBRE)` — 14 rows the migration left — is skipped on both sides.
|
||
|
||
**Not used for utility statements.** There the registrant genuinely is not the
|
||
customer (the `CATT, RANDY` finding above), so the same trick would be wrong,
|
||
not merely noisy.
|
||
|
||
## GMX ships two unrelated documents for the same policy
|
||
|
||
The office downloads both from the same portal, and either can land in a
|
||
batch. They share only the brand and the policy number, so `parseGmx` is a
|
||
two-line dispatcher over two real parsers — both returning `provider: "GMX"`,
|
||
because the matcher keys on the policy number alone and must not care which
|
||
artifact was uploaded.
|
||
|
||
| | **Caratula** (`…_Traduccion.pdf`) | **Especificación** (`…-CondicionesParticulares.pdf`) |
|
||
|---|---|---|
|
||
| Language | English (free translation) | Spanish |
|
||
| Shape | boxed header table + 4-column coverage table | 10 pages of prose, no tables at all |
|
||
| Header fields | Policy / Insured / Broker / Term / From / To / Currency | insured name, risk location, property description |
|
||
| Dates, broker, currency field | yes | **none printed** |
|
||
| Coverages | one row per risk | section heading + `Límite Máximo de Responsabilidad:` |
|
||
| Parser | `parseGmxCaratula` | `parseGmxEspecificacion` |
|
||
|
||
Selected by `isEspecificacion` on the PVL page header
|
||
(`ESPECIFICACIÓN QUE SE ADHIERE`, `PVL Hogar`, `Nombre del asegurado`).
|
||
|
||
Three things about the especificación are worth knowing before touching it:
|
||
|
||
- **The policy number's group widths differ between the two.** The caratula
|
||
reads `007-037-07005947-0000-02` and the especificación
|
||
`07-037-07006957-00000-01` — 2 digits in the first group, 5 in the fourth.
|
||
The original parser pinned the widths, so it read one family and returned
|
||
null on the other. `POLICY_NUMBER_SHAPE` now matches the shape, and since
|
||
the especificación prints the number on all ten page headers, the ten
|
||
readings cross-check each other (disagreement is noted, not resolved — the
|
||
same rule the zona federal parser applies to its clave).
|
||
- **Coverages are found by anchoring on the limit label and walking backwards
|
||
for the heading.** There is no row shape to match. A heading is a short line
|
||
*preceded by a blank line* — that last condition is the whole trick, since
|
||
length alone cannot tell a heading from the wrapped tail of the paragraph
|
||
above it (`efectuados.`, `Y CADA PÉRDIDA.`), and without it coverages get
|
||
named after the last word of the preceding prose.
|
||
- **Vigencia, agente and prima are absent by design**, not unread. The parser
|
||
says so in a note, so a reviewer seeing three empty fields does not read it
|
||
as a broken parse.
|
||
|
||
**These three are keyed in by hand** — confirmed 2026-08-14 with Luz, who
|
||
handles GMX policies at the office. The review screen already has editable
|
||
inputs for all three, and `postPremium` enables off the *typed* premium, so
|
||
a hand-entered prima posts to the ledger exactly like a parsed one. No code
|
||
change was needed to support this; it is a process decision, recorded here
|
||
because the parser's own note now instructs the reviewer accordingly.
|
||
|
||
> **A blank vigencia is silently permanent.** `Policy.policyTo` is nullable
|
||
> and the renewals window query filters `policyTo: { gte, lte }`
|
||
> (`renewals.service.ts`), so a policy confirmed without one **never matches
|
||
> and never gets a renewal notice** — no error, no warning, and nothing
|
||
> later notices. This is why the parser's note names the consequence instead
|
||
> of just listing the missing fields.
|
||
|
||
An excluded catastrophic risk is recorded as excluded **in the risk label**
|
||
(`Terremoto o erupción volcánica — Sección Edificio: EXCLUIDO`) with a null
|
||
amount, never as `0`: a coverage insured for zero and an excluded coverage are
|
||
the same number and very different facts, and `ParsedCoverage` has no field
|
||
for the distinction.
|
||
|
||
## A.N.A. ships two unrelated faces too
|
||
|
||
`A.N.A. Compañía de Seguros` is the Rosarito office's tourist auto book. Same
|
||
split as GMX, different reason: GMX ships two *documents about one policy*,
|
||
A.N.A. ships two *products*.
|
||
|
||
| | **AUTOMOBILE** (`SPECIAL POLICY FOR TOURISTS`) | **DRIVER´S POLICY** (the office says *licencia*) |
|
||
|---|---|---|
|
||
| Insures | a specific car | up to five named drivers, whatever they drive |
|
||
| Vehicle table | `ITEM / YEAR / MAKE / BODY / SERIAL No. / PLATES` | **none** |
|
||
| Insured | one `INSURED` cell | numbered `POLICY HOLDER` list |
|
||
| Value columns | one (`LIMIT OF LIABILITY`) | two (`SUM INSURED`, `PREMIUM`) |
|
||
| Sections | 9, numbered | 6, unnumbered, **in a different order** |
|
||
| Parser | `parseAnaAutomobile` | `parseAnaDriverPolicy` |
|
||
|
||
Selected by `isAnaDriverPolicy` on the title band.
|
||
|
||
The **four automobile products** the office sells — amplia and responsabilidad
|
||
civil, each annual or by-the-day — are the **same layout with different
|
||
numbers**. "Amplia" prints a vehicle value and `COVERED` on sections 1–2;
|
||
"resp. civil" prints `0.00` and `EXCLUDED`. That is data, not a layout, so
|
||
there is one parser rather than four.
|
||
|
||
Things worth knowing before touching the ANA parsers:
|
||
|
||
- **These are born-digital portal PDFs**, so `pdftotext -layout` returns exact
|
||
glyphs and exact columns. The driver's policy parser uses that: `SUM
|
||
INSURED` and `PREMIUM` print the same shape (`100,000.00 usd.` /
|
||
`18.70 usd.`) with no per-row label, so **horizontal position is the only
|
||
thing that separates them**. The split is computed from the header's own
|
||
column offsets rather than hardcoded, because they shift between products.
|
||
If a scan ever arrives without column fidelity, every amount is reported as
|
||
a sum insured and the reviewer is told the split failed.
|
||
- **The money row is read positionally, not by finding six amounts.** An
|
||
unused `DISCOUNT` prints as a bare `-`, so an "amounts in order" reading
|
||
shifts every value one column left on a discounted policy. The parser
|
||
requires exactly six whitespace-separated cells or reports the row unread.
|
||
- **Each PDF prints its face two or three times** (ORIGINAL, AGENT COPY, then
|
||
a summary receipt and three travel ID cards), and the pipeline concatenates
|
||
every page before parsing. The coverage walk is bounded to the first copy
|
||
and the driver list to the first `POLICY HOLDER` block. Unbounded, the
|
||
licencia returns the same person three times — which reads as a
|
||
three-driver policy, not as a bug, so nothing downstream would catch it.
|
||
- **Two five-digit numbers sit in the header band** and only one is the agent
|
||
clave: the other is the agent's own postal code
|
||
(`ROSARITO, BAJA CALIFORNIA 22710`). Likewise the agent's street address
|
||
reads `BENITO JUAREZ 25 No.50 INT 38`, three lines above the `No.` cell that
|
||
holds the policy number — hence the two-space floor after `No.`.
|
||
- **Sections 6–8 print a PREMIUM where the others print a limit.** $40 is what
|
||
legal aid *cost*, not a $40 liability limit, so it lands on
|
||
`ParsedCoverage.premium` (a field GMX never fills) and gets its own column
|
||
on the review screen. Adding the two together would be meaningless.
|
||
- **`coveragePeriodDays` matters here and nowhere else.** A.N.A. sells 3- and
|
||
4-day policies. `Policy.coveragePeriodDays` defaults to 365, so a weekend
|
||
policy left at the default sits in the renewals window a year out. The term
|
||
is derived from the two dates and cross-checked against the printed `DAYS`
|
||
cell; a disagreement is noted rather than resolved.
|
||
|
||
Exclusions follow the GMX rule — recorded in the risk label
|
||
(`MATERIAL DAMAGE — VEHICLE: EXCLUDED`) with a null amount, never as `0`. It
|
||
matters more here: a responsabilidad-civil policy prints `0.00` for material
|
||
damage, so the two are visually identical on the page.
|
||
|
||
### Vehicles and drivers
|
||
|
||
A.N.A. is the first provider whose face carries either, so confirm now writes
|
||
`Vehicle` and `InsuredDriver` rows alongside the `Policy`
|
||
(`applyVehiclesAndDrivers`). The parsed values are stored on the document as
|
||
`extractedVehiclesJson` / `extractedDriversJson` and shown read-only in the
|
||
review queue, so a misread VIN is catchable before it is applied.
|
||
|
||
Both inserts skip a row that already exists on the policy, matched on the
|
||
identifier the document prints — VIN then plate for a vehicle (A.N.A.'s
|
||
TRAILER and TOWING slots have no VIN), licence number then name for a driver.
|
||
The case that forces this is confirming a **renewal** onto an existing policy:
|
||
a blind insert leaves the customer with the same VIN listed twice and no way
|
||
to tell which row the renewal belongs to.
|
||
|
||
Nothing is ever updated or deleted there. A vehicle whose plate changed lands
|
||
as a second row for a human to reconcile — the safe half of the mistake, since
|
||
an overwrite would destroy the only record of what was insured last term.
|
||
|
||
The vehicle table is parsed **by token role, not by column offset**, because
|
||
`BODY` is the cell that wraps: `PACIFICA` is one token and `GENESIS SEDAN` is
|
||
two, so a fixed token count reads the VIN out of the wrong slot on the second.
|
||
The 17-character VIN is the anchor and `BODY` is whatever sits between the
|
||
make and it.
|
||
|
||
## What the parser reads, and the field it cannot
|
||
|
||
`ParsedPolicy` fields are all nullable on purpose: each carrier prints a
|
||
different subset, and the matcher and review queue both work better with
|
||
"field was read" vs "field was not" than with a guess.
|
||
|
||
Read from the GMX certificate: policy number, insured name, additional
|
||
insured, broker (→ `Policy.agentName`), legal address, ZIP, `policyFrom` /
|
||
`policyTo` / `policyDate`, currency, premium-payment cadence, and the full
|
||
per-coverage table (risk, insured amount, deductible, loss participation)
|
||
preserved verbatim.
|
||
|
||
Read from an A.N.A. face: all of the above except additional insured and
|
||
broker parens, **plus** the premium (A.N.A. prints it — see below), the policy
|
||
fee, the total, the term in days, the vehicle table, and the named drivers
|
||
with their US licence numbers. The tax and the agent clave have no column in
|
||
the schema and ride in the notes.
|
||
|
||
> **The GMX certificate carries no premium.** Not "sometimes missing" — the
|
||
> document does not have the figure. It lives on GMX's **separate `recibo`
|
||
> PDF**. The parser leaves `netPremium` / `policyFee` / `brokerFee` / `total`
|
||
> null and pushes a note onto the row —
|
||
> *"esta página no trae prima; revisar el recibo de GMX por separado"* — so
|
||
> the reviewer sees why the field is empty rather than assuming a read
|
||
> failure.
|
||
|
||
This is also why confirm never overwrites an existing `Policy.netPremium`
|
||
with null: the certificate not carrying a premium is not evidence that the
|
||
premium is gone.
|
||
|
||
A.N.A.'s faces do print one — the `DISCOUNT / PREMIUM / POLICY FEE / TAX /
|
||
LOCAL TAX / TOTAL` row is on the same page — so an ANA document reaches the
|
||
review queue with `netPremium` populated and `postPremium` already ticked.
|
||
|
||
Deductible and loss participation are stored as **strings** (`"5%"`, `"20%"`,
|
||
`"USD 1,000"`) — they are printed as a mix of percentages, currency amounts
|
||
and free text, and normalising them would lose the distinction.
|
||
|
||
## Policy type and carrier
|
||
|
||
Confirm sets `Policy.policyTypeId` and `Policy.insuranceProviderId` from what
|
||
the parser read.
|
||
|
||
| document | `policyTypeName` |
|
||
|---|---|
|
||
| ANA `AUTOMOBILE` | `AUTO` |
|
||
| ANA `DRIVER´S POLICY` | `LICENCIAS` |
|
||
| GMX caratula **and** especificación | `MULT` |
|
||
|
||
The parser emits a **name**, never an id — it is a pure function over text and
|
||
must not reach for the database, so `resolveLookups()` in the service turns the
|
||
name into a foreign key. A renamed lookup row is then a data change rather than
|
||
a parser change.
|
||
|
||
**Resolve, never create.** A missing `policy_types` row means a human deleted
|
||
it, and silently recreating it would undo that with no record. The field stays
|
||
null and the reviewer adds the row through the lookups screen. An explicit
|
||
`policyTypeId` / `insuranceProviderId` on the confirm payload always wins.
|
||
|
||
Two judgement calls worth recording:
|
||
|
||
- **GMX is `MULT`, not `INCENDIO`.** The caratula's own header reads "Multiple
|
||
Policy / Home" and the especificación is "PVL Hogar" — one product, two
|
||
artifacts. `MULT` is the live row carrying 769 of them; `INCENDIO` is
|
||
fire-only and no policy in the book has ever used it.
|
||
- **The parser's provider code is not the carrier's row name.** The office's
|
||
book is filed under `ANA SEGUROS`, so `PROVIDER_ROW_NAME` maps `ANA` onto it.
|
||
A bare `ANA` row with 1 policy also existed and is merged away by
|
||
`20260815160000_policy_type_repair`.
|
||
|
||
> **Deleting a lookup row used to be silent data loss.** `policies.policyTypeId`,
|
||
> `policies.insuranceProviderId` and `claims.adjusterId` are all
|
||
> `ON DELETE SET NULL`, and the lookups screen deleted unconditionally — so the
|
||
> delete returned 200 and blanked the field on every row that used it. That is
|
||
> how `M_EMPR` vanished and left 5 policies with no ramo, found months later by
|
||
> querying. All three deletes now refuse while the row is in use, naming it and
|
||
> the count. See `assertLookupUnused` and BACKLOG §2.1.
|
||
|
||
## Confirm: what actually gets written
|
||
|
||
Per confirmed document, in order:
|
||
|
||
1. **The `Policy` row** — updated if a policy was matched, created under the
|
||
picked customer if not. Only non-null `extracted*` fields are written; null
|
||
never overwrites existing data. `policyTypeId` and `insuranceProviderId` are
|
||
resolved first (above) and left untouched when unresolvable, so an existing
|
||
policy never loses a type or carrier it already had.
|
||
2. **`Vehicle` and `InsuredDriver` rows** — for the providers whose face
|
||
carries them (A.N.A.; never GMX Hogar), skipping any that already exist on
|
||
the policy. See *Vehicles and drivers* above.
|
||
3. **A `PolicyDocument`** — the source PDF is streamed into the policy's
|
||
storage namespace and attached, so the paperwork stays with the policy. Its
|
||
`documentType` is named after whichever parser claimed the page
|
||
(`ANA_POLICY`, `GMX_POLICY`).
|
||
4. **Optionally a `Transaction`** — `INSURANCE` domain, negative amount
|
||
(a charge), `captureSource: "OCR"`, `captureRef` = the document id.
|
||
|
||
The ledger write is **opt-in twice over**: staff must tick `postPremium`
|
||
*and* a premium must have parsed to a positive number. Without that gate the
|
||
premium-less certificate above would silently book a $0 charge on every
|
||
confirm.
|
||
|
||
`createdPolicyId` and `postedTransactionId` are unique columns on the
|
||
document row, so a double-confirm cannot re-apply — and a `POSTED` document
|
||
is refused outright.
|
||
|
||
Discarding a batch is refused once any page is `POSTED`: a partly-applied
|
||
batch has already written `Policy` (and possibly `Transaction`) rows, and
|
||
hiding the paperwork behind a "discarded" label would leave those rows
|
||
unexplained. Reject the remaining pages individually instead.
|
||
|
||
## API surface
|
||
|
||
| Method | Route | Ability |
|
||
|---|---|---|
|
||
| `GET` | `/policy-ocr/status` (is OCR + storage available) | authenticated |
|
||
| `GET` | `/policy-ocr/batches`, `/batches/:id`, `/batches/:id/documents` | authenticated |
|
||
| `GET` | `/policy-ocr/documents/:id/page` (streams the source PDF) | authenticated |
|
||
| `POST` | `/policy-ocr/batches` (upload) | `policy:ingest` |
|
||
| `PATCH` | `/policy-ocr/documents/:id` (edit the extracted fields) | `policy:ocr-review` |
|
||
| `POST` | `/policy-ocr/documents/:id/reject` | `policy:ocr-review` |
|
||
| `POST` | `/policy-ocr/batches/:id/discard` | `policy:ocr-review` |
|
||
| `POST` | `/policy-ocr/batches/:id/confirm` | `policy:ocr-review` |
|
||
|
||
## Requirements
|
||
|
||
Same as statement OCR: object storage (`S3_ENDPOINT` + credentials) for the
|
||
source PDFs and page images, and `tesseract-ocr` / `tesseract-ocr-data-spa` /
|
||
`poppler-utils` in the API image. `GET /policy-ocr/status` reports both; if
|
||
either is missing the feature reports itself unavailable and only this
|
||
feature is disabled.
|
||
|
||
## Tests
|
||
|
||
`apps/api/src/policy-ocr/parsers/policy-parser.spec.ts` — 58 cases against
|
||
verbatim text extracted from five real documents, indentation and blank lines
|
||
included (the column positions are what the parser reads, so a cleaned-up
|
||
fixture would test nothing — and on A.N.A.'s driver's policy the offsets are
|
||
literally the only thing separating two columns).
|
||
|
||
From `HC_Folio_000767_Traduccion.pdf` (caratula): provider detection from the
|
||
wordmark and from the footer URL, the header fields, every coverage row off
|
||
the second page, the deductible/loss-participation strings, the
|
||
missing-premium note, the broker line with the agent-number parens absent,
|
||
and a page with no GMX signal at all (which must yield no provider rather
|
||
than a bad guess).
|
||
|
||
From `007_LGS-HGMX_07006957_01_0-CondicionesParticulares.pdf`
|
||
(especificación): the differently-grouped policy number, the risk location
|
||
read across its wrapped line, the empty `Asegurado Adicional` cell that must
|
||
not capture the next line, the absent-by-design fields, currency taken from
|
||
the USD limits rather than the M.N. sublimits in the body prose, a limit
|
||
split under `Edificio` / `Contenidos` sub-labels, a limit printed on the
|
||
label's own line, a deductible stated as a sentence *above* its limit, a
|
||
sublimit block whose amount sits after both a blank line and a page break,
|
||
the excluded earthquake coverage, and the hydrometeorological deductible and
|
||
coinsurance pulled from their own per-zone block.
|
||
|
||
Plus four cases in `apps/api/src/policies/lookup-delete-guard.spec.ts` pinning
|
||
the refusal that stops a lookup delete from silently blanking the rows that use
|
||
it, and four in the parser suite on the policy-type NAME each document yields.
|
||
|
||
From the three A.N.A. PDFs: brand detection (and that GMX's layout rules
|
||
cannot claim an ANA page), the header band, DD MM YYYY read out of three
|
||
separate column cells, the six money cells with `DISCOUNT` printed as a bare
|
||
`-`, the vehicle row with a one-word and a two-word `BODY` cell, the empty
|
||
TRAILER/TOWING slots, the agent street number and postal code that must *not*
|
||
be read as the policy number and clave, the declared values labelled by item
|
||
slot, the `$500.00` inside the deductible sentence that is not a sum insured,
|
||
the per-person/per-accident split in both of its printed forms, an add-on's
|
||
figure recorded as a premium, section 9's parenthesised limit, the by-the-day
|
||
term, the excluded sections, the column-position split on the driver's policy,
|
||
its different section order, and — for both faces — that a doubled or tripled
|
||
input yields one set of coverages and one driver rather than one per copy.
|
||
|
||
`apps/api/src/policy-ocr/name-matcher.spec.ts` — 21 cases on the customer name
|
||
suggestions, every fixture name lifted from the real book: the reversed name,
|
||
the printed middle name, the exact row outranking the row that merely contains
|
||
it, the joint account reached from one spouse (and refused when only given
|
||
names are printed), the Spanish double surname with the comma in either place,
|
||
the 54 rows with no comma at all, the `(SIN NOMBRE)` placeholder, a bare shared
|
||
surname, and the page-sized blob. Four more in
|
||
`policy-matcher.service.spec.ts` pin the wiring: suggestions on the zero-hit
|
||
and unreadable-number paths, no book read at all when the policy number hits,
|
||
and one book read across a batch.
|
||
|
||
Four of the GMX cases are regression tests for ways the parser can silently attach
|
||
the *wrong* value rather than none — a neighbouring coverage's prose read as
|
||
a deductible, the page-level `DEDUCIBLES:` paragraph read as one, a coverage
|
||
named after a wrapped prose tail, and one section's per-zone deductible
|
||
adopted by the coverage above it. Each was a real defect caught by running
|
||
the parser against the full ten-page document.
|
||
|
||
## Not built
|
||
|
||
- **GMX and A.N.A. only.** The dispatcher (`detectPolicyProvider`) is a table
|
||
of `[provider, pattern]` pairs plus a `parsers` map, so adding Qualitas is a
|
||
parser function and two entries — but no other carrier's layout has been
|
||
seen yet, and guessing at one produces a parser nobody can verify.
|
||
- **The `recibo` PDF.** Reading the premium off GMX's separate receipt
|
||
document, and pairing it to the certificate it belongs to, is the obvious
|
||
next piece. It is what would let `postPremium` stop being a manual tick.
|
||
- **Renewals from OCR.** A re-issued policy arrives as a new certificate with
|
||
the same number; confirm updates the existing row rather than versioning
|
||
it. Nothing tracks "this is the 2027 issue of that policy".
|
||
|
||
## Related
|
||
|
||
- [`STATEMENT_OCR.md`](STATEMENT_OCR.md) — the utility statement pipeline this
|
||
was lifted from, as built. [`RECEIPT_CAPTURE_SPEC.md`](RECEIPT_CAPTURE_SPEC.md)
|
||
§2 is its design and the measured evidence behind it. Between them they are
|
||
the origin of three rules the policy parser applies: detect the provider by brand before layout, only
|
||
ever apply the Tesseract digit-confusion map (`O→0`, `S→5`, `B→8`, …) to
|
||
fields known to be digits, and parse amounts by separator *position* rather
|
||
than assuming `,` is thousands.
|
||
|
||
Those last two are **duplicated on purpose**, not imported: the module is
|
||
kept self-contained, since sharing a helper would couple two unrelated
|
||
domains through it. If you fix a bug in one, check the other.
|
||
- [`INSURANCE_FEATURES_SPEC.md`](INSURANCE_FEATURES_SPEC.md) — the four
|
||
insurance features that *were* planned. This is not one of them.
|