feat(policy-ocr): read GMX's Spanish PVL especificación layout
GMX ships two unrelated documents for the same policy and the office downloads both from the same portal. The parser only knew the English caratula, so a `…-CondicionesParticulares.pdf` parsed to an almost entirely empty row — including the policy number, which the matcher needs. `parseGmx` becomes a dispatcher over `parseGmxCaratula` (unchanged behaviour) and the new `parseGmxEspecificacion`. Both still report `provider: "GMX"`: the matcher keys on the policy number alone and must not care which artifact was uploaded. The especificación has no tables. Coverages are found by anchoring on `Límite … Responsabilidad:` and walking backwards for the heading, where a heading is a short line *preceded by a blank line* — length alone cannot tell one from the wrapped tail of the paragraph above it, and without that condition coverages get named after the last word of the preceding prose. Also fixed, both pre-existing: - The policy number's group widths are not the same across the two families (`007-037-…-0000-02` vs `07-037-…-00000-01`). The pinned-width regex is replaced by a shape, so both read. - The caratula's ZIP fallback pushed a note saying it had read the ZIP from the address, then never assigned it. Verified against the full ten-page real document: all 17 coverages, amounts, deductibles and the excluded earthquake section match what is printed. 24 parser tests (was 8), four of them regressions for ways this layout can silently attach the *wrong* value rather than none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+78
-9
@@ -84,8 +84,9 @@ feature's core assumption.
|
||||
Utility statements arrive **bundled, one customer per page** — so there, one
|
||||
page is one document and the parser runs per page. A policy PDF is the
|
||||
opposite: the GMX certificate is a 2-page document where page 1 carries the
|
||||
contract header and page 2 carries the per-coverage table, and **both pages
|
||||
describe the same policy**. So the pipeline concatenates every page's text
|
||||
contract header and page 2 carries the per-coverage table (and the PVL
|
||||
especificación runs to ten), and **every page describes the same policy**. So
|
||||
the pipeline concatenates every page's text
|
||||
(`\n\n` between pages, which also keeps `ocrRawText` readable for debugging)
|
||||
and runs the parser and the matcher exactly **once per file**.
|
||||
|
||||
@@ -133,6 +134,52 @@ customers do occur (one group policy bound by two related parties), and
|
||||
picking arbitrarily would silently book the wrong coverage against the wrong
|
||||
person.
|
||||
|
||||
## GMX ships two unrelated documents for the same policy
|
||||
|
||||
The office downloads both from the same portal, and either can land in a
|
||||
batch. They share only the brand and the policy number, so `parseGmx` is a
|
||||
two-line dispatcher over two real parsers — both returning `provider: "GMX"`,
|
||||
because the matcher keys on the policy number alone and must not care which
|
||||
artifact was uploaded.
|
||||
|
||||
| | **Caratula** (`…_Traduccion.pdf`) | **Especificación** (`…-CondicionesParticulares.pdf`) |
|
||||
|---|---|---|
|
||||
| Language | English (free translation) | Spanish |
|
||||
| Shape | boxed header table + 4-column coverage table | 10 pages of prose, no tables at all |
|
||||
| Header fields | Policy / Insured / Broker / Term / From / To / Currency | insured name, risk location, property description |
|
||||
| Dates, broker, currency field | yes | **none printed** |
|
||||
| Coverages | one row per risk | section heading + `Límite Máximo de Responsabilidad:` |
|
||||
| Parser | `parseGmxCaratula` | `parseGmxEspecificacion` |
|
||||
|
||||
Selected by `isEspecificacion` on the PVL page header
|
||||
(`ESPECIFICACIÓN QUE SE ADHIERE`, `PVL Hogar`, `Nombre del asegurado`).
|
||||
|
||||
Three things about the especificación are worth knowing before touching it:
|
||||
|
||||
- **The policy number's group widths differ between the two.** The caratula
|
||||
reads `007-037-07005947-0000-02` and the especificación
|
||||
`07-037-07006957-00000-01` — 2 digits in the first group, 5 in the fourth.
|
||||
The original parser pinned the widths, so it read one family and returned
|
||||
null on the other. `POLICY_NUMBER_SHAPE` now matches the shape, and since
|
||||
the especificación prints the number on all ten page headers, the ten
|
||||
readings cross-check each other (disagreement is noted, not resolved — the
|
||||
same rule the zona federal parser applies to its clave).
|
||||
- **Coverages are found by anchoring on the limit label and walking backwards
|
||||
for the heading.** There is no row shape to match. A heading is a short line
|
||||
*preceded by a blank line* — that last condition is the whole trick, since
|
||||
length alone cannot tell a heading from the wrapped tail of the paragraph
|
||||
above it (`efectuados.`, `Y CADA PÉRDIDA.`), and without it coverages get
|
||||
named after the last word of the preceding prose.
|
||||
- **Vigencia, agente and prima are absent by design**, not unread. The parser
|
||||
says so in a note, so a reviewer seeing four empty fields does not read it
|
||||
as a broken parse. The caratula is the document that carries them.
|
||||
|
||||
An excluded catastrophic risk is recorded as excluded **in the risk label**
|
||||
(`Terremoto o erupción volcánica — Sección Edificio: EXCLUIDO`) with a null
|
||||
amount, never as `0`: a coverage insured for zero and an excluded coverage are
|
||||
the same number and very different facts, and `ParsedCoverage` has no field
|
||||
for the distinction.
|
||||
|
||||
## What the parser reads, and the field it cannot
|
||||
|
||||
`ParsedPolicy` fields are all nullable on purpose: each carrier prints a
|
||||
@@ -210,13 +257,35 @@ feature is disabled.
|
||||
|
||||
## Tests
|
||||
|
||||
`apps/api/src/policy-ocr/parsers/policy-parser.spec.ts` — 8 cases, all
|
||||
against verbatim text extracted from one real document,
|
||||
`HC_Folio_000767_Traduccion.pdf`: provider detection from the wordmark and
|
||||
from the footer URL, the header fields, every coverage row off the second
|
||||
page, the deductible/loss-participation strings, the missing-premium note,
|
||||
the broker line with the agent-number parens absent, and a page with no GMX
|
||||
signal at all (which must yield no provider rather than a bad guess).
|
||||
`apps/api/src/policy-ocr/parsers/policy-parser.spec.ts` — 24 cases against
|
||||
verbatim text extracted from two real documents, indentation and blank lines
|
||||
included (the column positions are what the parser reads, so a cleaned-up
|
||||
fixture would test nothing).
|
||||
|
||||
From `HC_Folio_000767_Traduccion.pdf` (caratula): provider detection from the
|
||||
wordmark and from the footer URL, the header fields, every coverage row off
|
||||
the second page, the deductible/loss-participation strings, the
|
||||
missing-premium note, the broker line with the agent-number parens absent,
|
||||
and a page with no GMX signal at all (which must yield no provider rather
|
||||
than a bad guess).
|
||||
|
||||
From `007_LGS-HGMX_07006957_01_0-CondicionesParticulares.pdf`
|
||||
(especificación): the differently-grouped policy number, the risk location
|
||||
read across its wrapped line, the empty `Asegurado Adicional` cell that must
|
||||
not capture the next line, the absent-by-design fields, currency taken from
|
||||
the USD limits rather than the M.N. sublimits in the body prose, a limit
|
||||
split under `Edificio` / `Contenidos` sub-labels, a limit printed on the
|
||||
label's own line, a deductible stated as a sentence *above* its limit, a
|
||||
sublimit block whose amount sits after both a blank line and a page break,
|
||||
the excluded earthquake coverage, and the hydrometeorological deductible and
|
||||
coinsurance pulled from their own per-zone block.
|
||||
|
||||
Four of those are regression tests for ways the parser can silently attach
|
||||
the *wrong* value rather than none — a neighbouring coverage's prose read as
|
||||
a deductible, the page-level `DEDUCIBLES:` paragraph read as one, a coverage
|
||||
named after a wrapped prose tail, and one section's per-zone deductible
|
||||
adopted by the coverage above it. Each was a real defect caught by running
|
||||
the parser against the full ten-page document.
|
||||
|
||||
## Not built
|
||||
|
||||
|
||||
Reference in New Issue
Block a user