fa38ff581e5e006ea38282c3c364b83f1c2336cb
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  rmancinasandClaude Opus 5
|
75e9f582b4 |
feat(policy-ocr): store the IVA A.N.A. already prints
The parser has been reading A.N.A.'s `TAX` cell since the ANA layout landed,
but `ParsedPolicy` had nowhere to put it, so the figure only ever reached a
review note and every OCR-confirmed policy was written with `tax` null — even
though the paper states it.
`TAX` now flows parser -> `extractedTax` -> `Policy.tax`, alongside the premium
fields beside it. GMX stays null: its certificate carries no premium at all, so
there is no tax on it either.
The other two cells stay out, for reasons worth keeping:
- `LOCAL TAX` is a separate levy with no destination column, and summing it
into `tax` would produce an IVA that no longer divides back to a rate —
which is the whole reason to store the figure. It reads 0.00 on every
policy seen so far; a non-zero one now raises a note saying the total will
not reconcile, instead of quietly inflating the IVA.
- `DISCOUNT` has no column and prints as a bare "-" when unused, which is
what makes the row positional rather than "find six amounts".
`taxRate` is left null by confirm. A.N.A. prints the amount, not the rate, and
back-dividing it would mint a rate the document never stated; the capture form
resolves one from the line of business instead.
The review screen gains derecho de póliza next to the new IVA field. It was
already parsed and already written on confirm, but never shown — and an IVA
with no fee beside it leaves the reviewer unable to see why premium + fee + tax
equals the printed total.
The spec's assertion moved off the note and onto the field, plus a check that
the row reconciles: 298.61 + 30.00 at 8% is 26.29, totalling 354.90. That
agreement is what proves the positional mapping landed on the right cells
rather than merely on six numbers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
48e01ddd21 |
feat(policies): capture the full premium breakdown
The capture form only ever had prima neta, derecho de póliza and comisión.
The Access form it replaces has seven figures, and the four that were missing
are the ones that make a policy paid in installments add up.
Adds recargo, IVA, prima total and forma de pago to the policy header, the
same breakdown per installment, and a per-line-of-business IVA rate.
IVA and prima total are the only derived figures:
base = prima neta + recargo + derecho de póliza
IVA = round(base * tasa)
total = base + IVA
The recargo is inside the taxable base. That is not a guess — policy 7006785
prints IVA 52.03 on 610.86 + 8.55 + 31.00, and leaving the recargo out gives
51.35, which matches nothing on the page. Both of its money rows are asserted
in premium.spec.ts. The recargo itself is never derived: the carrier quotes it,
so staff key it in, and the field is disabled on ANNUAL/SINGLE. Both derived
figures are stored rather than recomputed on read, and stay editable, because
the printed policy is the record of truth and a later rate change must not
silently restate what was issued.
The rate lives on PolicyType (seeded to 0.08, editable in Catálogos), which is
the legacy one-row IMPUESTOS / IMPUESTOS_AUTOS tables made configurable. The
rate applied is stamped on the policy so an old one reads back at its original
rate.
Per-installment, not two fixed slots on the header: a policy split into several
exhibiciones prices each payment separately — that is why the Access form drew
the money row twice — and a trimestral policy needs four, which the Access
layout could not hold.
Also fixes two losses in the ETL, which is how these went missing:
- `forma_pago` was marked consumed by the coverage sweep and then never
written to any column, so FORMA PAGO existed nowhere in the platform.
- `recargo` and the whole second money row fell into `coveragesJson` as
loose strings, mislabeled as coverage amounts.
transform_policies.py now writes all of it directly;
backfill_policy_premium_breakdown.py recovers it on a database that must not be
re-imported, and strips the migrated keys back out of coveragesJson. Both are
COALESCE-only, so a figure a human has corrected in the app wins.
IVA and TOTAL are NOT backfilled: they were unbound calculated controls on the
Access form, never columns, so there is nothing to recover and every migrated
policy reads null until it is edited.
The backfill warns on 5 annual policies that carry a non-zero recargo — a
contradiction that predates this change and is left for a human, not silently
corrected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
022d1935ad |
feat(policy-ocr): set policyTypeId and insuranceProviderId on confirm
The BACKLOG claimed this was blocked on incomplete `policy_types` rows.
Querying the dev database says otherwise: AUTO (1316 policies) and LICENCIAS
(306) are both live and healthy, so ANA's two faces were never blocked at
all. Three separate things had been conflated.
What the parser now emits is a NAME, not an id -- it is a pure function over
text and must not reach for the database:
ANA AUTOMOBILE -> AUTO
ANA DRIVER'S POLICY -> LICENCIAS
GMX (both documents) -> MULT
`resolveLookups()` turns that into a foreign key at confirm, and does the
same for the carrier off the parser's provider code. It resolves, never
creates: a missing `policy_types` row means a human deleted it, and silently
recreating it would undo that with no record. An explicit `policyTypeId` /
`insuranceProviderId` on the confirm payload always wins.
GMX is MULT rather than INCENDIO because the caratula's own header reads
"Multiple Policy / Home" and the especificación is "PVL Hogar" -- one product,
two artifacts. MULT is the live row carrying 769 of them; INCENDIO is fire-only
and no policy in the book has ever used it.
The parser's provider code is not the carrier's row name, so PROVIDER_ROW_NAME
maps ANA onto "ANA SEGUROS", which is where the office's 738 ANA policies
already are.
--- the actual defect underneath -----------------------------------------
`policies.policyTypeId`, `policies.insuranceProviderId` and
`claims.adjusterId` are all ON DELETE SET NULL, and the lookups screen deleted
unconditionally. So deleting a lookup row returned 200 and silently blanked
the field on every row referencing it -- no error, nothing in the UI. That is
how M_EMPR disappeared and left 5 policies with no ramo, found months later
only by querying.
All three deletes now refuse while the row is in use, naming it and the count
("El tipo de póliza «M_EMPR» está en uso por 5 póliza(s)"). The schema-level
`onDelete: Restrict` the spec once recommended is deliberately not used: a raw
FK error is not something the operator can act on.
`20260815160000_policy_type_repair` cleans up what already happened:
- restores M_EMPR and re-points its 5 policies, scoped to
`policyTypeId IS NULL AND legacySourceTable = 'm_empr'` so it can never
claim a policy blanked for some other reason
- merges the duplicate "ANA" carrier (1 policy) into "ANA SEGUROS" (738).
OCR is about to start assigning the carrier automatically and two rows
would keep splitting the book. Written as joins, not subqueries, so both
statements are no-ops when either row is absent -- a subquery form would
resolve to NULL and blank the carrier off every ANA policy.
- does NOT restore INCENDIO. It is the other row the migration would have
produced, but the legacy INCENDIO table has 1 row that never loaded, so
the type has zero policies and restoring it would only put a dead option
in the type picker.
Verified by running the repair against the real broken dev data inside a
transaction and rolling back: 5 orphans -> 0, ANA/ANA SEGUROS -> one row with
739, and a second run in the same transaction changes nothing. The DDL half
matches `prisma migrate diff` exactly.
186 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
d645ba51d3 |
feat(policy-ocr): read A.N.A. Seguros' two policy faces
A.N.A. is the Rosarito office's tourist auto book and the second carrier
the policy OCR pipeline reads. It ships two unrelated faces, and the split
is different from GMX's: GMX ships two documents about one policy, A.N.A.
ships two products.
AUTOMOBILE (SPECIAL POLICY FOR TOURISTS) insures a car; vehicle table,
9 numbered sections, one
LIMIT OF LIABILITY column
DRIVER'S POLICY (the office: "licencia") insures up to 5 named drivers;
no vehicle at all, 6 unnumbered
sections in a different order,
SUM INSURED + PREMIUM columns
The four automobile products the office sells (amplia / responsabilidad
civil, annual / by-the-day) are the same layout with different numbers, so
they get one parser rather than four.
These are born-digital portal PDFs, so pdftotext -layout returns exact
columns and the driver's-policy parser uses that: its two value columns
print the same shape (100,000.00 usd. / 18.70 usd.) with no per-row label,
so horizontal position is the only thing separating them. The split comes
from the header's own offsets, not a constant, because they shift between
products; when it can't be read every amount is reported as a sum insured
and the reviewer is told, rather than half the premiums being filed as
coverage limits.
Three things the layout will punish a naive read for:
- Each PDF prints its face two or three times (ORIGINAL, AGENT COPY, then
a receipt and three travel cards) and the pipeline concatenates every
page before parsing. The coverage walk is bounded to the first copy and
the driver list to the first POLICY HOLDER block. Unbounded, the licencia
returns the same person three times, which reads as a three-driver policy
rather than as a bug.
- The money row is read positionally off its header. An unused DISCOUNT
prints as a bare "-", so "find the six amounts" shifts every value one
column left on a discounted policy.
- Two five-digit numbers sit in the header band and only one is the agent
clave; the agent's street address is "BENITO JUAREZ 25 No.50 INT 38",
three lines above the No. cell holding the policy number.
Sections 6-8 print a PREMIUM where the others print a limit, so
ParsedCoverage gains an optional `premium` (GMX never fills it) and the
review table a column: $40 is what legal aid cost, not a $40 liability
limit. Exclusions follow the GMX rule and go in the risk label with a null
amount -- which matters more here, since a responsabilidad-civil policy
prints 0.00 for material damage and the two are identical on the page.
Also in this change:
- coveragePeriodDays is parsed and written. A.N.A. sells 3- and 4-day
policies; Policy.coveragePeriodDays defaults to 365, so a weekend policy
left at the default sits in the renewals window a year out. Derived from
the dates, cross-checked against the printed DAYS cell, disagreement
noted not resolved.
- Vehicles and named drivers are parsed, shown read-only in review, and
written as Vehicle / InsuredDriver rows on confirm, skipping any already
on the policy (VIN then plate; licence then name). The case that forces
the skip is confirming a renewal onto an existing policy. Nothing is ever
updated or deleted -- a changed plate lands as a second row for a human.
- Batch.provider is set from what the parsers actually claimed instead of
being hardcoded "GMX", so a mixed upload is labelled as mixed and the
header can never contradict its own documents. PolicyDocument.documentType
follows the same rule (was hardcoded GMX_POLICY).
- matchNote becomes TEXT. It was VARCHAR(191) and the note trail was sliced
to 190 chars, which cut the tail notes -- the "could not read X" ones.
- The policy detail page renders an array coveragesJson as a table. Both
shapes have always been possible there, but the object renderer was the
only one, so an OCR-confirmed policy showed a row per array index labelled
"0", "1", "2" with [object Object] as the value. ANA makes that routine.
GMX is untouched behaviourally; its two parsers now spread a shared empty
base instead of listing every null field. 29 new parser cases against
verbatim pdftotext output of three real ANA PDFs, 53 in the suite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
17d83291c3 |
feat(migration): refuse a full re-import that would delete native rows
A full run_all.py pass truncates and rebuilds every table it owns from the Access extract. That was harmless while the platform was a read-only mirror -- every row came from the extract, so wiping and rebuilding lost nothing. It stopped being harmless once the platform started minting rows Access has never heard of: allocated portal NUMids, customers created in the staff UI, OCR-captured policies, app-booked ledger rows, uploaded documents. REIMPORT is a button in /operaciones, so that was one click away. native_guard.py counts what only exists here and exits 3; run_all.py runs it before the first truncate and stops. Detecting an allocated NUMid needs the staged Parquet -- the customer holds an ordinary-looking (utilities, DATGRAL, '1172') ref, so "customer has no refs" cannot see it and only comparing against the extract can. Missing staging is therefore treated as blocking rather than as "nothing to protect". The guard does not teach full mode to preserve anything: --sync already upserts legacy rows against the existing refs and leaves the rest alone, and rebuilding that inside full mode would re-implement it. --force-full (checkbox in the REIMPORT confirm, recorded in the audit log) deletes them deliberately. Verified against dev: clean before, exit 3 listing utilities/1172 with a synthetic ref present, clean again after removing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
6a97242fc3 |
feat(customers): allocate portal NUMids, with an audit for reusable ones
Customers created in the staff UI had no NUMid and so could not log in to my.jorgecuadros.com at all: the id is a CustomerLegacyRef row, not a column, and create() deliberately writes none. Allocation is a staff action (POST /customers/:id/portal-access, MANAGER) rather than part of create, because insurance is expected to move to the platform before utilities and an insurance-only customer has no reason to spend a utilities id. The audit that decides which ids are reusable took three passes. "Owns no rows" matches nobody -- migration gave all 1,171 NUMids a property and a transaction. "No transaction in N years" also matches nobody -- every customer carries a synthetic Jan-1 opening-balance row, so everyone looks active this year. Subtracting that row is what makes dormancy measurable, and it leaves 4 never-used ids and 10 dormant ones on dev. Two further traps are encoded in the queries: insurance/DATGRAL is a separate id space that reuses the sourceTable name and runs past 4,000, and ACCOUNT CANCELED is a transaction line type, not an account state -- all 8 customers carrying it have current-year activity. Recycling ships switched off (numid.recycleEmpty, default false). Every reusable id still exists in Access DATGRAL, and a --sync run reassigns refs with ON DUPLICATE KEY UPDATE customerId, so an id recycled before the utilities cutover is silently handed back to its Access owner. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
e77e5546d8 |
docs: SES secrets created, ship blocker cleared
Five documents asserted the SES_* secrets were unset in Gitea. They now exist, so all five are corrected rather than leaving the claim to rot in whichever one a reader opens first. Replaces the blocker with the two things creating the secrets does NOT establish, since both fail in ways that look identical to a missing config: SES_FROM must be a verified identity in SES_REGION, and the account must be out of the SES sandbox — in sandbox SES only delivers to verified recipients, so a sweep across 815 policyholders would fail almost every send while the configuration reads as correct. Recommends running the first sweep with debug on, which diverts every recipient and, on the pólizas side, leaves the avisos pending so a failed test consumes nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
3e12597204 |
docs: add BACKLOG.md, one list of everything outstanding
Open work was spread across six documents: PLAN's per-step status, RESUME §6, two specs' collected open questions, and the "Not built" sections of the two OCR docs. Nothing tracked the two live data defects except a paragraph inside INSURANCE_FEATURES_SPEC, and nothing at all recorded that master is 14 commits and 5 migrations past the last tag. Compiled by reading those six, then checking each claim against the code and the dev database rather than trusting the prose — which is how the dead-table finding surfaced and how both insurance defects were confirmed still open. Leads with the ship blocker: SES_* is unset in Gitea while the pólizas sweep defaults to enabled at 06:00, so deploying current master gives a nightly sweep that fails every run. Set the secrets or disable the schedule before cutting v1.0.7. Findings not previously written down anywhere: - policy_types still holds only AUTO/LICENCIAS/MULT and 5 policies still have a NULL policyTypeId; policyTypeId is still `String?` with Prisma's default SetNull, so the spec's recommended Restrict was never applied. - EmailTemplate / EmailCampaign / EmailLog have zero references in apps/api/src or apps/web/src. Scaffolded for step 10's "email campaigns"; notificaciones shipped against email_notification_log instead. Either wire them or drop them. - Customer.customerNumber does not exist, so recycling is not merely unbuilt but unstarted at the schema level. Linked from PLAN.md and README so it is findable from either entry point. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |