Compare commits

...
28 Commits
Author SHA1 Message Date
gitea-actions f7507f2370 chore(release): v1.0.26
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m50s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m26s
Deploy on tag / Deploy to galactus (push) Successful in 38s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.26.
2026-08-20 02:44:12 +00:00
rmancinasandClaude Opus 5 2f9a9afc0d fix(billing): the cash receipt book is not a second ledger
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m49s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m28s
EFECTIVO is a journal, not a ledger. The office writes a numbered paper
receipt for money handed over the counter and then posts that same receipt
to the utilities ledger as reference `C<folio>`. Legacy summed the ledger
alone — ledger_repository.php reads `datosfreak`, materialized from DATOS2
only — but the migration flattened both tables into one `transactions`
table, so every balance counted each counter payment twice.

Confirmed against the live legacy database rather than inferred: of the 297
receipts written in 2026, 296 carry a matching DATOS2 posting. Six of them
post converted to pesos under a mistyped folio, which is why matching pairs
on folio and amount found fewer duplicates than exist — and why this
excludes the whole journal instead of a list of confirmed pairs. Only folio
13536 (CL 717, $400 USD) has no posting anywhere; that one wants a human.

The database qualifier is load-bearing. `SEGUROS 16_be` keeps its own table
also called EFECTIVO, and that one is the insurance line's only ledger —
nothing posts it anywhere else. Excluding by table name alone would erase
55,444.95 USD and 63,957.78 MXN across 102 customers, 99 of whom have no
other rows at all. Extending the qualified rule to the statement and the
customer file also gives those 99 back a statement that is not empty.

The same queries were missing the archive window the statement already had,
so the worklist and the book also counted a closed year twice for customers
floored inside an archive.

Measured on production, utilities MXN: NUMid 6 and 173 unchanged to the
cent, 501 unchanged at -10,874.33 (still the portal's number), 10 drops
2,362.20 -> -1,137.80 (exactly the 3,500.00 duplicate), 295 drops
14,377.46 -> 4,377.46 — which is what his statement already said. The
worklist and the statement now agree, which is the point.

Left alone deliberately: the movement browser, which is inventory rather
than balance and should still show what was captured; and stats()'s
outstanding rows, which turn on a client decision that is still open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 18:36:49 -07:00
gitea-actions 7e71a993d0 chore(release): v1.0.25
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m0s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m40s
Deploy on tag / Deploy to galactus (push) Successful in 1m9s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.25.
2026-08-20 00:31:07 +00:00
rmancinasandClaude Opus 5 aa5867c8ea fix(statements): an archive is history below the year start, not nothing
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m49s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m20s
c6feae9 excluded imported periods from the current period outright. That is
right for the rows an archive spills into the following January — those already
sit inside the next year's BALANCE FORWARD, which is the sum of the whole
archive, so counting them again would double-book them and file a closed year's
row as current.

It is wrong for everything below the year start. Those rows are what `opening`
exists for, and for a customer whose newest BALANCE FORWARD lives *inside* an
archive they are the only carry there is: the corte skipped NUMid 295 in 2026,
so his floor is the archive's own January 1 and excluding it dropped his entire
2025 closing balance. His statement read 3,592.00 against a true 4,377.46.

So the rule is a window, not an exclusion: an archive row counts below the year
start and never at or above it. Applied identically to statement(), the
edo-cuenta-datos report and the customer-file card, which have to agree.

Spelled as a positive OR rather than NOT(tag AND date). `NOT (col LIKE '...'
AND ...)` is NULL for a row with no legacySourceTable, so the negated form would
have silently dropped every app-captured movement — the same NULL trap the
source-table exclusion was already fixed for.

Verified against prod, which now carries datos2@2025: exactly one customer is
affected and the book moves by his 785.46. NUMid 501 is unchanged at -10,874.33
and still matches the portal; 6 and 173 are unchanged to the cent; the two
customers whose archives spill into January 2026 still keep those rows out of
the current period.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 17:29:08 -07:00
gitea-actions 5549a1e0cf chore(release): v1.0.24
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m52s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m4s
Deploy on tag / Deploy to galactus (push) Successful in 22s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.24.
2026-08-19 23:58:27 +00:00
rmancinasandClaude Opus 5 c6feae9522 fix(customers): the customer file's balances are the statement's balances
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m41s
Build and Push Images / Build jorgecuadros-api (push) Successful in 3m7s
The /clientes/:id ledger card is titled "Estado de cuenta" and links straight to
the statement, but its per-line tiles came from a groupBy with `voidedAt: null`
and nothing else — no balance floor, no source exclusion, no outstanding rule.
It was a raw lifetime sum, double-counting the pre-cutover history that each
BALANCE FORWARD row already absorbs, and the card said so in its own footnote
rather than being fixed.

Importing prior periods turned that from wrong into badly wrong. Every closed
year is now also held as its own tagged copy, so an unfloored sum adds each one
a second time on top of the opening balance that contains it. NUMid 501 read
-7,119.29 before the archives landed and -15,270.59 after, against a true
-10,715.29 — the gap being exactly the 2024 and 2025 closing balances.

The tiles now take the same three rules the statement takes, and agree with it
for all 600 customers sampled.

Two places needed the archives excluded explicitly, because the balance floor
does not do it:

  - A customer whose newest BALANCE FORWARD lives inside an archive floors at
    that archive's own January 1, so every row of it clears the floor. One
    customer, 28 rows.
  - The archives are not cleanly bounded. datos2@2024 carries rows dated 2022,
    2023, 2025 and one in 2026; datos2@2025 two more. Those clear any floor and
    land in the current year next to the live ledger's own copy of them.

That second point is a defect in 93f8171, not only in this card: statement()
and the edo-cuenta-datos report were both filtering the current period by
source and date without excluding the period tags, so four customers on the
dev book would have read a closed year's rows as current. Both are fixed here.
The portal already had it right.

The year's movement list on the card keeps showing every non-archive row, as
before — it is a list of what happened, not a balance.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 16:19:06 -07:00
rmancinasandClaude Opus 5 93f817158e feat(statements): a year selector, reading each closed year from its archive
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m49s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m4s
The statement has been pinned to the calendar year in progress since bc74905.
Now that prior years are imported, the year becomes a choice: the current one
still reads the live ledger, and any earlier one reads that year's archive.

A period is selected by its `datos2@YYYY` tag, not by a date range. That is how
legacy addressed it — one table per closed year, `SELECT ... FROM `2025`` — and
the distinction is load-bearing: the archives carry rows dated a day or two into
the following January, so a date window would file them under the wrong year in
one direction and drop them in the other.

Two things the archive branch must not inherit:

  - The balance floor. It exists to stop a later opening balance double-counting
    the history it summarizes; for a period view that history is precisely what
    is being asked for, so applying it would return nothing at all.
  - The cash-source exclusion. It reproduces legacy's DATOS2-only datosfreak,
    and an archive is DATOS2 rows already.

No fold into an opening balance either — the archive holds its own Jan-1 BALANCE
FORWARD row, which is the carry, listed exactly as legacy listed it.

The current period stays deliberately open-ended at the top. A period is a table
in legacy, not a date range, so whatever the office filed in it belongs to it,
including the future-dated rows the live ledger carries out to 2028. Bounding it
would hide them from every view.

`availableYears` reports the periods a customer actually has, so the picker never
offers a year that would render empty — "you had no activity in 2019" is a
different claim from "2019 was never imported", and only one of them is true.
The selector hides itself entirely for a customer with a single period, and a
year outside the list is a 404 rather than a silent fall back to the current one.

The same period rule lands on the printable twin (edo-cuenta-datos gains a
"Periodo (año)" parameter) and on the portal, where fetchLedgerRowsPlatform was
also filtering by date with no source exclusion at all — so period=2025 would
have returned the archive rows on top of that year's EFECTIVO receipts, counting
every prior-year payment twice. The portal's allowlist is now built per data
source and validated at the point of use: DreamHost holds the current year plus
one archive table, the platform holds however many were imported, and
fetchLedgerRowsLegacy interpolates the period as a table name, so a
platform-only year must not reach it — including on the fallback path when the
platform is unreachable.

Left alone: the /clientes/:id ledger card still shows the current year. It reads
transactionYear off the customers endpoint rather than the statement, it is a
summary that links to the full statement, and giving it its own year state would
duplicate the page it links to.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 15:48:25 -07:00
rmancinasandClaude Opus 5 29ae9fa5bc feat(migration): import prior periods so a closed year can be shown on its own
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m43s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m19s
Legacy ran a year-end corte: it summed the closing year, wrote that total back
as each customer's Jan-1 BALANCE FORWARD, and started the next year clean. The
platform inherited those opening rows but never the years behind them — only
the current year's charges were ever staged, so every prior year held receipts
and no bills. Rendering one would have shown a customer credits with nothing
owed against them, which is worse than showing nothing.

The archives are whole-database Access snapshots named for the period they
hold, so `2025.accdb` is discovered by filename, staged, and loaded through the
existing DATOS2 branch — a snapshot's `datos2` is the identical shape, one year
older. Only the ledger and DATGRAL come out of a snapshot; everything else in
it is a year-stale copy of a live table. The cash side is deliberately left
behind: EFECTIVO is a lifetime journal, so the snapshot's copy is a subset of
the live one and importing it would double-book every prior-year receipt.

Period travels with the row, in legacySourceTable as `datos2@2025`. That tag,
not the date, is what a year view should filter on: the archives are not
cleanly bounded (2025 carries ten undated rows and two dated into 2026) and
legacy never filtered by date either — its reader is `SELECT ... FROM \`2025\``.
The tag also keeps legacyId safe, since it is a positional ordinal that
restarts at 0 in every archive and would otherwise collide row-for-row.

Two guards, because attaching a prior year by NUMid is the one thing here that
can go quietly wrong:

  - Reissued numbers are skipped, not imported. Comparing each archive's
    DATGRAL against the live one, 13 names moved since 2025 and 40 since 2024;
    most are the same customer re-described, but a few are a different
    household holding a recycled number, and filing their ledger under the new
    owner would show a stranger's charges. Sharing any word of three or more
    characters separates a rename from a reissue. Names are compared
    legacy-to-legacy: `customers.name` has been through blank-name recovery,
    and comparing to it reported 121 drifts where there are 13.
  - Every period is checked against the corte identity it must satisfy —
    SUM(year N) == BALANCE FORWARD(N+1) — and the result is reported per year.
    A truncated export, a file dropped under the wrong year, or a botched
    customer match all fail loudly here. 2025 reconciles 1,159/1,167 (99.3%)
    and 2024 1,144/1,156 (99.0%); the recycle guard raised 2024 from 98.2%.

Balances are untouched: BALANCE_FLOOR_JOIN floors on the newest BALANCE FORWARD
per customer, so rows behind it are already excluded from every balance read.

Uploads go through the existing ingest endpoint, allowlisted by an anchored
`AAAA.accdb` pattern that also keeps a caller-supplied name inside the ingest
directory. The Operaciones page grows an entry point for an archive that has no
row yet, reading the period off the chosen file's own name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 15:26:58 -07:00
rmancinasandClaude Opus 5 9973488330 fix(billing): folio typos hid two double-booked receipts from the audit
Matching the datos2 `CN` reference against EFECTIVO folio `N` finds pairs
only where both were keyed correctly. Jorge Cuadros Jr's account carries
`C13647` against EFECTIVO folio `13649` — same day, same 3,500.00 MXN, one
receipt — and POWERS carries `C135808` against `13508`. Folio matching alone
calls both accounts clean, which is exactly backwards: an account used as a
validator reporting a false negative is worse than no audit.

A second pass now sweeps the C-refs the first pass left orphaned, on
proximity alone (same customer, same three-day window), and both passes are
judged by the same money rules. The folio is demoted to a lead: it can be
wrong in either direction, so it never decides anything on its own.

278 confirmed pairs, up from 276 — 989,740.00 MXN and 88,392.00 USD on the
EFECTIVO side. Of the 97 orphaned C-refs only 3 had any EFECTIVO row nearby,
so the remaining 94 are datos2-only postings rather than misses.

Rejections rise to 3. RAMIREZ, SUSANA pairs 3,320.00 MXN against 18,000.00
the next day; that is a partial application, not a duplicate, and it needs a
human rather than a rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 23:35:53 -07:00
rmancinasandClaude Opus 5 75dcbc11b8 feat(billing): audit the two populations the balance floor misses
Read-only. Reports what a corte would touch without touching it.

The platform inherited legacy's BALANCE FORWARD rows (1,170, all dated
2026-01-01) but not the yearly process that produces them, and
BillingService floors balances per customer on those rows. Two populations
fall outside that floor, and they are unrelated defects that happen to
surface as the same symptom — a customer whose balance reads as a credit
the office does not owe.

A. 102 customers have no BALANCE FORWARD row, so their balance is a raw
   lifetime sum. Only the current-year charge ledger (datos2) was migrated;
   the per-year charge tables stayed in DreamHost. What survives before the
   cutover is the EFECTIVO cash journal, and it shows: 199 of their 201
   pre-cutover rows are credits. Flooring them at 2026-01-01 moves the book
   by -82,297.78 MXN and -52,020.20 USD. 91 of the 102 are left with no rows
   at all, so their balance becomes zero — an assertion, not a figure
   recovered from anywhere, which is why this script proposes and does not
   apply.

B. 276 cash receipts booked twice in 2026 — once in EFECTIVO under folio N,
   once in datos2 as reference CN — across 120 customers, 986,240.00 MXN and
   88,252.00 USD on the EFECTIVO side. Both rows sit after the floor so both
   count. The statement already hides them via
   STATEMENT_EXCLUDED_SOURCE_TABLES; the balances worklist, the movement
   browser and the /clientes/:id card do not.

The folio alone does not establish a pair — folios are reused. Every pair is
corroborated on money too: equal amounts when both legs share a currency, or
an implied USD->MXN rate inside the band exchange_rates observed that year.
Two of 278 folio matches fail that test and are reported apart rather than
counted, both same-day partial applications that need a human.

Matches the balance-forward row in both shapes, as numid.service.ts does.
Databases imported before the type was minted carry those rows with typeId
NULL, and name-only matching reports every customer as floorless on such a
copy — including the dev database, which the API itself currently reads as
+20,653,109.15 MXN against a floored -9,194.61.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 23:18:48 -07:00
rmancinasandClaude Opus 5 9929a9a3ac feat(customers): the customer file's estado de cuenta follows the same year rule
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m48s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m5s
`/clientes/:id` carries its own "Estado de cuenta" card, fed by
CustomersService.detail rather than by BillingService.statement, so the
previous commit left it reading the old way: the last 100 movements of all
time, newest first. Two pages, one label, two orders.

It now covers the current calendar year oldest-first, like the statement and
like the sheet the office prints. The `take: 100` is gone with it — capping a
descending list hid the oldest rows, but capping an ascending one would hide
the newest, and a single year is small (365 rows for the heaviest customer in
the book).

The per-domain totals above the list are untouched and still historical: they
are an unfloored groupBy, so they double-count every customer's opening
balance. That is a separate bug from this one; the card now says out loud that
those figures are lifetime, not this year's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 20:21:51 -07:00
gitea-actions 8b8de0fdca chore(release): v1.0.23
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m0s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m19s
Deploy on tag / Deploy to galactus (push) Successful in 29s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.23.
2026-08-19 03:10:16 +00:00
rmancinasandClaude Opus 5 bc749055e7 feat(statements): scope the estado de cuenta to the current year, oldest-first
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m11s
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m50s
The office's EDO CUENTA sheet has always been a *year* statement: a balance
forward line dated January 1st, then that year's movements in the order they
happened. Both of ours read the other way — every year the customer ever had,
newest first — so staff comparing the screen against the printed sheet were
reading two different documents.

Movements are now bounded to the calendar year and returned ascending, on the
screen (/estado-cuenta/[id]) and in the printable `edo-cuenta-datos` report
alike.

Earlier rows are dropped from the *list*, not from the arithmetic. The balance
floor normally lands on January 1st already, so for most customers nothing
extra is dropped at all; when it doesn't — a customer the last legacy publish
skipped, or one that never had an opening balance — the earlier rows are
folded into a carried balance and shown as a single "saldo anterior" line.
Discarding them instead would restart every balance at zero on January 1st and
nothing would throw; the numbers would just be wrong, which is how the
double-counting bug survived for years. `opening` is exposed per currency and
per business line so the totals still reconcile against the last running
balance printed.

Two things the report was missing on its own are fixed while it is being
touched, since it must agree with the screen to the peso:

  - it never applied the balance floor, so every pre-cutover row was counted
    twice — once inside the opening balance and once as itself;
  - its source-table exclusion used a bare `notIn`, and `NULL NOT IN (...)` is
    NULL rather than true, so every app-captured row (which has no
    legacySourceTable) silently vanished from the printout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 08:45:15 -07:00
rmancinasandClaude Opus 5 75e9f582b4 feat(policy-ocr): store the IVA A.N.A. already prints
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m16s
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m50s
The parser has been reading A.N.A.'s `TAX` cell since the ANA layout landed,
but `ParsedPolicy` had nowhere to put it, so the figure only ever reached a
review note and every OCR-confirmed policy was written with `tax` null — even
though the paper states it.

`TAX` now flows parser -> `extractedTax` -> `Policy.tax`, alongside the premium
fields beside it. GMX stays null: its certificate carries no premium at all, so
there is no tax on it either.

The other two cells stay out, for reasons worth keeping:

  - `LOCAL TAX` is a separate levy with no destination column, and summing it
    into `tax` would produce an IVA that no longer divides back to a rate —
    which is the whole reason to store the figure. It reads 0.00 on every
    policy seen so far; a non-zero one now raises a note saying the total will
    not reconcile, instead of quietly inflating the IVA.
  - `DISCOUNT` has no column and prints as a bare "-" when unused, which is
    what makes the row positional rather than "find six amounts".

`taxRate` is left null by confirm. A.N.A. prints the amount, not the rate, and
back-dividing it would mint a rate the document never stated; the capture form
resolves one from the line of business instead.

The review screen gains derecho de póliza next to the new IVA field. It was
already parsed and already written on confirm, but never shown — and an IVA
with no fee beside it leaves the reviewer unable to see why premium + fee + tax
equals the printed total.

The spec's assertion moved off the note and onto the field, plus a check that
the row reconciles: 298.61 + 30.00 at 8% is 26.29, totalling 354.90. That
agreement is what proves the positional mapping landed on the right cells
rather than merely on six numbers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 07:38:23 -07:00
rmancinasandClaude Opus 5 48e01ddd21 feat(policies): capture the full premium breakdown
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m19s
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m1s
The capture form only ever had prima neta, derecho de póliza and comisión.
The Access form it replaces has seven figures, and the four that were missing
are the ones that make a policy paid in installments add up.

Adds recargo, IVA, prima total and forma de pago to the policy header, the
same breakdown per installment, and a per-line-of-business IVA rate.

IVA and prima total are the only derived figures:

    base  = prima neta + recargo + derecho de póliza
    IVA   = round(base * tasa)
    total = base + IVA

The recargo is inside the taxable base. That is not a guess — policy 7006785
prints IVA 52.03 on 610.86 + 8.55 + 31.00, and leaving the recargo out gives
51.35, which matches nothing on the page. Both of its money rows are asserted
in premium.spec.ts. The recargo itself is never derived: the carrier quotes it,
so staff key it in, and the field is disabled on ANNUAL/SINGLE. Both derived
figures are stored rather than recomputed on read, and stay editable, because
the printed policy is the record of truth and a later rate change must not
silently restate what was issued.

The rate lives on PolicyType (seeded to 0.08, editable in Catálogos), which is
the legacy one-row IMPUESTOS / IMPUESTOS_AUTOS tables made configurable. The
rate applied is stamped on the policy so an old one reads back at its original
rate.

Per-installment, not two fixed slots on the header: a policy split into several
exhibiciones prices each payment separately — that is why the Access form drew
the money row twice — and a trimestral policy needs four, which the Access
layout could not hold.

Also fixes two losses in the ETL, which is how these went missing:

  - `forma_pago` was marked consumed by the coverage sweep and then never
    written to any column, so FORMA PAGO existed nowhere in the platform.
  - `recargo` and the whole second money row fell into `coveragesJson` as
    loose strings, mislabeled as coverage amounts.

transform_policies.py now writes all of it directly;
backfill_policy_premium_breakdown.py recovers it on a database that must not be
re-imported, and strips the migrated keys back out of coveragesJson. Both are
COALESCE-only, so a figure a human has corrected in the app wins.

IVA and TOTAL are NOT backfilled: they were unbound calculated controls on the
Access form, never columns, so there is nothing to recover and every migrated
policy reads null until it is edited.

The backfill warns on 5 annual policies that carry a non-zero recargo — a
contradiction that predates this change and is left for a human, not silently
corrected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 00:24:22 -07:00
gitea-actions 8c144fe8c4 chore(release): v1.0.22
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m6s
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m0s
Deploy on tag / Deploy to galactus (push) Successful in 9s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.22.
2026-08-15 20:06:28 +00:00
rmancinasandClaude Opus 5 4f2f064955 fix(web): OCR batch review returns to capture, not the policy list
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m1s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m3s
"Volver a pólizas" on /polizas/captura/[id] dropped the reviewer at
/polizas, so getting back to the batch queue meant navigating in again.
Points at /polizas/captura and reads "Volver a captura", matching the
statement review screen, which returns to /recibos.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 13:04:14 -07:00
rmancinasandClaude Opus 5 2f99bd5f98 fix(web): define the layout utilities the screens were already using
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m5s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m50s
The policy OCR review header read
"Para revisarPágina 1700489616· PAMELA DENISE WAGONERLICENCIASANA".

`.row`, `.stack`, `.tag`, `.page-sub` and `.state-warn` are used across the
app but no rule ever defined them. Without `display: flex` the `gap` those
call sites pass does nothing, and JSX drops the newline between sibling
elements, so the header's spans concatenated. `.tag` rendered as prose rather
than as a chip for the same reason.

Defined against the existing design tokens: `.tag` takes `.badge`'s shape,
`.state-warn` is the gold sibling of `.state-error`, and `p.page-sub` keeps
the block margin while the inline form drops it so a flex row still centres.

Also dropped the hand-rolled "· " separator and margin from the OCR header —
the flex gap does that now, and the literal dot was left floating in it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 12:58:25 -07:00
gitea-actions 458b2b272d chore(release): v1.0.21
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m13s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m22s
Deploy on tag / Deploy to galactus (push) Successful in 8s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.21.
2026-08-15 19:41:02 +00:00
rmancinasandClaude Opus 5 81938877ed feat(policy-ocr): suggest the customer from the printed insured name
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m57s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m5s
The office books customers surname-first ("WAGONER, PAMELA") and carriers
print them given-name-first ("PAMELA DENISE WAGONER"), so the review screen
made staff retype a name the parser had already read. Comparing normalized
token sets makes the two orderings the same thing.

Only on the zero-hit path, where the policy number found nothing and a human
has to pick a customer anyway. The suggestions are written to a new
`customerSuggestions` column rather than `matchCandidates`, which the review
screen reads as policy-number hits, and they never set `matchedCustomerId` or
`confident` — matching on `Policy.policyNumber` is unchanged.

Two tiers, drawn where the real book has cliffs: EXACT (identical token sets)
and PARTIAL (containment, >=2 shared tokens, surname present). Of 1536
customers, 1487 have a distinct token set, so EXACT cross-person collisions
are ~0; loosen to surname + first given name and 131 (8.5%) collide, and 185
surnames are shared by 524 customers, which is why one token is never enough
and the surname must be printed explicitly. Replaying every book row as a
carrier would print it: 97.9% top-ranked correct, 1.2% a different row, all
but two of those the same human on a duplicate or variant row.

Normalization folds accents (OCR's MUNOZ reaches the book's MUÑOZ), drops
initials, Spanish particles, JR/S.A. DE C.V., and any token with a digit —
ANA prints the phone hard against the name as `Ph.3102001538`. Names over 8
tokens or 80 characters are refused outright, because GMX's especificación
has no field labels and the parser has handed its whole first page over as
`insuredName`.

Not used for utility statements: there the registrant genuinely is not the
customer, so the same trick would be wrong rather than noisy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 12:37:23 -07:00
gitea-actions d854dff091 chore(release): v1.0.20
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m2s
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m3s
Deploy on tag / Deploy to galactus (push) Successful in 24s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.20.
2026-08-15 18:59:38 +00:00
rmancinasandClaude Opus 5 19864f16f2 fix(ocr): keep the printed layout when rebuilding text from word boxes
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m1s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m24s
The parsers are written against `pdftotext -layout`, and every policy-ocr
fixture is a verbatim excerpt of it. The runtime does not use it: it reads
`-bbox-layout` and rebuilds the page from word boxes, and that rebuild
collapsed all white space — no blank lines between blocks, one space
between columns. White space is the only thing marking a cell boundary on
these borderless forms, so the fixtures could not see any of it.

What it cost, on the GMX PVL especificación and the ANA driver's policy:

- `espectBlock` walks a wrapped cell until a blank line. With no blank
  line it ran to the end of the page, so the insured's name came back as
  the entire first page of the specification.
- `INSURED\s{2,}` and its siblings matched nothing, and the phone that
  shares the name cell rode along with it ("PAMELA DENISE WAGONER
  Ph.3102001538"), which matches no customer.
- `parseAnaDriverCoverages` splits SUM INSURED from PREMIUM by the
  header's own column offsets. Without offsets, every premium was filed
  as a sum insured.

So `toVisualRows` now emits a blank line where the reader sees one (a
vertical gap over 1.6 line heights — the two populations measure 0.3-1.1
and 2.1+, so the threshold sits in empty space) and pads each word to its
own column, using one space wherever words merely follow each other so
rounding drift cannot sprinkle false cell boundaries through prose.

Two independent guards, so neither failure can come back silently: the
ANA phone splits on a single space, and the especificación's cell walk is
capped at the one wrap the longest cell on that document actually uses.

Verified against the real PDFs: the especificación reads "EMMER .
KATHLEEN" with all 18 coverages named (they were "(sin nombre)"), and the
ten born-digital gas invoices parse byte-identically to before. The
scanned statements are untouched — they come through tesseract, not this
path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 11:56:23 -07:00
gitea-actions ca6432efc8 chore(release): v1.0.19
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m38s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m16s
Deploy on tag / Deploy to galactus (push) Successful in 37s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.19.
2026-08-15 08:33:01 +00:00
rmancinasandClaude Opus 5 022d1935ad feat(policy-ocr): set policyTypeId and insuranceProviderId on confirm
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m0s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m14s
The BACKLOG claimed this was blocked on incomplete `policy_types` rows.
Querying the dev database says otherwise: AUTO (1316 policies) and LICENCIAS
(306) are both live and healthy, so ANA's two faces were never blocked at
all. Three separate things had been conflated.

What the parser now emits is a NAME, not an id -- it is a pure function over
text and must not reach for the database:

  ANA AUTOMOBILE        -> AUTO
  ANA DRIVER'S POLICY   -> LICENCIAS
  GMX (both documents)  -> MULT

`resolveLookups()` turns that into a foreign key at confirm, and does the
same for the carrier off the parser's provider code. It resolves, never
creates: a missing `policy_types` row means a human deleted it, and silently
recreating it would undo that with no record. An explicit `policyTypeId` /
`insuranceProviderId` on the confirm payload always wins.

GMX is MULT rather than INCENDIO because the caratula's own header reads
"Multiple Policy / Home" and the especificación is "PVL Hogar" -- one product,
two artifacts. MULT is the live row carrying 769 of them; INCENDIO is fire-only
and no policy in the book has ever used it.

The parser's provider code is not the carrier's row name, so PROVIDER_ROW_NAME
maps ANA onto "ANA SEGUROS", which is where the office's 738 ANA policies
already are.

--- the actual defect underneath -----------------------------------------

`policies.policyTypeId`, `policies.insuranceProviderId` and
`claims.adjusterId` are all ON DELETE SET NULL, and the lookups screen deleted
unconditionally. So deleting a lookup row returned 200 and silently blanked
the field on every row referencing it -- no error, nothing in the UI. That is
how M_EMPR disappeared and left 5 policies with no ramo, found months later
only by querying.

All three deletes now refuse while the row is in use, naming it and the count
("El tipo de póliza «M_EMPR» está en uso por 5 póliza(s)"). The schema-level
`onDelete: Restrict` the spec once recommended is deliberately not used: a raw
FK error is not something the operator can act on.

`20260815160000_policy_type_repair` cleans up what already happened:

  - restores M_EMPR and re-points its 5 policies, scoped to
    `policyTypeId IS NULL AND legacySourceTable = 'm_empr'` so it can never
    claim a policy blanked for some other reason
  - merges the duplicate "ANA" carrier (1 policy) into "ANA SEGUROS" (738).
    OCR is about to start assigning the carrier automatically and two rows
    would keep splitting the book. Written as joins, not subqueries, so both
    statements are no-ops when either row is absent -- a subquery form would
    resolve to NULL and blank the carrier off every ANA policy.
  - does NOT restore INCENDIO. It is the other row the migration would have
    produced, but the legacy INCENDIO table has 1 row that never loaded, so
    the type has zero policies and restoring it would only put a dead option
    in the type picker.

Verified by running the repair against the real broken dev data inside a
transaction and rolling back: 5 orphans -> 0, ANA/ANA SEGUROS -> one row with
739, and a second run in the same transaction changes nothing. The DDL half
matches `prisma migrate diff` exactly.

186 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 01:28:26 -07:00
rmancinasandClaude Opus 5 5a277f4885 feat(deploy): apply migrations at api container start
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m3s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m32s
`prisma migrate deploy` ran in one place only: a workflow step on the Gitea
runner, which has to reach the target host's MySQL on 3306 directly. Two
paths went around it:

  - `skip_migrate=true`, the documented answer for when the runner cannot
    reach 3306, left the schema a release behind with nothing to catch it.
    The mismatch surfaced later as a column-not-found at runtime rather than
    as a failed deploy.
  - A container brought back by `restart: unless-stopped` after a host
    reboot, or a stack re-applied by hand in Portainer, never runs the
    workflow at all.

docker/api-entrypoint.sh becomes the api image's ENTRYPOINT: migrate, then
exec node. If the migration fails the container exits non-zero and the API
never listens — serving against a schema that does not match the code is
worse than being down, because the failures are partial and silent (a write
to a missing column breaks one feature while the rest looks healthy).

This does not replace the workflow step and is not a substitute for it. That
step still runs FIRST, while the old code is serving, which is the order
expand/contract migrations are designed around. `migrate deploy` is
idempotent, so on the normal path the container's run is a no-op query.

Behaviour:

  RUN_MIGRATIONS=false     skip and start anyway; plumbed through both app
                           stack files, for a schema moved by hand
  DATABASE_URL unset       refuse to start, and say why
  P1001 (unreachable)      retry, default 20 x 3s -- a cold db container, and
                           galactus's MagicDNS lookup right after a reboot
  anything else            exit at once; retrying a broken migration only
                           delays the same error. P3005 prints the
                           `migrate resolve --applied 0000_init` hint the
                           workflow step already printed.

Only P1001 retries, so a genuinely broken migration is not buried under a
minute of noise.

Both stacks are replicas: 1 and must stay so for an unrelated reason (the
servicios email sweep has no DB lock). The old comment claiming migrations
must not run per-container because "N replicas would race" is dropped: they
would not corrupt anything, since Prisma takes a database advisory lock and
the losers find nothing pending -- they would only each pay the wait.

The prisma CLI is already in the runtime layer (the image copies
/repo/node_modules wholesale), but which of the two plausible .bin paths
carries it is an implementation detail of pnpm's hoisted linker, so the
entrypoint accepts either and the Dockerfile asserts one exists at BUILD
time. A missing CLI breaks the image build, not a production boot.

Verified by running the entrypoint against stubbed prisma binaries: clean
run, P3005, P1001-to-exhaustion, P1001-then-recovery, RUN_MIGRATIONS=false,
missing DATABASE_URL, missing CLI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 01:17:01 -07:00
rmancinasandClaude Opus 5 d645ba51d3 feat(policy-ocr): read A.N.A. Seguros' two policy faces
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m39s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m21s
A.N.A. is the Rosarito office's tourist auto book and the second carrier
the policy OCR pipeline reads. It ships two unrelated faces, and the split
is different from GMX's: GMX ships two documents about one policy, A.N.A.
ships two products.

  AUTOMOBILE (SPECIAL POLICY FOR TOURISTS)  insures a car; vehicle table,
                                            9 numbered sections, one
                                            LIMIT OF LIABILITY column
  DRIVER'S POLICY (the office: "licencia")  insures up to 5 named drivers;
                                            no vehicle at all, 6 unnumbered
                                            sections in a different order,
                                            SUM INSURED + PREMIUM columns

The four automobile products the office sells (amplia / responsabilidad
civil, annual / by-the-day) are the same layout with different numbers, so
they get one parser rather than four.

These are born-digital portal PDFs, so pdftotext -layout returns exact
columns and the driver's-policy parser uses that: its two value columns
print the same shape (100,000.00 usd. / 18.70 usd.) with no per-row label,
so horizontal position is the only thing separating them. The split comes
from the header's own offsets, not a constant, because they shift between
products; when it can't be read every amount is reported as a sum insured
and the reviewer is told, rather than half the premiums being filed as
coverage limits.

Three things the layout will punish a naive read for:

- Each PDF prints its face two or three times (ORIGINAL, AGENT COPY, then
  a receipt and three travel cards) and the pipeline concatenates every
  page before parsing. The coverage walk is bounded to the first copy and
  the driver list to the first POLICY HOLDER block. Unbounded, the licencia
  returns the same person three times, which reads as a three-driver policy
  rather than as a bug.
- The money row is read positionally off its header. An unused DISCOUNT
  prints as a bare "-", so "find the six amounts" shifts every value one
  column left on a discounted policy.
- Two five-digit numbers sit in the header band and only one is the agent
  clave; the agent's street address is "BENITO JUAREZ 25 No.50 INT 38",
  three lines above the No. cell holding the policy number.

Sections 6-8 print a PREMIUM where the others print a limit, so
ParsedCoverage gains an optional `premium` (GMX never fills it) and the
review table a column: $40 is what legal aid cost, not a $40 liability
limit. Exclusions follow the GMX rule and go in the risk label with a null
amount -- which matters more here, since a responsabilidad-civil policy
prints 0.00 for material damage and the two are identical on the page.

Also in this change:

- coveragePeriodDays is parsed and written. A.N.A. sells 3- and 4-day
  policies; Policy.coveragePeriodDays defaults to 365, so a weekend policy
  left at the default sits in the renewals window a year out. Derived from
  the dates, cross-checked against the printed DAYS cell, disagreement
  noted not resolved.
- Vehicles and named drivers are parsed, shown read-only in review, and
  written as Vehicle / InsuredDriver rows on confirm, skipping any already
  on the policy (VIN then plate; licence then name). The case that forces
  the skip is confirming a renewal onto an existing policy. Nothing is ever
  updated or deleted -- a changed plate lands as a second row for a human.
- Batch.provider is set from what the parsers actually claimed instead of
  being hardcoded "GMX", so a mixed upload is labelled as mixed and the
  header can never contradict its own documents. PolicyDocument.documentType
  follows the same rule (was hardcoded GMX_POLICY).
- matchNote becomes TEXT. It was VARCHAR(191) and the note trail was sliced
  to 190 chars, which cut the tail notes -- the "could not read X" ones.
- The policy detail page renders an array coveragesJson as a table. Both
  shapes have always been possible there, but the object renderer was the
  only one, so an OCR-confirmed policy showed a row per array index labelled
  "0", "1", "2" with [object Object] as the value. ANA makes that routine.

GMX is untouched behaviourally; its two parsers now spread a shared empty
base instead of listing every null field. 29 new parser cases against
verbatim pdftotext output of three real ANA PDFs, 53 in the suite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 01:09:16 -07:00
rmancinasandClaude Opus 5 14c4d44acb docs(policy-ocr): vigencia/agente/prima are keyed in by hand on the PVL layout
Build and Push Images / Build jorgecuadros-web (push) Successful in 3m4s
Build and Push Images / Build jorgecuadros-api (push) Successful in 3m14s
Confirmed with Luz, who handles GMX policies at the office: the three
fields the especificación does not carry are entered manually. The review
screen already supports it — all three are editable and `postPremium`
enables off the typed premium, so no code change was needed.

The parser's note said "esos datos están en la carátula de la póliza",
which now sends the reviewer looking for the wrong document. It says
"captúrelos a mano" instead, and names the consequence of leaving the
vigencia blank: `Policy.policyTo` is nullable and the renewals window
filters `policyTo: { gte, lte }`, so a policy confirmed without one never
matches and never gets a renewal notice — silently, permanently, with
nothing downstream erroring.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 00:20:09 -07:00
rmancinasandClaude Opus 5 45be0ad77d feat(policy-ocr): read GMX's Spanish PVL especificación layout
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m16s
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m16s
GMX ships two unrelated documents for the same policy and the office
downloads both from the same portal. The parser only knew the English
caratula, so a `…-CondicionesParticulares.pdf` parsed to an almost
entirely empty row — including the policy number, which the matcher needs.

`parseGmx` becomes a dispatcher over `parseGmxCaratula` (unchanged
behaviour) and the new `parseGmxEspecificacion`. Both still report
`provider: "GMX"`: the matcher keys on the policy number alone and must
not care which artifact was uploaded.

The especificación has no tables. Coverages are found by anchoring on
`Límite … Responsabilidad:` and walking backwards for the heading, where a
heading is a short line *preceded by a blank line* — length alone cannot
tell one from the wrapped tail of the paragraph above it, and without that
condition coverages get named after the last word of the preceding prose.

Also fixed, both pre-existing:

- The policy number's group widths are not the same across the two
  families (`007-037-…-0000-02` vs `07-037-…-00000-01`). The pinned-width
  regex is replaced by a shape, so both read.
- The caratula's ZIP fallback pushed a note saying it had read the ZIP
  from the address, then never assigned it.

Verified against the full ten-page real document: all 17 coverages,
amounts, deductibles and the excluded earthquake section match what is
printed. 24 parser tests (was 8), four of them regressions for ways this
layout can silently attach the *wrong* value rather than none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 00:10:26 -07:00
70 changed files with 7548 additions and 344 deletions
+15 -5
View File
@@ -17,6 +17,12 @@
# 3. prisma migrate deploy forward-only. Prisma has no down-migrations; see
# docs/DEPLOY_AND_MIGRATIONS.md — expand/contract is
# the rule, the backup is the emergency lever.
# Done HERE so the schema moves while the OLD code is
# still serving. The api container ALSO migrates at
# start (docker/api-entrypoint.sh); `migrate deploy`
# is idempotent, so the second run is a no-op and the
# container is what covers a restart that never goes
# through this workflow at all.
# 4. app (api + web) the new images.
# 5. verify ask the running API what it actually is.
#
@@ -55,10 +61,10 @@
# uses until somebody saves them there
# These are NOT galactus-specific (no _GALACTUS suffix) — one SES identity
# serves every deployment.
# - The runner (which lives on cubex) must be able to reach BOTH
# galactus:9443 (Portainer) and galactus:3306 (MySQL, for migrate deploy).
# If it cannot reach 3306, run the migration by hand from a host that can
# and dispatch with skip_migrate=true.
# - The runner (which lives on cubex) must be able to reach galactus:9443
# (Portainer). It should also reach galactus:3306 for step 3, but that is
# no longer load-bearing: dispatch with skip_migrate=true and the api
# container applies the migrations itself at start.
# - ONE-TIME, on a database that predates migration history (i.e. one built
# with `prisma db push`): baseline it before the first run, or step 3 fails
# with P3005 "database schema is not empty":
@@ -88,7 +94,7 @@ on:
required: false
default: false
skip_migrate:
description: "Skip prisma migrate deploy (use when the runner cannot reach MySQL and you migrated by hand)"
description: "Skip the runner-side migrate step (safe: the api container migrates at start)"
type: boolean
required: false
default: false
@@ -237,6 +243,10 @@ jobs:
run: node deploy/scripts/pre-migrate-backup.mjs
# --- schema, forward-only ---------------------------------------------
# Belt to the container's braces: this runs while the OLD code is still
# serving, which is the order expand/contract is designed around. The
# api container repeats it at start for the paths this step cannot
# reach (skip_migrate, a host reboot, a stack re-applied by hand).
- name: Apply database migrations
if: ${{ github.event.inputs.skip_migrate != 'true' }}
env:
+7 -4
View File
@@ -12,6 +12,7 @@
# git tag v1.2.3 into image tag 1.2.3. Tag v1.2.3, dispatch 1.2.3.
#
# Order: db+minio (full only) -> pre-migrate backup -> prisma migrate deploy ->
# (the api container also migrates at start; see docker/api-entrypoint.sh)
# app -> verify the API reports the version you asked for. Rollback = dispatch
# an older tag; that rolls back CODE only, never the schema, which is why every
# schema change must be expand/contract. See docs/DEPLOY_AND_MIGRATIONS.md.
@@ -47,9 +48,11 @@
# # Database stack (full only)
# MYSQL_PASSWORD app-user password (matches DATABASE_URL)
# MYSQL_ROOT_PASSWORD mysql root password
# - the runner must reach BOTH Portainer (9443) and MySQL (3306) — the
# migration step connects to the database directly. If it cannot reach 3306,
# migrate by hand and dispatch with skip_migrate=true.
# - the runner must reach Portainer (9443). It should also reach MySQL (3306)
# for the migrate step, but that is no longer load-bearing: dispatch with
# skip_migrate=true and the api container applies the migrations itself at
# start (docker/api-entrypoint.sh). `migrate deploy` is idempotent, so the
# two never conflict.
# - ONE-TIME on a database built with `prisma db push` (i.e. every database
# that exists today): baseline it before the first run, or the migrate step
# fails with P3005 "database schema is not empty":
@@ -79,7 +82,7 @@ on:
required: false
default: false
skip_migrate:
description: "Skip prisma migrate deploy (use when the runner cannot reach MySQL and you migrated by hand)"
description: "Skip the runner-side migrate step (safe: the api container migrates at start)"
type: boolean
required: false
default: false
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@jorgecuadros/api",
"version": "1.0.18",
"version": "1.0.26",
"private": true,
"scripts": {
"build": "nest build",
+45 -11
View File
@@ -115,13 +115,26 @@ describe("balance floor", () => {
);
});
/**
* statement() issues two findMany calls: first the period discovery (which
* years this customer has an archive for), then the statement rows. Select
* the rows query by its shape so adding another lookup later moves nothing
* here — the previous version indexed call 0 and broke the moment period
* support landed.
*/
function rowsQuery(findMany: jest.Mock) {
const call = findMany.mock.calls.find((c) => c[0]?.orderBy);
if (!call) throw new Error("statement() issued no ordered rows query");
return call[0];
}
it("bounds the statement at the floor, inclusive", async () => {
const floor = new Date("2026-01-01T00:00:00Z");
const { service, findMany } = serviceWith(floor);
await service.statement("c1");
expect(findMany.mock.calls[0][0].where).toMatchObject({
expect(rowsQuery(findMany).where).toMatchObject({
customerId: "c1",
transactionDate: { gte: floor },
});
@@ -132,23 +145,44 @@ describe("balance floor", () => {
await service.statement("c1");
expect(findMany.mock.calls[0][0].where).not.toHaveProperty(
"transactionDate",
);
expect(rowsQuery(findMany).where).not.toHaveProperty("transactionDate");
});
it("keeps the source-table exclusion alongside the floor", async () => {
// The two guards answer different questions — one reproduces legacy's
// DATOS2-only materialization, the other drops superseded history — and
// dropping either one changes the customer's balance.
// The three guards answer different questions — one windows imported
// periods, one drops the cash receipt book the ledger already posts, the
// floor drops superseded history — and dropping any one of them changes
// the customer's balance.
const { service, findMany } = serviceWith(new Date("2026-01-01T00:00:00Z"));
await service.statement("c1");
const where = findMany.mock.calls[0][0].where;
expect(where.OR).toEqual([
{ legacySourceTable: null },
{ legacySourceTable: { notIn: expect.arrayContaining(["EFECTIVO"]) } },
const where = rowsQuery(findMany).where;
// Imported periods are windowed rather than excluded: history below the
// year start (the only carry a floored-by-archive customer has), never
// at or above it (those rows sit inside the next BALANCE FORWARD).
expect(where.AND).toEqual([
{
OR: [
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: "datos2@" } } },
{ transactionDate: { lt: expect.any(Date) } },
],
},
// The cash journal, qualified by the database it came from — the
// insurance line has its own EFECTIVO and that one is a real ledger.
{
OR: [
{ legacySourceDb: null },
{ legacySourceDb: { not: "UTILITIES" } },
{ legacySourceTable: null },
{
legacySourceTable: {
notIn: expect.arrayContaining(["EFECTIVO"]),
},
},
],
},
]);
});
});
+16 -3
View File
@@ -119,10 +119,23 @@ export class BillingController {
return this.billing.byCheck(n);
}
/** One customer's full statement across both business lines. */
/**
* One customer's statement across both business lines, for one period.
*
* `year` omitted means the current one. Any earlier year is served from its
* imported archive; the response carries `availableYears` so the caller can
* offer only the periods this customer actually has.
*/
@Get("customers/:id")
statement(@Param("id") id: string) {
return this.billing.statement(id);
statement(@Param("id") id: string, @Query("year") year?: string) {
let parsed: number | undefined;
if (year !== undefined && year !== "") {
parsed = Number(year);
if (!Number.isInteger(parsed)) {
throw new BadRequestException("year debe ser un año de cuatro dígitos");
}
}
return this.billing.statement(id, parsed);
}
/** Cross-customer movement browser. */
+306 -61
View File
@@ -210,18 +210,47 @@ export const BALANCE_FLOOR_JOIN = Prisma.sql`
export const NOT_SUPERSEDED = Prisma.sql`(bfloor.floorDate IS NULL OR t.transactionDate >= bfloor.floorDate)`;
/**
* Source tables excluded from the customer-facing statement.
* `legacySourceTable` of an imported prior period.
*
* The legacy portal's `datosfreak` table was materialized from DATOS2 only
* (`objects.json:1358`), so the customer's "current balance" never saw
* EFECTIVO / EFECTIVO FM3 / CHEQUE FM3 / EFECTIVO_BACKUP cash receipts, nor
* the IVA 2015 snapshot. The unified `transactions` table has all of them, so
* the statement must drop them to match the legacy number the customer has
* been quoted for years. The staff-facing balances worklist and movement
* browser keep them — they're real money, just tracked separately
* (FM3 = visa fee stream, EFECTIVO = cash receipt stream).
* A closed year arrives as its own Access snapshot and is tagged rather than
* dated (see migration/transform_transactions.py). The tag is what a period
* view filters on: the archives are not cleanly date-bounded — 2025 carries
* rows dated into 2026 — and legacy did not filter by date either, it selected
* `FROM \`2025\``. Filtering on provenance reproduces the legacy period exactly.
*/
const STATEMENT_EXCLUDED_SOURCE_TABLES: readonly string[] = [
export const periodSourceTable = (year: number) => `datos2@${year}`;
/** Matches any imported period tag, for discovering which years a customer has. */
export const PERIOD_TABLE_PREFIX = "datos2@";
/**
* The legacy cash receipt book — a journal, not a ledger.
*
* `EFECTIVO` is the office's numbered receipt pad: money is handed over the
* counter, a folio is written, and the same receipt is then *posted* to the
* utilities ledger (`DATOS2`) as reference `C<folio>`. Legacy summed the ledger
* alone — `ws/v2/lib/ledger_repository.php` reads `datosfreak`, which is
* materialized from DATOS2 only (`objects.json:1358`). The migration flattened
* both tables into one `transactions` table, so anything summing a customer's
* rows counts every cash receipt twice.
*
* Verified against the live legacy database on 2026-08-20: of the 297 receipts
* written in 2026, 296 carry a matching DATOS2 posting. Only folio 13536 (CL
* 717, $400 USD) has no posting anywhere, and it wants a human's eyes rather
* than a code change. Six receipts post converted to pesos under a mistyped
* folio, which is why matching on folio and amount found fewer duplicate pairs
* than actually exist — a reason to exclude the whole journal rather than to
* exclude a list of confirmed pairs.
*
* THE DATABASE QUALIFIER IS LOAD-BEARING. `SEGUROS 16_be` keeps its own table
* also called `EFECTIVO`, and that one is the insurance line's *only* ledger —
* nothing posts it anywhere else. Excluding by table name alone erases the
* whole insurance balance: 55,444.95 USD and 63,957.78 MXN across 102
* customers, 99 of whom have no other rows at all.
*/
export const CASH_JOURNAL_SOURCE_DB = "UTILITIES";
export const CASH_JOURNAL_SOURCE_TABLES: readonly string[] = [
"EFECTIVO",
"EFECTIVO_BACKUP",
"EFECTIVO FM3",
@@ -229,6 +258,53 @@ const STATEMENT_EXCLUDED_SOURCE_TABLES: readonly string[] = [
"IVA 2015",
];
/**
* Prisma form of the cash-journal exclusion.
*
* Spelled as a positive OR on purpose. `notIn` alone compiles to SQL `NOT IN`,
* and `NULL NOT IN (...)` is NULL rather than true, so every app-captured row
* (no `legacySourceTable`) would silently vanish. Same for the database test.
*/
export const notCashJournal = (): Prisma.TransactionWhereInput => ({
OR: [
{ legacySourceDb: null },
{ legacySourceDb: { not: CASH_JOURNAL_SOURCE_DB } },
{ legacySourceTable: null },
{ legacySourceTable: { notIn: [...CASH_JOURNAL_SOURCE_TABLES] } },
],
});
/** Raw-SQL form, for the aggregate queries that cannot use Prisma's builder. */
export const NOT_CASH_JOURNAL = Prisma.sql`(
t.legacySourceDb IS NULL
OR t.legacySourceDb <> ${CASH_JOURNAL_SOURCE_DB}
OR t.legacySourceTable IS NULL
OR t.legacySourceTable NOT IN (${Prisma.join([
...CASH_JOURNAL_SOURCE_TABLES,
])}))`;
/**
* Keeps an imported prior period out of the *current* period, NULL-safely.
*
* The balance floor does not settle the archives on its own, in both
* directions. A customer whose newest BALANCE FORWARD lives *inside* an archive
* floors at that archive's own January 1st, so every row of it clears the floor
* — and that is correct, because below the year start the archive is the only
* carry there is. At or above the year start it must go: the archives spill a
* couple of rows into the following January and those already sit inside the
* next year's BALANCE FORWARD, which is the sum of the whole archive.
*
* Spelled as a positive OR for the same NULL reason as above.
*/
export const archiveIsHistorySql = (yearStart: Date) => Prisma.sql`(
t.legacySourceTable IS NULL
OR t.legacySourceTable NOT LIKE ${`${PERIOD_TABLE_PREFIX}%`}
OR t.transactionDate < ${yearStart})`;
/** January 1st of the running year, UTC — the current period's lower bound. */
export const currentYearStart = () =>
new Date(Date.UTC(new Date().getUTCFullYear(), 0, 1));
@Injectable()
export class BillingService {
constructor(private readonly prisma: PrismaService) {}
@@ -398,6 +474,16 @@ export class BillingService {
async balances(params: BalanceParams) {
const { query, page, pageSize, currency, balance, domain, sort } = params;
// A balance is what the customer owes, so it takes the same rules the
// statement takes: the floor, the cash journal, and the archive window.
// Without the last two the worklist quoted a different number than the
// customer's own statement — NUMid 295 read 14,377.46 against a statement
// of 4,377.46, and NUMid 10 read 2,362.20 against -1,137.80, the gap in
// each case being a cash receipt already posted to the ledger.
const scope = Prisma.sql`AND ${NOT_CASH_JOURNAL} AND ${archiveIsHistorySql(
currentYearStart(),
)}`;
const filters: Prisma.Sql[] = [];
if (domain) filters.push(Prisma.sql`t.domain = ${domain}`);
const txFilter = filters.length
@@ -460,7 +546,7 @@ export class BillingService {
FROM customers c
JOIN transactions t ON t.customerId = c.id
${BALANCE_FLOOR_JOIN}
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND ${NOT_SUPERSEDED} ${nameFilter} ${txFilter}
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND ${NOT_SUPERSEDED} ${scope} ${nameFilter} ${txFilter}
GROUP BY c.id, c.name, c.nameSource, c.nameMissing, c.city, c.state
${having}
${orderBy}
@@ -476,7 +562,7 @@ export class BillingService {
-- Must match the page query's filters exactly, or the total disagrees
-- with the rows. (The void exclusion was missing here before the
-- outstanding work; a voided-only customer inflated the count.)
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND ${NOT_SUPERSEDED} ${nameFilter} ${txFilter}
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND ${NOT_SUPERSEDED} ${scope} ${nameFilter} ${txFilter}
GROUP BY c.id
${having}
) x
@@ -523,11 +609,22 @@ export class BillingService {
* Two different questions live here and they use different row sets.
* `movements`, `ledgerCustomers`, `crossLineCustomers` and the date range are
* INVENTORY — what is stored — and count everything not voided. Everything
* under `byCurrency` / `byDomain` is a BALANCE, so it applies NOT_SUPERSEDED
* and drops rows an opening balance already accounts for. The four aggregates
* moved from Prisma groupBy to raw SQL to express that join; groupBy cannot.
* under `byCurrency` / `byDomain` is a BALANCE, so it takes the same scope
* `balances()` takes — the opening-balance floor, the cash journal and the
* archive window — and the book has to agree with the worklist that sits
* under it. The four aggregates moved from Prisma groupBy to raw SQL to
* express that join; groupBy cannot.
*
* KNOWN DIVERGENCE, left deliberately: these three do not drop outstanding
* rows, while `balances()` does. Reconciling them moves the book by about
* 1.95M MXN and turns on whether an unfunded charge is owed by the customer,
* which is the client's call and not settled yet.
*/
async stats() {
const scope = Prisma.sql`AND ${NOT_CASH_JOURNAL} AND ${archiveIsHistorySql(
currentYearStart(),
)}`;
const [movements, ledgerCustomers] = await Promise.all([
this.prisma.transaction.count({ where: NOT_VOIDED }),
this.prisma.transaction
@@ -559,7 +656,7 @@ export class BillingService {
SUM(t.amount > 0) AS creditCount
FROM transactions t
${BALANCE_FLOOR_JOIN}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED} ${scope}
GROUP BY t.currency
`;
@@ -575,7 +672,7 @@ export class BillingService {
SUM(t.amount) AS net, COUNT(*) AS count
FROM transactions t
${BALANCE_FLOOR_JOIN}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED} ${scope}
GROUP BY t.domain, t.currency
`;
@@ -596,7 +693,7 @@ export class BillingService {
SELECT t.customerId, t.currency, SUM(t.amount) AS bal
FROM transactions t
${BALANCE_FLOOR_JOIN}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED} ${scope}
GROUP BY t.customerId, t.currency
) x
GROUP BY currency
@@ -701,13 +798,22 @@ export class BillingService {
/**
* One customer's statement across both business lines.
*
* Returns the *whole* ledger rather than a page of it: the heaviest customer
* Scoped to one calendar year and listed oldest-first, matching the legacy
* EDO CUENTA report the office has printed for years: an opening balance at
* the top, then the year's movements in the order they happened.
*
* `year` selects the period. The current year is read from the live tables;
* any earlier year is read from its imported archive, which legacy kept as a
* separate table and this reads by its `datos2@YYYY` tag. `availableYears`
* reports which periods this customer actually has, so a caller never offers
* a year that would render empty.
*
* Returns the *whole* year rather than a page of it: the heaviest customer
* carries 365 movements (mean 26), and a running balance is meaningless if
* the client only holds a slice. The running balance is accumulated per
* currency in chronological order, then the list is handed back newest-first
* with each row's balance-after already attached.
* currency in chronological order, with each row's balance-after attached.
*/
async statement(customerId: string) {
async statement(customerId: string, year?: number) {
const customer = await this.prisma.customer.findUnique({
where: { id: customerId },
select: {
@@ -731,6 +837,49 @@ export class BillingService {
throw new NotFoundException(`Customer ${customerId} not found`);
}
// Which periods this customer has. The current year is always offered —
// it is the live ledger even when empty — and each imported archive adds
// the year it holds.
const archives = await this.prisma.transaction.findMany({
where: {
customerId,
voidedAt: null,
legacySourceTable: { startsWith: PERIOD_TABLE_PREFIX },
},
distinct: ["legacySourceTable"],
select: { legacySourceTable: true },
});
const thisYear = new Date().getUTCFullYear();
const archiveYears = archives
.map((a) => Number(a.legacySourceTable?.slice(PERIOD_TABLE_PREFIX.length)))
.filter((y) => Number.isInteger(y) && y < thisYear);
const availableYears = [...new Set([thisYear, ...archiveYears])].sort(
(a, b) => b - a,
);
// An unknown year would silently render as the current one, which reads as
// "this customer had no activity in 2019" rather than "there is no 2019".
const requested = year ?? thisYear;
if (!availableYears.includes(requested)) {
throw new NotFoundException(
`El cliente no tiene movimientos del periodo ${requested}.`,
);
}
const isArchive = requested !== thisYear;
//
// Deliberately open-ended at the top. A period is a table in legacy, not a
// date range, so whatever the office filed in it belongs to it — including
// the future-dated rows the current ledger carries (it runs to 2028). An
// upper bound would hide them from every view, which is not what legacy did
// and not what the office has been reading.
//
// An archive needs no fold at all: it *is* the period, and its own Jan-1
// BALANCE FORWARD row is the carry, listed exactly as legacy listed it.
const yearStart = isArchive
? new Date(0)
: new Date(Date.UTC(requested, 0, 1));
// One customer, so the balance floor is a single date rather than the
// derived table the aggregate queries join. See NOT_SUPERSEDED: rows before
// the opening balance are already inside it, and showing them would both
@@ -756,21 +905,48 @@ export class BillingService {
const rows = await this.prisma.transaction.findMany({
where: {
customerId,
...(floor ? { transactionDate: { gte: floor.transactionDate } } : {}),
// NULL-safe exclusion. `notIn` alone compiles to SQL `NOT IN`, and
// `NULL NOT IN (...)` is NULL, not true — so every app-captured row
// (which has no legacySourceTable) silently vanished from the
// statement while still showing in the movement browser. Rows the app
// books must appear on the customer's statement, so the null case is
// spelled out.
OR: [
{ legacySourceTable: null },
{
legacySourceTable: {
notIn: STATEMENT_EXCLUDED_SOURCE_TABLES as string[],
},
},
],
...(isArchive
? // An archive is already exactly one period's ledger, so the tag is
// the whole filter. The balance floor is deliberately NOT applied:
// it exists to stop a later opening balance double-counting the
// history it summarizes, and here that history is the thing being
// asked for. The exclusion list is moot too — an archive holds only
// DATOS2 rows, which is what legacy's year table held.
{ legacySourceTable: periodSourceTable(requested) }
: {
...(floor ? { transactionDate: { gte: floor.transactionDate } } : {}),
// An archive row belongs to this period only as history. Below
// the year start it is exactly what `opening` is for, and for the
// one customer whose newest BALANCE FORWARD lives *inside* an
// archive it is the only carry there is — dropping it outright
// understated NUMid 295 by his whole 2025 closing balance, 785.46.
//
// At or above the year start it must go. The archives spill a
// couple of rows into the following January, and those are
// already inside the next year's BALANCE FORWARD (which is the
// sum of the whole archive), so listing them here would both
// double-count and file a closed year's row as current.
//
// Spelled as a positive OR because `NOT (col LIKE ... AND ...)`
// is NULL for an app-captured row, which would drop every one.
AND: [
{
OR: [
{ legacySourceTable: null },
{
legacySourceTable: {
not: { startsWith: PERIOD_TABLE_PREFIX },
},
},
{ transactionDate: { lt: yearStart } },
],
},
// The cash receipt book is the ledger's own postings written a
// second time, so listing it here would show every counter
// payment twice and double the credit side.
notCashJournal(),
],
}),
},
orderBy: [{ transactionDate: "asc" }, { id: "asc" }],
select: {
@@ -790,15 +966,50 @@ export class BillingService {
},
});
// The statement covers one calendar year. The floor above normally lands on
// January 1st of it already — the legacy publish writes one BALANCE FORWARD
// per customer per year — in which case nothing extra is dropped here. When
// it doesn't (a customer the last publish skipped, or one that never had an
// opening balance), the earlier rows still have to be *counted* or every
// balance below is wrong, so they are folded into `opening` rather than
// listed. That is the same thing a BALANCE FORWARD row does, just computed.
const running = new Map<string, Prisma.Decimal>();
const movements = rows.map((r) => {
/** Balance carried into `yearStart`, per currency. */
const opening = new Map<string, Prisma.Decimal>();
/** The same carried balance split by business line, keyed `domain|currency`. */
const openingByDomain = new Map<
string,
{ domain: TransactionDomain; currency: string; amount: Prisma.Decimal }
>();
/** The rows the statement lists — this year's. Totals are built from these. */
const visible: typeof rows = [];
const movements = rows.flatMap((r) => {
const voided = r.voidedAt != null;
const prev = running.get(r.currency) ?? new Prisma.Decimal(0);
// Neither a voided row nor an outstanding (unpaid) one moves the running
// balance — both show tagged, with the balance unchanged from the previous
// live movement. Outstanding rows start counting once resolved.
const next = voided || r.outstanding ? prev : prev.plus(r.amount);
const counted = !voided && !r.outstanding;
const next = counted ? prev.plus(r.amount) : prev;
running.set(r.currency, next);
if (r.transactionDate < yearStart) {
if (counted) {
opening.set(r.currency, next);
const dk = `${r.domain}|${r.currency}`;
const od = openingByDomain.get(dk) ?? {
domain: r.domain,
currency: r.currency,
amount: new Prisma.Decimal(0),
};
od.amount = od.amount.plus(r.amount);
openingByDomain.set(dk, od);
}
return [];
}
visible.push(r);
return {
id: r.id,
transactionDate: r.transactionDate,
@@ -818,7 +1029,6 @@ export class BillingService {
balanceAfter: next.toFixed(2),
};
});
movements.reverse();
// Per-currency summary, and the same split by business line so the two
// ledgers are visibly one statement without being illegally added up.
@@ -846,7 +1056,29 @@ export class BillingService {
}
>();
for (const r of rows) {
for (const [currency] of opening) {
perCurrency.set(currency, {
currency,
charges: new Prisma.Decimal(0),
credits: new Prisma.Decimal(0),
chargeCount: 0,
creditCount: 0,
count: 0,
first: null,
last: null,
});
}
for (const [key, o] of openingByDomain) {
perDomain.set(key, {
domain: o.domain,
currency: o.currency,
charges: new Prisma.Decimal(0),
credits: new Prisma.Decimal(0),
count: 0,
});
}
for (const r of visible) {
// Voided rows never enter a total; outstanding rows don't either until
// they're resolved (legacy SALDOS ULTIMO 0's `HAVING NOPAGO = 0`).
if (r.voidedAt != null || r.outstanding) continue;
@@ -896,7 +1128,7 @@ export class BillingService {
string,
{ name: string; currency: string; total: Prisma.Decimal; count: number }
>();
for (const r of rows) {
for (const r of visible) {
if (r.voidedAt != null || r.outstanding) continue;
if (!r.amount.lessThan(0)) continue;
const name = r.type?.nameEs || r.type?.nameEn || "Sin clasificar";
@@ -915,25 +1147,38 @@ export class BillingService {
propertyCount: customer._count.properties,
policyCount: customer._count.policies,
},
summary: [...perCurrency.values()].map((c) => ({
currency: c.currency,
charges: c.charges.toFixed(2),
credits: c.credits.toFixed(2),
balance: c.charges.plus(c.credits).toFixed(2),
chargeCount: c.chargeCount,
creditCount: c.creditCount,
count: c.count,
firstMovement: c.first,
lastMovement: c.last,
})),
byDomain: [...perDomain.values()].map((d) => ({
domain: d.domain,
currency: d.currency,
charges: d.charges.toFixed(2),
credits: d.credits.toFixed(2),
balance: d.charges.plus(d.credits).toFixed(2),
count: d.count,
})),
year: requested,
availableYears,
summary: [...perCurrency.values()].map((c) => {
const open = opening.get(c.currency) ?? new Prisma.Decimal(0);
return {
currency: c.currency,
/** Balance carried in from before this year — legacy's BALANCE FORWARD. */
opening: open.toFixed(2),
charges: c.charges.toFixed(2),
credits: c.credits.toFixed(2),
balance: open.plus(c.charges).plus(c.credits).toFixed(2),
chargeCount: c.chargeCount,
creditCount: c.creditCount,
count: c.count,
firstMovement: c.first,
lastMovement: c.last,
};
}),
byDomain: [...perDomain.values()].map((d) => {
const open =
openingByDomain.get(`${d.domain}|${d.currency}`)?.amount ??
new Prisma.Decimal(0);
return {
domain: d.domain,
currency: d.currency,
opening: open.toFixed(2),
charges: d.charges.toFixed(2),
credits: d.credits.toFixed(2),
balance: open.plus(d.charges).plus(d.credits).toFixed(2),
count: d.count,
};
}),
byType: [...byType.values()]
.map((t) => ({
name: t.name,
+77
View File
@@ -0,0 +1,77 @@
import { Prisma } from "@jorgecuadros/database";
import {
CASH_JOURNAL_SOURCE_DB,
CASH_JOURNAL_SOURCE_TABLES,
NOT_CASH_JOURNAL,
notCashJournal,
} from "./billing.service";
/**
* `EFECTIVO` is the office's paper receipt book, and every receipt in it is
* also posted to the utilities ledger as `C<folio>`. Both copies were imported
* into one `transactions` table, so a balance that reads the journal counts
* each counter payment twice — 1,094,347.78 MXN of phantom credit book-wide,
* and 3,500.00 of it on NUMid 10 alone.
*
* These fail silently in the worst way: the numbers stay plausible, they are
* just too generous to the customer. Two shapes of mistake are easy to make
* here and both are covered below — dropping the database qualifier (which
* erases the insurance line's only ledger) and writing the exclusion as a bare
* `NOT IN` (which erases every app-captured row).
*/
describe("cash journal exclusion", () => {
describe("raw SQL form", () => {
it("binds the source database rather than interpolating it", () => {
expect(NOT_CASH_JOURNAL.values).toContain(CASH_JOURNAL_SOURCE_DB);
});
it("qualifies the table names with the database they came from", () => {
// `SEGUROS 16_be` has its own EFECTIVO and it is the insurance line's
// ONLY ledger — nothing posts it anywhere else. Matching on the table
// name alone erases 55,444.95 USD and 63,957.78 MXN across 102 customers.
expect(NOT_CASH_JOURNAL.sql).toContain("t.legacySourceDb <>");
expect(NOT_CASH_JOURNAL.values).toContain(CASH_JOURNAL_SOURCE_DB);
});
it("spells both null cases out instead of relying on NOT IN", () => {
// `NULL NOT IN (...)` is NULL, not true. Without these branches every
// app-captured row — the ones staff key in by hand — drops out of the
// balance while still showing in the movement browser.
expect(NOT_CASH_JOURNAL.sql).toContain("t.legacySourceDb IS NULL");
expect(NOT_CASH_JOURNAL.sql).toContain("t.legacySourceTable IS NULL");
});
it("covers the whole cash family, not just EFECTIVO", () => {
for (const table of CASH_JOURNAL_SOURCE_TABLES) {
expect(NOT_CASH_JOURNAL.values).toContain(table);
}
});
it("is a single parenthesised term, safe to AND into a WHERE clause", () => {
// It is composed as `... AND ${NOT_CASH_JOURNAL} AND ...`. An unbracketed
// OR chain would swallow every condition after it and silently widen the
// whole query to the entire table.
const sql = NOT_CASH_JOURNAL.sql.trim();
expect(sql.startsWith("(")).toBe(true);
expect(sql.endsWith(")")).toBe(true);
});
});
describe("Prisma form", () => {
it("matches the raw form's terms so the two cannot drift apart", () => {
const branches = notCashJournal().OR as Prisma.TransactionWhereInput[];
expect(branches).toEqual([
{ legacySourceDb: null },
{ legacySourceDb: { not: CASH_JOURNAL_SOURCE_DB } },
{ legacySourceTable: null },
{ legacySourceTable: { notIn: [...CASH_JOURNAL_SOURCE_TABLES] } },
]);
});
it("returns a fresh object each call", () => {
// It is spread into `AND: [...]` arrays that Prisma may mutate; a shared
// singleton would leak one query's filters into the next.
expect(notCashJournal()).not.toBe(notCashJournal());
});
});
});
@@ -0,0 +1,36 @@
import { periodSourceTable } from "./billing.service";
/**
* A closed year is imported as its own tagged set of rows rather than being
* identified by date. The tag is written by migration/transform_transactions.py
* and read by BillingService.statement, the edo-cuenta-datos report, and the
* PHP portal — three places that must agree on the exact string.
*/
describe("periodSourceTable", () => {
it("names the archive the migration writes", () => {
expect(periodSourceTable(2025)).toBe("datos2@2025");
expect(periodSourceTable(2024)).toBe("datos2@2024");
});
it("stays distinct from the live ledger's own table", () => {
// The live table is plain `datos2`. legacyId is a positional ordinal that
// restarts at 0 in every archive, so a shared name would collide with the
// current year row-for-row on the unique key.
expect(periodSourceTable(2025)).not.toBe("datos2");
expect(periodSourceTable(2025).startsWith("datos2@")).toBe(true);
});
it("is not matched by the statement's cash-source exclusion list", () => {
// STATEMENT_EXCLUDED_SOURCE_TABLES drops the EFECTIVO family to reproduce
// legacy's DATOS2-only datosfreak. An archive holds DATOS2 rows, so it must
// survive that filter or a prior year renders empty.
const excluded = [
"EFECTIVO",
"EFECTIVO_BACKUP",
"EFECTIVO FM3",
"CHEQUE FM3",
"IVA 2015",
];
expect(excluded).not.toContain(periodSourceTable(2025));
});
});
+145
View File
@@ -0,0 +1,145 @@
import { Prisma } from "@jorgecuadros/database";
import { BillingService } from "./billing.service";
/**
* The statement is a *year* statement, like the EDO CUENTA report the office
* prints: this year's movements, oldest-first, opening on the balance carried
* in from before it.
*
* The carrying is the part worth testing. Dropping earlier rows from the list
* is easy; dropping them from the arithmetic too would restart every balance at
* zero on January 1st, and nothing would throw — the numbers would just be
* wrong, which is exactly how the double-counting bug lived for years.
*/
describe("statement year scoping", () => {
const YEAR = new Date().getUTCFullYear();
function d(iso: string) {
return new Date(`${iso}T00:00:00.000Z`);
}
type RowSpec = {
id: string;
date: Date;
amount: string;
currency?: string;
domain?: string;
voidedAt?: Date | null;
outstanding?: boolean;
};
function row(r: RowSpec) {
return {
id: r.id,
transactionDate: r.date,
domain: r.domain ?? "UTILITY",
amount: new Prisma.Decimal(r.amount),
currency: r.currency ?? "MXN",
reference: null,
period: null,
checkNumber: null,
message: null,
legacySourceTable: null,
voidedAt: r.voidedAt ?? null,
outstanding: r.outstanding ?? false,
type: { nameEn: "WATER", nameEs: "AGUA" },
};
}
/** No BALANCE FORWARD row, so the floor is null and every row is fetched. */
function serviceWith(rows: RowSpec[]) {
const prisma = {
customer: {
findUnique: jest.fn().mockResolvedValue({
id: "c1",
name: "CUADROS, JORGE H.",
preferredCurrency: "MXN",
_count: { properties: 0, policies: 0 },
}),
},
transaction: {
findFirst: jest.fn().mockResolvedValue(null),
findMany: jest.fn().mockResolvedValue(rows.map(row)),
},
};
return new BillingService(prisma as never);
}
it("lists the year's movements oldest-first", async () => {
const s = await serviceWith([
{ id: "a", date: d(`${YEAR}-01-02`), amount: "-100" },
{ id: "b", date: d(`${YEAR}-03-04`), amount: "250" },
{ id: "c", date: d(`${YEAR}-07-16`), amount: "-40" },
]).statement("c1");
expect(s.movements.map((m) => m.id)).toEqual(["a", "b", "c"]);
});
it("leaves earlier years off the list", async () => {
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 1}-11-30`), amount: "-500" },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-100" },
]).statement("c1");
expect(s.movements.map((m) => m.id)).toEqual(["new"]);
});
it("carries the earlier years' balance instead of discarding it", async () => {
// 1,000 credit left over from last year, 300 charged this year: the
// customer is 700 in credit, not 300 in debt.
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 1}-12-15`), amount: "1000" },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-300" },
]).statement("c1");
const mxn = s.summary.find((x) => x.currency === "MXN");
expect(mxn?.opening).toBe("1000.00");
expect(mxn?.charges).toBe("-300.00");
expect(mxn?.balance).toBe("700.00");
// The running balance on the listed row picks up where last year left off.
expect(s.movements[0].balanceAfter).toBe("700.00");
});
it("carries it per business line as well", async () => {
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 1}-12-15`), amount: "1000", domain: "INSURANCE" },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-300", domain: "INSURANCE" },
]).statement("c1");
const line = s.byDomain.find((x) => x.domain === "INSURANCE");
expect(line?.opening).toBe("1000.00");
expect(line?.balance).toBe("700.00");
});
it("still reports a currency that only moved in earlier years", async () => {
// Otherwise a customer sitting on a dollar credit they haven't touched all
// year would appear to have no dollar balance at all.
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 2}-05-01`), amount: "180.83", currency: "USD" },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-300" },
]).statement("c1");
const usd = s.summary.find((x) => x.currency === "USD");
expect(usd?.balance).toBe("180.83");
expect(usd?.count).toBe(0);
});
it("does not carry a voided earlier row", async () => {
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 1}-12-15`), amount: "1000", voidedAt: d(`${YEAR - 1}-12-16`) },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-300" },
]).statement("c1");
const mxn = s.summary.find((x) => x.currency === "MXN");
expect(mxn?.opening).toBe("0.00");
expect(mxn?.balance).toBe("-300.00");
});
it("reports the year it covers", async () => {
const s = await serviceWith([
{ id: "a", date: d(`${YEAR}-01-02`), amount: "-100" },
]).statement("c1");
expect(s.year).toBe(YEAR);
});
});
@@ -0,0 +1,122 @@
import { CustomersService } from "./customers.service";
import { BALANCE_FORWARD_TYPE } from "../billing/billing.service";
/**
* The /clientes/:id ledger card is titled "Estado de cuenta" and links straight
* to the statement, so its per-line totals must be the statement's numbers.
*
* They were a raw lifetime sum — no floor, no source exclusion — which
* double-counted the pre-cutover history each BALANCE FORWARD row absorbs.
* Importing prior periods made it visibly worse: every closed year is now held
* a second time as its own tagged copy, so an unfloored sum adds each one on
* top of the opening balance that already contains it.
*/
describe("customer file ledger card", () => {
function serviceWith(floor: Date | null) {
const groupBy = jest.fn().mockResolvedValue([]);
const prisma = {
customer: {
findUnique: jest.fn().mockResolvedValue({ id: "c1", transactions: [] }),
},
transaction: {
findFirst: jest
.fn()
.mockResolvedValue(floor ? { transactionDate: floor } : null),
groupBy,
},
};
return {
service: new CustomersService(prisma as never),
prisma,
groupBy,
};
}
it("takes the same balance floor the statement takes", async () => {
const floor = new Date("2026-01-01T00:00:00Z");
const { service, prisma, groupBy } = serviceWith(floor);
await service.detail("c1");
expect(prisma.transaction.findFirst).toHaveBeenCalledWith(
expect.objectContaining({
where: expect.objectContaining({
type: { nameEn: BALANCE_FORWARD_TYPE },
}),
}),
);
expect(groupBy.mock.calls[0][0].where).toMatchObject({
transactionDate: { gte: floor },
});
});
it("applies no floor when the customer never had an opening balance", async () => {
// 102 customers have no BALANCE FORWARD row at all. Inventing a floor for
// them would hide their whole ledger.
const { service, groupBy } = serviceWith(null);
await service.detail("c1");
expect(groupBy.mock.calls[0][0].where).not.toHaveProperty("transactionDate");
});
it("counts an archive as history but never as current", async () => {
// The floor alone is not enough: a customer floored by an archive clears
// it with every row of that archive, and the rows archives spill into the
// following January clear any floor. But excluding archives outright is
// wrong too — below the year start they are the only carry a
// floored-by-archive customer has (NUMid 295, 785.46).
const { service, groupBy } = serviceWith(new Date("2026-01-01T00:00:00Z"));
await service.detail("c1");
const and = groupBy.mock.calls[0][0].where.AND;
const rule = and.find((c: { OR?: unknown[] }) =>
JSON.stringify(c).includes("datos2@"),
);
expect(rule.OR).toEqual([
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: "datos2@" } } },
{ transactionDate: { lt: expect.any(Date) } },
]);
});
it("keeps the cash-source exclusion so it reads like the statement", async () => {
const { service, groupBy } = serviceWith(new Date("2026-01-01T00:00:00Z"));
await service.detail("c1");
const and = groupBy.mock.calls[0][0].where.AND;
const sourceRule = and.find((c: { OR?: unknown[] }) =>
JSON.stringify(c).includes("EFECTIVO"),
);
expect(sourceRule).toBeDefined();
});
it("drops outstanding rows, as every balance does", async () => {
const { service, groupBy } = serviceWith(null);
await service.detail("c1");
expect(groupBy.mock.calls[0][0].where).toMatchObject({
outstanding: false,
});
});
it("keeps archives out of the year's movement list too", async () => {
// datos2@2024 carries rows dated into 2026; a date test alone would show
// them as current-year movements next to the live ledger's own copy.
const { service, prisma } = serviceWith(null);
await service.detail("c1");
const include = prisma.customer.findUnique.mock.calls[0][0].include;
expect(include.transactions.where.OR).toEqual([
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: "datos2@" } } },
// Nothing below yearStart reaches this list, so the third branch never
// admits an archive row here — it is carried for one shared rule.
{ transactionDate: { lt: expect.any(Date) } },
]);
});
});
+88 -4
View File
@@ -3,6 +3,35 @@ import { Prisma } from "@jorgecuadros/database";
import { PrismaService } from "../prisma/prisma.service";
import { CreateCustomerDto } from "./create-customer.dto";
import { UpdateCustomerDto } from "./update-customer.dto";
import {
BALANCE_FORWARD_TYPE,
notCashJournal,
PERIOD_TABLE_PREFIX,
} from "../billing/billing.service";
/**
* Keeps an imported prior period out of the *current* period, NULL-safely.
*
* A closed year is imported as its own tagged copy (`datos2@2025`). Below the
* year start it is history and counts — for the one customer whose newest
* BALANCE FORWARD lives inside an archive it is the only carry there is, and
* dropping it understated NUMid 295 by his entire 2025 closing balance. At or
* above the year start it must go: the archives spill a couple of rows into the
* following January, and those already sit inside the next year's BALANCE
* FORWARD, which is the sum of the whole archive.
*
* Spelled as a positive OR because `NOT (col LIKE ... AND ...)` evaluates to
* NULL for an app-captured row (no legacySourceTable), dropping every one.
*/
const archiveIsHistory = (
yearStart: Date,
): Prisma.TransactionWhereInput => ({
OR: [
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: PERIOD_TABLE_PREFIX } } },
{ transactionDate: { lt: yearStart } },
],
});
export interface ListParams {
query?: string;
@@ -96,6 +125,13 @@ export class CustomersService {
/** Full unified customer view: identity + both business lines + ledger. */
async detail(id: string) {
// The movement list on the customer file is the same statement the office
// prints, so it follows the same rule as BillingService.statement: this
// calendar year, oldest-first. No `take` any more — the cap used to hide
// the end of a busy customer's year once the order flipped, and a single
// year is small (365 rows for the heaviest customer in the book).
const yearStart = new Date(Date.UTC(new Date().getUTCFullYear(), 0, 1));
const customer = await this.prisma.customer.findUnique({
where: { id },
include: {
@@ -117,8 +153,21 @@ export class CustomersService {
},
},
transactions: {
orderBy: { transactionDate: "desc" },
take: 100,
// Archives are excluded by tag, not by date. They are not cleanly
// bounded — datos2@2024 carries rows dated 2022, 2023, 2025 and one
// in 2026, datos2@2025 two more — so a date test alone would surface
// a closed year's rows in the current year's list, duplicating the
// live ledger's own copy of them for three customers.
// Archives are kept out by tag, not by date. They are not cleanly
// bounded — datos2@2025 carries rows dated into 2026 — so a date test
// alone would surface a closed year's rows in the current year's
// list. Nothing below yearStart reaches this list anyway, so the
// window rule reduces to a plain exclusion here.
where: {
transactionDate: { gte: yearStart },
...archiveIsHistory(yearStart),
},
orderBy: [{ transactionDate: "asc" }, { id: "asc" }],
include: { type: true },
},
},
@@ -130,16 +179,51 @@ export class CustomersService {
// Ledger totals per domain + currency (the "one statement across both
// business lines" payoff), computed in the DB rather than in JS.
//
// These have to answer the same question BillingService.statement answers,
// because this card is titled "Estado de cuenta" and links straight to it —
// two screens quoting one customer two different balances is worse than
// either number alone. So it takes the same three rules the statement uses:
// the balance floor, the cash-source exclusion, and dropping outstanding
// rows the office has not paid yet.
//
// Without the floor these were a raw lifetime sum, double-counting the
// pre-cutover history each BALANCE FORWARD row already absorbs. Importing
// prior periods made that visibly worse: for NUMid 501 the tiles read
// -7,119.29 before the archives landed and -15,270.59 after, against a true
// -10,715.29 — the difference being exactly the 2024 and 2025 closing
// balances, added a second time on top of the opening row that contains
// them.
const floor = await this.prisma.transaction.findFirst({
where: {
customerId: id,
voidedAt: null,
type: { nameEn: BALANCE_FORWARD_TYPE },
},
orderBy: { transactionDate: "desc" },
select: { transactionDate: true },
});
const summary = await this.prisma.transaction.groupBy({
by: ["domain", "currency"],
// Exclude voided rows so the per-domain balance matches the statement.
where: { customerId: id, voidedAt: null },
where: {
customerId: id,
voidedAt: null,
outstanding: false,
...(floor ? { transactionDate: { gte: floor.transactionDate } } : {}),
// The floor alone does not settle the archives: a customer floored by
// an archive clears it with every row of that archive, and the rows
// archives spill into the following January clear any floor.
AND: [archiveIsHistory(yearStart), notCashJournal()],
},
_sum: { amount: true },
_count: { _all: true },
});
return {
...customer,
/** Calendar year the movement list covers. */
transactionYear: yearStart.getUTCFullYear(),
transactionSummary: summary.map((s) => ({
domain: s.domain,
currency: s.currency,
+78 -10
View File
@@ -31,6 +31,29 @@ export const INGEST_FILES = [
] as const;
export type IngestName = (typeof INGEST_FILES)[number];
/**
* A prior-period archive: one Access snapshot per closed year, named for the
* period it holds. `2025.accdb` is UTILITIES as it stood when 2025 was cut.
*
* The filename is the entire declaration of the period — nothing inside the
* file names its year, because a snapshot's `datos2` is indistinguishable from
* the live one — so this pattern is both the allowlist and the contract. It is
* anchored and allows no separator, which is what keeps an upload from
* escaping the ingest directory.
*/
const PERIOD_FILE_RE = /^(\d{4})\.accdb$/i;
/** Earliest period we will accept, so a typo'd year cannot mint a bogus one. */
const PERIOD_MIN_YEAR = 1990;
export function periodYearOf(name: string): number | null {
const m = PERIOD_FILE_RE.exec(name);
if (!m) return null;
const year = Number(m[1]);
if (year < PERIOD_MIN_YEAR || year > new Date().getUTCFullYear()) return null;
return year;
}
/**
* Prefix for every command containing a pipe. Without it the exit status of
* `mysqldump | gzip` is gzip's, so a dump that failed immediately still looks
@@ -120,19 +143,30 @@ export class OpsService implements OnModuleInit {
/* -------------------------------------------------------------- ingest */
private assertIngestName(name: string): IngestName {
if (!INGEST_FILES.includes(name as IngestName)) {
throw new BadRequestException(
`Archivo no permitido. Debe ser uno de: ${INGEST_FILES.join(", ")}`,
);
}
return name as IngestName;
private assertIngestName(name: string): string {
if (INGEST_FILES.includes(name as IngestName)) return name;
if (periodYearOf(name) !== null) return name;
throw new BadRequestException(
`Archivo no permitido. Debe ser uno de: ${INGEST_FILES.join(", ")}` +
`, o un archivo de periodo anterior con nombre AAAA.accdb (por ejemplo 2025.accdb).`,
);
}
async listIngest(): Promise<
{ name: string; present: boolean; size: number | null; modifiedAt: string | null }[]
{
name: string;
present: boolean;
size: number | null;
modifiedAt: string | null;
/** Set only on a prior-period archive; null on the four fixed sources. */
periodYear: number | null;
}[]
> {
return Promise.all(
// The four fixed sources are listed whether present or not — they are
// required, so "missing" is the useful state to show. Period archives are
// optional and unbounded, so they are listed only once uploaded, newest
// year first.
const fixed = await Promise.all(
INGEST_FILES.map(async (name) => {
try {
const st = await fs.stat(path.join(this.ingestDir, name));
@@ -141,12 +175,46 @@ export class OpsService implements OnModuleInit {
present: true,
size: st.size,
modifiedAt: st.mtime.toISOString(),
periodYear: null as number | null,
};
} catch {
return { name, present: false, size: null, modifiedAt: null };
return {
name,
present: false,
size: null,
modifiedAt: null,
periodYear: null as number | null,
};
}
}),
);
let entries: string[] = [];
try {
entries = await fs.readdir(this.ingestDir);
} catch {
entries = [];
}
const periods = (
await Promise.all(
entries
.map((name) => ({ name, year: periodYearOf(name) }))
.filter((e): e is { name: string; year: number } => e.year !== null)
.sort((a, b) => b.year - a.year)
.map(async ({ name, year }) => {
const st = await fs.stat(path.join(this.ingestDir, name));
return {
name,
present: true,
size: st.size,
modifiedAt: st.mtime.toISOString(),
periodYear: year,
};
}),
)
).filter(Boolean);
return [...fixed, ...periods];
}
async saveIngest(name: string, data: Buffer): Promise<void> {
+48
View File
@@ -0,0 +1,48 @@
import { periodYearOf } from "./ops.service";
/**
* `periodYearOf` is the upload allowlist for prior-period archives, so it is
* doing two jobs at once: deciding what counts as a period file, and keeping a
* caller-supplied name from escaping the ingest directory. Both are pinned here.
*/
describe("periodYearOf", () => {
it("accepts a four-digit year archive", () => {
expect(periodYearOf("2025.accdb")).toBe(2025);
expect(periodYearOf("1999.accdb")).toBe(1999);
});
it("is case-insensitive on the extension", () => {
expect(periodYearOf("2025.ACCDB")).toBe(2025);
});
it("rejects a path that would escape the ingest directory", () => {
// The name is joined onto the ingest path, so anything with a separator or
// a parent reference has to fail before it reaches the filesystem.
expect(periodYearOf("../2025.accdb")).toBeNull();
expect(periodYearOf("../../etc/passwd")).toBeNull();
expect(periodYearOf("sub/2025.accdb")).toBeNull();
expect(periodYearOf("2025.accdb/../../x")).toBeNull();
});
it("rejects names that only look like a period", () => {
expect(periodYearOf("202.accdb")).toBeNull();
expect(periodYearOf("20255.accdb")).toBeNull();
expect(periodYearOf("2025.mdb")).toBeNull();
expect(periodYearOf("copia 2025.accdb")).toBeNull();
expect(periodYearOf("2025.accdb.bak")).toBeNull();
expect(periodYearOf("UTILITIES.accdb")).toBeNull();
});
it("rejects years outside the plausible range", () => {
// A typo'd year would otherwise mint a period nobody can ever reconcile:
// there is no BALANCE FORWARD for the year after it to check against.
expect(periodYearOf("1889.accdb")).toBeNull();
expect(periodYearOf(`${new Date().getUTCFullYear() + 1}.accdb`)).toBeNull();
});
it("accepts the current year, which is the earliest a period can be cut", () => {
expect(periodYearOf(`${new Date().getUTCFullYear()}.accdb`)).toBe(
new Date().getUTCFullYear(),
);
});
});
+18
View File
@@ -12,6 +12,10 @@ import { Currency } from "@jorgecuadros/database";
// Each child DTO covers create; updates reuse the same shape with all fields
// optional via the corresponding Update class. Route supplies the policyId.
// A policy split into several exhibiciones prices each payment on its own —
// the Access form printed the whole money row once per pago — so the premium
// breakdown repeats here. `amount` remains what was actually collected and is
// never recomputed from the breakdown; the two differ by rounding in the books.
export class InstallmentDto {
@IsInt() sequence!: number;
@IsOptional() @IsNumber() amount?: number;
@@ -20,6 +24,13 @@ export class InstallmentDto {
@IsOptional() @IsString() paidDate?: string;
@IsOptional() @IsString() checkNumber?: string;
@IsOptional() @IsBoolean() isCash?: boolean;
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() surcharge?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() taxRate?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsNumber() commission?: number;
}
export class UpdateInstallmentDto {
@IsOptional() @IsInt() sequence?: number;
@@ -29,6 +40,13 @@ export class UpdateInstallmentDto {
@IsOptional() @IsString() paidDate?: string;
@IsOptional() @IsString() checkNumber?: string;
@IsOptional() @IsBoolean() isCash?: boolean;
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() surcharge?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() taxRate?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsNumber() commission?: number;
}
export class VehicleDto {
@@ -0,0 +1,94 @@
import { BadRequestException } from "@nestjs/common";
import { PoliciesService } from "./policies.service";
/**
* Deleting a lookup row that policies still reference used to succeed and
* silently blank the field on every one of them, because both FKs are
* `ON DELETE SET NULL` (`0000_init`). That is not a hypothetical: it is how
* the `M_EMPR` policy type disappeared from the dev database and left 5
* policies with a null `policyTypeId`, found only by querying months later.
*
* These tests pin the refusal. They drive the service with a stub client
* rather than a database because what is being asserted is the guard, not
* Prisma — and a test that needed a live MySQL would not run in CI.
*/
function serviceWith(counts: {
policies?: number;
claims?: number;
}): { service: PoliciesService; deleted: string[] } {
const deleted: string[] = [];
const prisma = {
policy: { count: async () => counts.policies ?? 0 },
claim: { count: async () => counts.claims ?? 0 },
insuranceProvider: {
findUnique: async () => ({ id: "p1", name: "ANA SEGUROS" }),
delete: async () => {
deleted.push("provider");
return { id: "p1" };
},
},
policyType: {
findUnique: async () => ({ id: "t1", name: "M_EMPR" }),
delete: async () => {
deleted.push("policyType");
return { id: "t1" };
},
},
adjuster: {
findUnique: async () => ({ id: "a1", name: "JUAN PEREZ" }),
delete: async () => {
deleted.push("adjuster");
return { id: "a1" };
},
},
};
const storage = {} as never;
return {
service: new PoliciesService(prisma as never, storage),
deleted,
};
}
describe("lookup deletes refuse while the row is in use", () => {
it("refuses a policy type that policies still carry, and names the count", () => {
const { service, deleted } = serviceWith({ policies: 5 });
return service.removePolicyType("t1").then(
() => {
throw new Error("expected the delete to be refused");
},
(err: unknown) => {
expect(err).toBeInstanceOf(BadRequestException);
// The operator has to be told WHICH row and HOW MANY, or the message
// is not actionable.
expect((err as Error).message).toContain("M_EMPR");
expect((err as Error).message).toContain("5");
expect(deleted).toEqual([]);
},
);
});
it("refuses a carrier that policies still carry", async () => {
const { service, deleted } = serviceWith({ policies: 738 });
await expect(service.removeProvider("p1")).rejects.toBeInstanceOf(
BadRequestException,
);
expect(deleted).toEqual([]);
});
it("refuses an adjuster still assigned to claims", async () => {
// Same `ON DELETE SET NULL` trap, on `claims.adjusterId`.
const { service, deleted } = serviceWith({ claims: 2 });
await expect(service.removeAdjuster("a1")).rejects.toBeInstanceOf(
BadRequestException,
);
expect(deleted).toEqual([]);
});
it("allows the delete once nothing references the row", async () => {
const { service, deleted } = serviceWith({ policies: 0, claims: 0 });
await service.removePolicyType("t1");
await service.removeProvider("p1");
await service.removeAdjuster("a1");
expect(deleted).toEqual(["policyType", "provider", "adjuster"]);
});
});
+7 -1
View File
@@ -1,4 +1,4 @@
import { IsOptional, IsString, MinLength } from "class-validator";
import { IsNumber, IsOptional, IsString, Max, Min, MinLength } from "class-validator";
export class ProviderDto {
@IsString() @MinLength(1) name!: string;
@@ -7,13 +7,19 @@ export class UpdateProviderDto {
@IsOptional() @IsString() @MinLength(1) name?: string;
}
// `taxRate` is the IVA fraction for this line of business (0.08 = 8%), the
// legacy one-row IMPUESTOS / IMPUESTOS_AUTOS tables made editable. Bounded at
// 1 because a rate is a fraction, not a percentage: 8 entered here would tax a
// $600 premium $4,800, and the mistake is easy to make.
export class PolicyTypeDto {
@IsString() @MinLength(1) name!: string;
@IsOptional() @IsString() shortDescription?: string;
@IsOptional() @IsNumber() @Min(0) @Max(1) taxRate?: number;
}
export class UpdatePolicyTypeDto {
@IsOptional() @IsString() @MinLength(1) name?: string;
@IsOptional() @IsString() shortDescription?: string;
@IsOptional() @IsNumber() @Min(0) @Max(1) taxRate?: number;
}
export class AdjusterDto {
+79 -6
View File
@@ -1,4 +1,4 @@
import { Injectable, NotFoundException } from "@nestjs/common";
import { BadRequestException, Injectable, NotFoundException } from "@nestjs/common";
import { randomUUID } from "node:crypto";
import { Prisma } from "@jorgecuadros/database";
import { PrismaService } from "../prisma/prisma.service";
@@ -250,7 +250,16 @@ export class PoliciesService {
const [types, providers] = await this.prisma.$transaction([
this.prisma.policyType.findMany({
orderBy: { name: "asc" },
select: { id: true, name: true, _count: { select: { policies: true } } },
select: {
id: true,
name: true,
shortDescription: true,
// The capture form computes IVA client-side as the operator types,
// so the rate has to travel with the type list it already loads —
// an extra round-trip per keystroke is not an option.
taxRate: true,
_count: { select: { policies: true } },
},
}),
this.prisma.insuranceProvider.findMany({
orderBy: { name: "asc" },
@@ -259,7 +268,13 @@ export class PoliciesService {
]);
return {
types: types.map((t) => ({ id: t.id, name: t.name, count: t._count.policies })),
types: types.map((t) => ({
id: t.id,
name: t.name,
shortDescription: t.shortDescription,
taxRate: t.taxRate,
count: t._count.policies,
})),
providers: providers.map((p) => ({
id: p.id,
name: p.name,
@@ -397,6 +412,13 @@ export class PoliciesService {
paidDate: toDate(dto.paidDate) ?? undefined,
checkNumber: dto.checkNumber,
isCash: dto.isCash,
netPremium: dto.netPremium,
surcharge: dto.surcharge,
policyFee: dto.policyFee,
tax: dto.tax,
taxRate: dto.taxRate,
total: dto.total,
commission: dto.commission,
},
});
}
@@ -412,6 +434,13 @@ export class PoliciesService {
...(dto.paidDate !== undefined && { paidDate: toDate(dto.paidDate) }),
checkNumber: dto.checkNumber,
isCash: dto.isCash,
netPremium: dto.netPremium,
surcharge: dto.surcharge,
policyFee: dto.policyFee,
tax: dto.tax,
taxRate: dto.taxRate,
total: dto.total,
commission: dto.commission,
},
});
}
@@ -554,7 +583,40 @@ export class PoliciesService {
updateProvider(id: string, dto: UpdateProviderDto) {
return this.prisma.insuranceProvider.update({ where: { id }, data: dto });
}
removeProvider(id: string) {
/**
* Deleting a lookup row that policies still point at is silent data loss.
*
* Both FKs are `ON DELETE SET NULL` (see `0000_init`), so the delete
* succeeds, returns 200, and blanks the field on every policy that used it
* — with no error and nothing in the UI to suggest anything happened. That
* is how the `M_EMPR` policy type disappeared and left 5 policies with a
* null `policyTypeId`, only found later by querying.
*
* Refusing is the whole fix. There is no "are you sure": the operator
* reassigns those policies first, which is work the app cannot do for them
* because only they know which type is correct.
*/
private async assertLookupUnused(
kind: "provider" | "policyType",
id: string,
): Promise<void> {
const where = kind === "provider" ? { insuranceProviderId: id } : { policyTypeId: id };
const count = await this.prisma.policy.count({ where });
if (count === 0) return;
const label =
kind === "provider"
? (await this.prisma.insuranceProvider.findUnique({ where: { id } }))?.name
: (await this.prisma.policyType.findUnique({ where: { id } }))?.name;
const noun = kind === "provider" ? "La aseguradora" : "El tipo de póliza";
throw new BadRequestException(
`${noun} «${label ?? id}» está en uso por ${count} póliza(s). ` +
"Reasígnelas antes de eliminarlo.",
);
}
async removeProvider(id: string) {
await this.assertLookupUnused("provider", id);
return this.prisma.insuranceProvider.delete({ where: { id } });
}
@@ -564,7 +626,8 @@ export class PoliciesService {
updatePolicyType(id: string, dto: UpdatePolicyTypeDto) {
return this.prisma.policyType.update({ where: { id }, data: dto });
}
removePolicyType(id: string) {
async removePolicyType(id: string) {
await this.assertLookupUnused("policyType", id);
return this.prisma.policyType.delete({ where: { id } });
}
@@ -574,7 +637,17 @@ export class PoliciesService {
updateAdjuster(id: string, dto: UpdateAdjusterDto) {
return this.prisma.adjuster.update({ where: { id }, data: dto });
}
removeAdjuster(id: string) {
/** Same `ON DELETE SET NULL` trap as the two above, on `claims.adjusterId`:
* deleting a busy adjuster would quietly strip them off their claims. */
async removeAdjuster(id: string) {
const count = await this.prisma.claim.count({ where: { adjusterId: id } });
if (count > 0) {
const row = await this.prisma.adjuster.findUnique({ where: { id } });
throw new BadRequestException(
`El ajustador «${row?.name ?? id}» está asignado a ${count} siniestro(s). ` +
"Reasígnelos antes de eliminarlo.",
);
}
return this.prisma.adjuster.delete({ where: { id } });
}
}
+14 -2
View File
@@ -6,12 +6,16 @@ import {
IsString,
MinLength,
} from "class-validator";
import { Currency } from "@jorgecuadros/database";
import { Currency, PaymentFrequency } from "@jorgecuadros/database";
import { IsEnum } from "class-validator";
/** Editable policy-header fields. coveragesJson (freeform legacy blob) is not
* exposed for editing. Dates arrive as ISO strings and are coerced by the
* service. `total` is legacy-dead data — the UI uses netPremium. */
* service. `total` is legacy-dead data on migrated rows — list and sort code
* still uses netPremium — but the capture form writes it going forward, along
* with `tax`, from the arithmetic in premium.ts. Both arrive as plain numbers
* rather than being recomputed server-side: the printed policy is the record
* of truth and staff must be able to key its rounding verbatim. */
export class CreatePolicyDto {
@IsString() @MinLength(1) policyNumber!: string;
@IsString() @MinLength(1) customerId!: string;
@@ -24,10 +28,14 @@ export class CreatePolicyDto {
@IsOptional() @IsString() policyTo?: string;
@IsOptional() @IsInt() coveragePeriodDays?: number;
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() surcharge?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() brokerFee?: number;
@IsOptional() @IsNumber() commission?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() taxRate?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsEnum(PaymentFrequency) paymentFrequency?: PaymentFrequency;
@IsOptional() @IsEnum(Currency) currency?: Currency;
@IsOptional() @IsString() observations?: string;
@IsOptional() @IsString() notes?: string;
@@ -48,10 +56,14 @@ export class UpdatePolicyDto {
@IsOptional() @IsString() policyTo?: string;
@IsOptional() @IsInt() coveragePeriodDays?: number;
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() surcharge?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() brokerFee?: number;
@IsOptional() @IsNumber() commission?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() taxRate?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsEnum(PaymentFrequency) paymentFrequency?: PaymentFrequency;
@IsOptional() @IsEnum(Currency) currency?: Currency;
@IsOptional() @IsString() observations?: string;
@IsOptional() @IsString() notes?: string;
+90
View File
@@ -0,0 +1,90 @@
import {
DEFAULT_TAX_RATE,
computeTax,
computeTotal,
resolveTaxRate,
surchargeApplies,
taxableBase,
} from "./premium";
/**
* The reference case is policy 7006785 (MULT, semestral, GMX, two payments) as
* it stands in the Access books — the screen Jorge sent. Both of its money
* rows are asserted, because the second one is the case that proves the
* surcharge belongs in the taxable base and that a zero policy fee is a real
* value rather than a missing one.
*/
describe("premium arithmetic", () => {
it("matches the first payment of policy 7006785", () => {
const parts = { netPremium: 610.86, surcharge: 8.55, policyFee: 31.0 };
expect(taxableBase(parts)).toBe(650.41);
expect(computeTax(parts, 0.08)).toBe(52.03);
expect(computeTotal(parts, 0.08)).toBe(702.44);
});
it("matches the second payment of policy 7006785", () => {
const parts = { netPremium: 589.71, surcharge: 8.26, policyFee: 0 };
expect(computeTax(parts, 0.08)).toBe(47.84);
expect(computeTotal(parts, 0.08)).toBe(645.81);
});
it("excluding the surcharge does NOT reconcile", () => {
// Guards the one decision in this module that is easy to get wrong: the
// spoken-language version of the rule ("prima neta + derecho * 8%") gives
// 51.35, and the printed policy says 52.03.
const withoutSurcharge = { netPremium: 610.86, surcharge: 0, policyFee: 31.0 };
expect(computeTax(withoutSurcharge, 0.08)).not.toBe(52.03);
});
it("treats blank and null money as zero, not NaN", () => {
expect(taxableBase({ netPremium: "610.86", surcharge: null, policyFee: "" })).toBe(
610.86,
);
expect(computeTax({ netPremium: undefined, surcharge: null, policyFee: null }, 0.08))
.toBe(0);
});
it("rounds half-up to cents", () => {
// 100.06 * 0.08 = 8.0048 -> 8.00; 100.13 * 0.08 = 8.0104 -> 8.01.
expect(computeTax({ netPremium: 100.06, surcharge: 0, policyFee: 0 }, 0.08)).toBe(8);
expect(computeTax({ netPremium: 100.13, surcharge: 0, policyFee: 0 }, 0.08)).toBe(8.01);
});
describe("surchargeApplies", () => {
it("is false for the two single-payment frequencies", () => {
expect(surchargeApplies("ANNUAL")).toBe(false);
expect(surchargeApplies("SINGLE")).toBe(false);
});
it("is true for every split frequency", () => {
expect(surchargeApplies("SEMIANNUAL")).toBe(true);
expect(surchargeApplies("QUARTERLY")).toBe(true);
expect(surchargeApplies("MONTHLY")).toBe(true);
});
it("allows it when the frequency is unknown", () => {
// Every migrated policy is null here — the original ETL dropped FORMA
// PAGO — and those rows DO carry recargo figures in the legacy data.
expect(surchargeApplies(null)).toBe(true);
expect(surchargeApplies(undefined)).toBe(true);
});
});
describe("resolveTaxRate", () => {
it("prefers the rate the policy was issued at", () => {
expect(resolveTaxRate(0.16, 0.08)).toBe(0.16);
});
it("falls back to the line of business", () => {
expect(resolveTaxRate(null, 0.08)).toBe(0.08);
});
it("falls back to the default when nothing is configured", () => {
expect(resolveTaxRate(null, null)).toBe(DEFAULT_TAX_RATE);
expect(resolveTaxRate(undefined, "")).toBe(DEFAULT_TAX_RATE);
});
it("accepts a zero rate as a real choice, not as absent", () => {
// An exempt line of business must read 0, not silently fall through to 8%.
expect(resolveTaxRate(null, 0)).toBe(0);
});
it("accepts Prisma's decimal strings", () => {
expect(resolveTaxRate(null, "0.0800")).toBe(0.08);
});
});
});
+86
View File
@@ -0,0 +1,86 @@
/**
* The premium arithmetic the Access capture form did in unbound calculated
* controls, moved somewhere it can be tested.
*
* Two figures are derived, everything else is keyed by hand:
*
* base = netPremium + surcharge + policyFee
* tax = round(base * rate)
* total = base + tax
*
* The surcharge IS part of the taxable base. That is not an assumption — it is
* the only reading that reconciles the books. Policy 7006785 (MULT, semestral,
* two payments) prints IVA 52.03 and 47.84 against net premiums 610.86 / 589.71,
* surcharges 8.55 / 8.26 and policy fees 31.00 / 0.00; excluding the surcharge
* gives 51.35, which matches nothing on the page.
*
* The surcharge itself is NEVER derived. It is the carrier's financing charge
* for paying in installments, quoted per policy, so staff key it in. It only
* ever appears on a policy that is not paid annually or in a single exhibición
* — `surchargeApplies` is what the UI uses to grey the field out.
*/
/** Used when neither the policy nor its type carries a rate. Matches the
* single row both legacy IMPUESTOS tables held (0.08 = 8%). */
export const DEFAULT_TAX_RATE = 0.08;
export type PaymentFrequencyValue =
| "ANNUAL"
| "SEMIANNUAL"
| "QUARTERLY"
| "MONTHLY"
| "SINGLE";
/** Paying in more than one exhibición is what earns a surcharge. A null
* frequency (every migrated row — Access's FORMA PAGO was dropped by the
* original ETL) is treated as "unknown, allow it" rather than "annual":
* refusing to show a figure that is sitting in the legacy data would hide it. */
export function surchargeApplies(
frequency: PaymentFrequencyValue | null | undefined,
): boolean {
return frequency !== "ANNUAL" && frequency !== "SINGLE";
}
function num(v: unknown): number {
if (v === null || v === undefined || v === "") return 0;
const n = typeof v === "number" ? v : Number(v);
return Number.isFinite(n) ? n : 0;
}
/** Half-up to cents, the way the printed policy rounds. */
export function round2(n: number): number {
return Math.round((n + Number.EPSILON) * 100) / 100;
}
export interface PremiumParts {
netPremium?: unknown;
surcharge?: unknown;
policyFee?: unknown;
}
export function taxableBase(p: PremiumParts): number {
return round2(num(p.netPremium) + num(p.surcharge) + num(p.policyFee));
}
export function computeTax(p: PremiumParts, rate: number): number {
return round2(taxableBase(p) * rate);
}
export function computeTotal(p: PremiumParts, rate: number): number {
return round2(taxableBase(p) + computeTax(p, rate));
}
/** Rate ladder: the figure stored on the policy (so an old policy keeps the
* rate it was issued at even after the catalog changes), else the rate on its
* line of business, else the shipped default. */
export function resolveTaxRate(
policyRate: unknown,
policyTypeRate: unknown,
): number {
for (const candidate of [policyRate, policyTypeRate]) {
if (candidate === null || candidate === undefined || candidate === "") continue;
const n = Number(candidate);
if (Number.isFinite(n) && n >= 0) return n;
}
return DEFAULT_TAX_RATE;
}
@@ -0,0 +1,161 @@
import {
nameTokens,
suggestCustomersByName,
suggestionNote,
type CustomerNameRow,
} from "./name-matcher";
/**
* Every row here is a real name out of the customer book (1536 rows, dev
* mirror of production), chosen because it is one of the shapes that breaks
* naive matching: surname-first ordering, a middle initial, a Spanish double
* surname, a joint account, a missing comma, and the `(SIN NOMBRE)`
* placeholder the migration left for customers whose DATGRAL row had no name.
*/
const BOOK: CustomerNameRow[] = [
{ id: "c1", name: "WAGONER, PAMELA" },
{ id: "c2", name: "MCWILLIAMS, BRIAN MICHAEL" },
{ id: "c3", name: "MCWILLIAMS, BRIAN" },
{ id: "c4", name: "WEAKLAND, RICHARD E." },
{ id: "c5", name: "ESTRADA, JERRY & MARILYN" },
{ id: "c6", name: "CABALLERO PRIETO, GUILLERMO" },
{ id: "c7", name: "GREENE STEPHANIE" },
{ id: "c8", name: "(SIN NOMBRE)" },
{ id: "c9", name: "MUÑOZ, LUIS ALBERTO" },
{ id: "c10", name: "SMITH, DANIEL" },
{ id: "c11", name: "SMITH, JOHN" },
];
describe("nameTokens", () => {
it("makes the two orderings the same set", () => {
expect(nameTokens("PAMELA WAGONER").sort()).toEqual(
nameTokens("WAGONER, PAMELA").sort(),
);
});
it("drops initials, particles and corporate suffixes", () => {
expect(nameTokens("WEAKLAND, RICHARD E.")).toEqual(["WEAKLAND", "RICHARD"]);
expect(nameTokens("GARCIA DE LA TORRE, ANA")).toEqual(["GARCIA", "TORRE", "ANA"]);
expect(nameTokens("CONSTRUCTORA BAJA S.A. DE C.V.")).toEqual([
"CONSTRUCTORA",
"BAJA",
]);
});
it("folds accents so OCR's MUNOZ reaches the book's MUÑOZ", () => {
expect(nameTokens("MUÑOZ")).toEqual(["MUNOZ"]);
});
it("drops the phone number ANA prints against the insured name", () => {
// Observed verbatim from the ANA automobile face.
expect(nameTokens("MARIA GARCIA Ph.3102001538")).toEqual([
"MARIA",
"GARCIA",
"PH",
]);
});
});
describe("suggestCustomersByName", () => {
it("matches the reversed name exactly", () => {
const [top] = suggestCustomersByName("PAMELA WAGONER", BOOK);
expect(top).toMatchObject({ customerId: "c1", tier: "EXACT", score: 1 });
});
it("treats a printed middle name the book lacks as a partial hit", () => {
const hits = suggestCustomersByName("PAMELA DENISE WAGONER", BOOK);
expect(hits[0]).toMatchObject({ customerId: "c1", tier: "PARTIAL" });
expect(hits[0].score).toBeCloseTo(2 / 3);
});
it("ranks the exact row above the row that merely contains it", () => {
// Both MCWILLIAMS rows are reachable from this name; the one that holds
// the middle name is the exact set and must come first.
const hits = suggestCustomersByName("BRIAN MICHAEL MCWILLIAMS", BOOK);
expect(hits.map((h) => h.customerId)).toEqual(["c2", "c3"]);
expect(hits[0].tier).toBe("EXACT");
expect(hits[1].tier).toBe("PARTIAL");
});
it("reaches a joint account from the one spouse the carrier printed", () => {
const hits = suggestCustomersByName("JERRY ESTRADA", BOOK);
expect(hits[0]).toMatchObject({ customerId: "c5", tier: "PARTIAL" });
});
it("will not reach a joint account on given names alone", () => {
// No surname printed: `JERRY MARILYN` overlaps ESTRADA, JERRY & MARILYN
// on two tokens, and matching on that would book a stranger's policy.
expect(suggestCustomersByName("JERRY MARILYN", BOOK)).toEqual([]);
});
it("matches a Spanish double surname regardless of where the comma fell", () => {
const [top] = suggestCustomersByName("GUILLERMO CABALLERO PRIETO", BOOK);
expect(top).toMatchObject({ customerId: "c6", tier: "EXACT" });
});
it("still matches a book row that has no comma", () => {
const [top] = suggestCustomersByName("STEPHANIE GREENE", BOOK);
expect(top).toMatchObject({ customerId: "c7", tier: "EXACT" });
});
it("never suggests the (SIN NOMBRE) placeholder", () => {
expect(suggestCustomersByName("SIN NOMBRE", BOOK)).toEqual([]);
expect(suggestCustomersByName("NOMBRE DEL ASEGURADO", BOOK)).toEqual([]);
});
it("returns nothing on a shared surname alone", () => {
// 185 surnames are shared by 524 customers; one token is not evidence.
expect(suggestCustomersByName("SMITH", BOOK)).toEqual([]);
});
it("returns nothing for a different person with the same surname", () => {
expect(suggestCustomersByName("ROBERT SMITH", BOOK)).toEqual([]);
});
it("refuses a page-sized blob", () => {
// GMX's especificación has no field labels and the parser has handed its
// whole first page over as the insured name.
const blob =
"ESPECIFICACION DE LA POLIZA DE SEGURO DE RESPONSABILIDAD CIVIL " +
"EXPEDIDA A FAVOR DE PAMELA WAGONER CON VIGENCIA DEL 01 DE ENERO";
expect(suggestCustomersByName(blob, BOOK)).toEqual([]);
});
it("caps the list", () => {
expect(suggestCustomersByName("BRIAN MICHAEL MCWILLIAMS", BOOK, 1)).toHaveLength(1);
});
it("handles a null insured name", () => {
expect(suggestCustomersByName(null, BOOK)).toEqual([]);
});
});
describe("suggestionNote", () => {
it("says nothing when there is nothing", () => {
expect(suggestionNote([])).toBeNull();
});
it("names a single exact hit", () => {
expect(suggestionNote(suggestCustomersByName("PAMELA WAGONER", BOOK))).toBe(
"posible cliente por nombre: WAGONER, PAMELA",
);
});
it("reports a tie rather than picking one", () => {
// The book really does hold EMERY, LAURA twice and KIRCHHOFF, CINDY
// three times.
const dupes: CustomerNameRow[] = [
{ id: "d1", name: "EMERY, LAURA" },
{ id: "d2", name: "EMERY, LAURA" },
];
expect(suggestionNote(suggestCustomersByName("LAURA EMERY", dupes))).toBe(
"2 clientes tienen ese mismo nombre; elija cuál",
);
});
it("lists partial hits", () => {
expect(suggestionNote(suggestCustomersByName("PAMELA DENISE WAGONER", BOOK))).toBe(
"posibles clientes por nombre: WAGONER, PAMELA",
);
});
});
+199
View File
@@ -0,0 +1,199 @@
/**
* Suggests which existing customer a printed insured name belongs to.
*
* The office books customers surname-first ("WAGONER, PAMELA") and carriers
* print them given-name-first ("PAMELA DENISE WAGONER"), so a string compare
* never hits. Comparing *token sets* does, and it is order-insensitive by
* construction — which is the whole trick.
*
* **These are suggestions, never matches.** Nothing here sets
* `matchedCustomerId` or `confident`; the review screen offers the ranked
* names and a human picks. That line is not caution, it is what the book
* measures out to: of 1536 customers, 1487 have a distinct normalized token
* set — but loosen the rule to surname + first given name only and 131 of
* them (8.5%) collide, because the book holds `MCWILLIAMS, BRIAN MICHAEL`
* *and* `MCWILLIAMS, BRIAN`, and `CUADROS, JORGE JR` alongside three
* `CUADROS, JORGE H.`. 185 surnames are shared by 524 customers, so a
* surname alone carries no information at all.
*
* The two tiers below are drawn at the two places that measurement puts a
* cliff: full token-set equality, where cross-person collisions are
* effectively zero, and strict containment, where they are common enough
* that the result can only ever be a hint.
*/
/** A customer row as the matcher needs it — id and the book's name. */
export interface CustomerNameRow {
id: string;
name: string;
}
export type NameMatchTier = "EXACT" | "PARTIAL";
export interface CustomerNameSuggestion {
customerId: string;
customerName: string;
/**
* `EXACT` — the two names carry the same tokens, in any order.
* `PARTIAL` — one name's tokens are all present in the other's, plus the
* surname. A printed middle name the book does not hold, or a joint
* account where the carrier named one spouse, both land here.
*/
tier: NameMatchTier;
/** Shared tokens over the longer name's token count, 0..1. */
score: number;
}
/**
* Words that carry no identity. Spanish particles and the ampersand joining
* a couple are noise; the corporate suffixes are dropped so `S.A. DE C.V.`
* does not make every company look alike.
*/
const NOISE = new Set([
"DE", "DEL", "LA", "LAS", "LOS", "Y", "AND", "VDA",
"JR", "SR", "II", "III", "IV",
"SA", "CV", "SAPI", "SRL", "RL", "SC", "INC", "LLC", "LTD", "CORP", "CO",
]);
/**
* Placeholder rows the migration left behind. Fourteen customers are named
* literally `(SIN NOMBRE)`; without this they would be one 14-way tie on
* every unreadable name.
*/
const PLACEHOLDER = new Set(["SIN NOMBRE", "NOMBRE SIN"]);
/**
* A name blob longer than this is not a name. GMX's PVL especificación has
* no field labels, and the parser has been seen handing its entire first
* page over as `insuredName`; matching that against the book would find
* a surname somewhere in the prose and suggest a stranger.
*/
const MAX_TOKENS = 8;
const MAX_CHARS = 80;
/**
* Splits a name into comparable tokens.
*
* Accents go first, and deliberately in both directions: the book holds
* `MUÑOZ` where OCR routinely reads `MUNOZ`, and folding both to the same
* ASCII makes that a hit rather than a miss.
*
* Tokens containing digits are dropped outright. ANA's automobile face
* prints the phone number hard against the insured name — the parser has
* emitted `MARIA GARCIA Ph.3102001538` — and the digits would otherwise
* be an extra token forever blocking `EXACT`.
*
* Single letters are dropped as initials: the book is full of
* `WEAKLAND, RICHARD E.`, and a carrier that prints the middle name in
* full should still match the row that abbreviates it.
*/
export function nameTokens(raw: string): string[] {
const cleaned = raw
.normalize("NFD")
.replace(/[\u0300-\u036f]/g, "")
.toUpperCase()
.replace(/[^A-Z0-9]+/g, " ")
.trim();
const tokens = cleaned
.split(" ")
.filter((t) => t.length > 1 && !/\d/.test(t) && !NOISE.has(t));
return [...new Set(tokens)];
}
/** The surname tokens — everything before the comma the book writes. */
function surnameTokens(bookName: string): string[] {
const comma = bookName.indexOf(",");
// 54 of 1536 rows have no comma at all ("GREENE STEPHANIE",
// "FAROOQ VAKIL"), and which half is the surname is unknowable. Requiring
// a surname we cannot identify would silently exclude those rows, so they
// fall back to requiring nothing beyond the containment rule.
if (comma < 0) return [];
return nameTokens(bookName.slice(0, comma));
}
function isPlaceholder(tokens: string[]): boolean {
return tokens.length === 0 || PLACEHOLDER.has([...tokens].sort().join(" "));
}
function containsAll(haystack: Set<string>, needles: string[]): boolean {
return needles.every((n) => haystack.has(n));
}
/**
* Ranks the book against one printed name.
*
* Returns at most `limit` suggestions, `EXACT` before `PARTIAL` and higher
* score first. An empty array means the printed name was unusable (too
* long, too few real tokens) or nothing in the book came close — both of
* which leave the review screen exactly as it is today.
*/
export function suggestCustomersByName(
printedName: string | null | undefined,
customers: CustomerNameRow[],
limit = 3,
): CustomerNameSuggestion[] {
if (!printedName || printedName.length > MAX_CHARS) return [];
const printed = nameTokens(printedName);
// One usable token is a surname or a given name on its own, and 34% of the
// book shares a surname with someone. Nothing useful can come of it.
if (printed.length < 2 || printed.length > MAX_TOKENS) return [];
const printedSet = new Set(printed);
const out: CustomerNameSuggestion[] = [];
for (const c of customers) {
const book = nameTokens(c.name);
if (isPlaceholder(book) || book.length < 2) continue;
const bookSet = new Set(book);
const overlap = printed.filter((t) => bookSet.has(t)).length;
// Two shared tokens is the floor: one is a bare surname collision.
if (overlap < 2) continue;
const bookInPrinted = containsAll(printedSet, book);
const printedInBook = containsAll(bookSet, printed);
if (!bookInPrinted && !printedInBook) continue;
// When the book's name is the shorter one, containment already proves
// the surname was printed. When the printed name is shorter — the book
// holds a middle name or a second spouse the carrier omitted — the
// surname must be there explicitly, or `JERRY MARILYN` would match
// `ESTRADA, JERRY & MARILYN` on given names alone.
if (!bookInPrinted && !containsAll(printedSet, surnameTokens(c.name))) continue;
out.push({
customerId: c.id,
customerName: c.name,
tier: bookInPrinted && printedInBook ? "EXACT" : "PARTIAL",
score: overlap / Math.max(book.length, printed.length),
});
}
out.sort((a, b) => {
if (a.tier !== b.tier) return a.tier === "EXACT" ? -1 : 1;
if (b.score !== a.score) return b.score - a.score;
return a.customerName.localeCompare(b.customerName);
});
return out.slice(0, limit);
}
/** Review-queue wording for what the suggestions amount to. */
export function suggestionNote(suggestions: CustomerNameSuggestion[]): string | null {
if (suggestions.length === 0) return null;
const exact = suggestions.filter((s) => s.tier === "EXACT");
// More than one exact hit is the duplicate-customer case the book really
// has (`EMERY, LAURA` twice, `KIRCHHOFF, CINDY` three times). Saying so is
// more useful than naming whichever one sorted first.
if (exact.length > 1) {
return `${exact.length} clientes tienen ese mismo nombre; elija cuál`;
}
if (exact.length === 1) {
return `posible cliente por nombre: ${exact[0].customerName}`;
}
return `posibles clientes por nombre: ${suggestions.map((s) => s.customerName).join(", ")}`;
}
@@ -15,6 +15,11 @@ function page(text: string): OcrPage {
return { text, words: [], confidence: 0.95 };
}
/** Coverages keyed by their risk label, so an assertion names the coverage
* it is about instead of an array index that shifts when one is added. */
const byRisk = (p: ReturnType<typeof parsePolicy>): Record<string, ParsedCoverage> =>
Object.fromEntries(p.coverages.map((c) => [c.risk, c]));
describe("detectPolicyProvider", () => {
it("claims GMX from the brand wordmark on the letterhead", () => {
expect(
@@ -111,6 +116,13 @@ describe("parsePolicy / GMX", () => {
expect(p.coverages.length).toBeGreaterThan(10);
});
it("names the product MULT for confirm to resolve", () => {
// The caratula's own header reads "Multiple Policy / Home". MULT is the
// legacy discriminator for that multi-line home policy; INCENDIO is
// fire-only and no policy in the book has ever used it.
expect(parsePolicy(GMX_FULL).policyTypeName).toBe("MULT");
});
it("leaves premium fields null on the certificate page and notes it", () => {
const p = parsePolicy(GMX_FULL);
expect(p.netPremium).toBeNull();
@@ -145,3 +157,778 @@ describe("parsePolicy / GMX", () => {
expect(eqTyped.lossParticipation).toBe("20%");
});
});
/**
* The second GMX document family: the Spanish PVL "especificación" the office
* receives as `…-CondicionesParticulares.pdf`. Verbatim excerpts from
* `007_LGS-HGMX_07006957_01_0-CondicionesParticulares.pdf` through
* `pdftotext -layout`, indentation included — the column positions and the
* blank lines between blocks are what the parser reads, so a cleaned-up
* fixture would test nothing.
*/
describe("parsePolicy / GMX especificación (PVL Hogar)", () => {
const HEADER =
" ESPECIFICACIÓN QUE SE ADHIERE Y FORMA PARTE INTEGRANTE DE LA PÓLIZA\n" +
" 07-037-07006957-00000-01\n" +
"\n";
const GMX_ESPEC = page(
HEADER +
"\n" +
" Nombre del asegurado EMMER . KATHLEEN\n" +
"\n" +
" Tipo Persona Asegurada Propietario\n" +
"\n" +
" Ubicación del riesgo LOS PELICANOS ESTE NO. 98 Col. LAS GAVIOTAS PLAYAS\n" +
" DE ROSARITO BAJA CALIFORNIA 22713\n" +
"\n" +
" Características del Inmueble Casa Tipo constructivo Combinado: Macizo y Madera.\n" +
" Consta de 2 pisos incluyendo sótanos y planta baja.\n" +
"\n" +
" -500 mts.cuerpo agua SI\n" +
"\n" +
" Asegurado Adicional\n" +
"\n" +
"PVL Hogar - GMX Seguros Página: 1 de 10\n" +
HEADER +
"\n" +
" SECCIÓN INCENDIO EDIFICIO Y CONTENIDOS\n" +
"\n" +
" EDIFICIO\n" +
"\n" +
" Límite Máximo de Responsabilidad:\n" +
" $200,000.00 USD\n" +
"\n" +
" Quedan amparados los muros de contención y bardas, así como puertas y portones, hasta un sublimite de $ 50,000.00 M.N. o su\n" +
" equivalente en dólares americanos, o hasta el 10% de la suma asegurada de la sección de Edificio, lo que resulte menor.\n" +
"\n" +
"\n" +
" CONTENIDOS\n" +
"\n" +
" Límite Máximo de Responsabilidad:\n" +
" $20,000.00 USD\n" +
"\n" +
" 2. Terremoto o erupción volcánica: Sección Edificio EXCLUIDO, Sección Contenidos EXCLUIDO\n" +
"\n" +
" 3. Fenómenos hidrometeorológicos: Sección Edificio $200,000.00 USD, Sección Contenidos $20,000.00 USD\n" +
"\n" +
" Riesgos adicionales.\n" +
"\n" +
" Remoción de escombros\n" +
"\n" +
" Límite Máximo de Responsabilidad:\n" +
" Edificio\n" +
" $20,000.00 USD\n" +
" Contenidos\n" +
" $2,000.00 USD\n" +
"\n" +
" Gastos extraordinarios para casa habitación\n" +
"\n" +
" En caso de siniestro por los riesgos cubiertos en esta póliza, GMX Seguros pagará la renta de casa o departamento, casa de\n" +
" huéspedes u hotel cuando se asegure el inmueble, así como los gastos de mudanza, seguro de transporte del menaje de casa y\n" +
" efectuados.\n" +
"\n" +
" Límite Máximo de Responsabilidad:\n" +
" $22,000.00 USD\n" +
" Periodo de indemnización: 4 meses.\n" +
"\n" +
" Bienes a la Intemperie:\n" +
"\n" +
"\n" +
" 5 POR CIENTO SOBRE SUMA ASEGURADA, 20 PORCIENTO DE PARTICIPACIÓN A CARGO DEL ASEGURADO DE TODA\n" +
" Y CADA PÉRDIDA.\n" +
"\n" +
"\n" +
" Límite Máximo de Responsabilidad: $10,000.00 USD\n" +
"\n" +
" DEDUCIBLES:\n" +
"\n" +
" El procedimiento que se seguirá para la aplicación de deducibles en caso de que la póliza cuente con cláusula inflacionaria en todas\n" +
" y/o en algunas de sus coberturas será como sigue:\n" +
"\n" +
" Fenómenos hidrometeorológicos\n" +
" Zona: A2\n" +
" Deducible\n" +
" Edificio: 1 POR CIENTO SOBRE SUMA ASEGURADA\n" +
" Coaseguro:\n" +
" Zona 1: (INTERIOR) Participación a cargo del asegurado del 10% de toda y cada pérdida.\n" +
" Zona 2: Participación a cargo del asegurado del 10% de toda y cada pérdida.\n" +
"\n" +
" Deducible\n" +
" Contenidos: 1 POR CIENTO SOBRE SUMA ASEGURADA\n" +
"\n" +
" II.- SECCIÓN DIVERSOS MISCELÁNEOS\n" +
"\n" +
" ROBO DE CONTENIDOS\n" +
"\n" +
" Límite de Responsabilidad:\n" +
" $4,000.00 USD\n" +
"\n" +
"\n" +
" Deducible:\n" +
" Sin deducible\n" +
"\n" +
"\n" +
" Sublímites:\n" +
" Joyas, artículos de oro y plata, armas, relojes, pieles, piedras preciosas montadas, colecciones, obras de arte y demás que por su\n" +
"\n" +
"PVL Hogar - GMX Seguros Página: 7 de 10\n" +
HEADER +
"\n" +
" naturaleza se consideran como objetos de difícil o imposible reposición\n" +
"\n" +
"\n" +
" Límite de Responsabilidad:\n" +
"\n" +
" $2,000.00 USD\n" +
"\n" +
"\n" +
" Deducible:\n" +
" Sin deducible\n" +
"\n" +
" Las condiciones generales que forman parte de la presente póliza son las identificadas bajo el nombre:\n" +
" W_HogarGMX_12.11.2025.pdf\n" +
"\n" +
"PVL Hogar - GMX Seguros Página: 10 de 10\n",
);
it("reads a policy number whose groups are not the caratula's widths", () => {
// 2-3-8-5-2 here vs 3-3-8-4-2 on the English caratula. Pinning the widths
// reads one family and returns null on the other.
expect(parsePolicy(GMX_ESPEC).policyNumber).toBe("07-037-07006957-00000-01");
});
it("reads the insured, the risk location across its wrapped line, and the ZIP", () => {
const p = parsePolicy(GMX_ESPEC);
expect(p.provider).toBe("GMX");
expect(p.insuredName).toBe("EMMER . KATHLEEN");
expect(p.legalAddress).toBe(
"LOS PELICANOS ESTE NO. 98 Col. LAS GAVIOTAS PLAYAS DE ROSARITO BAJA CALIFORNIA 22713",
);
expect(p.zip).toBe("22713");
// The cell is printed but empty on this policy — an empty label must not
// capture the next line of the form.
expect(p.additionalInsured).toBeNull();
});
it("leaves the fields this document does not carry null, and says so", () => {
const p = parsePolicy(GMX_ESPEC);
expect(p.policyFrom).toBeNull();
expect(p.policyTo).toBeNull();
expect(p.policyDate).toBeNull();
expect(p.agentName).toBeNull();
expect(p.netPremium).toBeNull();
expect(p.total).toBeNull();
// The note must tell the reviewer to key them in — those three are
// captured by hand on this layout — and must say what silently breaks if
// the vigencia is left empty.
const notes = p.notes.join(" ");
expect(notes).toMatch(/no trae vigencia, agente ni prima/i);
expect(notes).toMatch(/captúrelos a mano/i);
expect(notes).toMatch(/avisos de renovación/i);
});
it("takes the currency from the printed limits, not from the M.N. sublimits", () => {
// The body prose quotes sublimits in pesos ("$ 50,000.00 M.N."); every
// limit is in USD, and only the limits vote.
expect(parsePolicy(GMX_ESPEC).currency).toBe("USD");
});
it("reads each coverage under its own heading", () => {
const c = byRisk(parsePolicy(GMX_ESPEC));
expect(c.EDIFICIO?.insuredAmount).toBe(200000);
expect(c.CONTENIDOS?.insuredAmount).toBe(20000);
expect(c["ROBO DE CONTENIDOS"]?.insuredAmount).toBe(4000);
expect(c["ROBO DE CONTENIDOS"]?.deductible).toBe("Sin deducible");
});
it("splits a limit printed under Edificio / Contenidos sub-labels", () => {
const c = byRisk(parsePolicy(GMX_ESPEC));
expect(c["Remoción de escombros — Edificio"]?.insuredAmount).toBe(20000);
expect(c["Remoción de escombros — Contenidos"]?.insuredAmount).toBe(2000);
});
it("names a coverage after its heading, not after the wrapped tail of the prose above it", () => {
// Walking back from the limit hits "efectuados." — short, and the only
// thing separating it from a heading is that it is not preceded by a
// blank line.
const c = byRisk(parsePolicy(GMX_ESPEC));
expect(c["Gastos extraordinarios para casa habitación"]?.insuredAmount).toBe(22000);
expect(c["efectuados."]).toBeUndefined();
});
it("reads a limit printed on the label's own line", () => {
const c = byRisk(parsePolicy(GMX_ESPEC));
expect(c["Bienes a la Intemperie"]?.insuredAmount).toBe(10000);
});
it("reads a deductible stated as a sentence above the limit", () => {
const c = byRisk(parsePolicy(GMX_ESPEC));
expect(c["Bienes a la Intemperie"]?.deductible).toBe(
"5 POR CIENTO SOBRE SUMA ASEGURADA, 20 PORCIENTO DE PARTICIPACIÓN A CARGO DEL ASEGURADO DE TODA Y CADA PÉRDIDA.",
);
});
it("never borrows a neighbouring coverage's prose as a deductible", () => {
// "…o hasta el 10% de la suma asegurada de la sección de Edificio" is a
// sublimit rule for EDIFICIO, printed two paragraphs above CONTENIDOS.
const c = byRisk(parsePolicy(GMX_ESPEC));
expect(c.CONTENIDOS?.deductible).toBeNull();
expect(c.EDIFICIO?.deductible).toBeNull();
});
it("does not read the page-level DEDUCIBLES paragraph as a deductible", () => {
const p = parsePolicy(GMX_ESPEC);
expect(
p.coverages.some((c) => (c.deductible ?? "").includes("cláusula inflacionaria")),
).toBe(false);
});
it("reads a sublimit block as a sublimit OF the coverage above it", () => {
// The amount sits after a blank line AND a page break, and the block's
// own heading ("Sublímites:") names no risk.
const c = byRisk(parsePolicy(GMX_ESPEC));
expect(c["ROBO DE CONTENIDOS — sublímite"]?.insuredAmount).toBe(2000);
});
it("records an excluded catastrophic risk as excluded, never as zero", () => {
const p = parsePolicy(GMX_ESPEC);
const quake = p.coverages.filter((c) => /Terremoto/i.test(c.risk));
expect(quake).toHaveLength(2);
for (const c of quake) {
expect(c.risk).toMatch(/EXCLUIDO/);
// A coverage insured for $0 and an excluded coverage are the same
// number and very different facts.
expect(c.insuredAmount).toBeNull();
}
});
it("attaches the hydrometeorological deductible and coinsurance from its own block", () => {
const c = byRisk(parsePolicy(GMX_ESPEC));
const building = c["Fenómenos hidrometeorológicos — Sección Edificio"];
expect(building?.insuredAmount).toBe(200000);
expect(building?.deductible).toBe("1 POR CIENTO SOBRE SUMA ASEGURADA");
expect(building?.lossParticipation).toBe("10%");
expect(c["Fenómenos hidrometeorológicos — Sección Contenidos"]?.insuredAmount).toBe(20000);
});
it("names the same product as the caratula — one policy, two artifacts", () => {
expect(parsePolicy(GMX_ESPEC).policyTypeName).toBe("MULT");
});
it("carries the underwriting context the fields have no home for", () => {
const notes = parsePolicy(GMX_ESPEC).notes.join(" | ");
expect(notes).toMatch(/tipo de persona asegurada: Propietario/);
expect(notes).toMatch(/características del inmueble: Casa/);
expect(notes).toMatch(/cuerpo de agua/);
expect(notes).toMatch(/zona catastrófica declarada: A2/);
expect(notes).toMatch(/W_HogarGMX_12\.11\.2025\.pdf/);
});
});
/* ------------------------------------------------------------------ ANA */
/**
* Verbatim `pdftotext -layout` output of the PDFs A.N.A.'s portal produced
* for three real policies, cut at the end of the risk table (the legal
* boilerplate and the repeated AGENT COPY below it are not parsed, and the
* repeats are covered by their own test).
*
* The column padding is load-bearing on the driver's policy, which
* distinguishes SUM INSURED from PREMIUM by horizontal position alone — do
* not reflow these strings.
*/
const ANA_AUTO_AMPLIA = page(`A.N.A. COMPAÑIA DE SEGUROS SA DE CV
LUIS CABRERA #2033 INT. 201, Col. ZONA URBANA RIO TIJUANA
C.P. 22010 MUNICIPIO DE TIJUANA, BAJA CALIFORNIA
www.anaseguros.com.mx
AUTOMOBILE
ALL CLAIMS MUST BE REPORTED BEFORE LEAVING MEXICO
U.S. CELL PHONES TRY + 011-52-55-5322-82-66 MEXICAN CELL PHONES 800-911-911-9 SPECIAL POLICY FOR TOURISTS
TOLL-FREE FROM THE U.S.A. 888-335-7072 BELIZE CELL PHONES 00-52-55-5322-8266
WHATSAPP + 52-55-80-50-3633
No. 700489651
ISSUED BY: DATE ISSUED TERM OF INSURANCE
DAYS
JORGE HUMBERTO CUADROS DAY MONTH YEAR DAY MONTH YEAR TIME
BENITO JUAREZ 25 No.50 INT 38 CENTRO
04 08 2026 FROM 07 08 2026 12:01
365
ROSARITO, BAJA CALIFORNIA 22710
. 70175 TO 07 08 2027 12:01
DISCOUNT PREMIUM POLICY FEE TAX LOCAL TAX TOTAL
- 298.61 30.00 26.29 0.00 354.90
INSURED RAY DEAN II AND SUSAN ROCKHOLD
LICENSE P0066762
ADDRESS 10308 DONNA AVE EMAIL PROLABSALE@AOL.COM
CITY & STATE NORTHRIDGE, CA 91326 TELEPHONE 8184453524
PAYMENT DEADLINE
INSURANCE COMPANY LIEN HOLDER
IMMEDIATE
ITEM YEAR MAKE BODY SERIAL No. PLATES
VEHICLE 2017 CHRYSLER PACIFICA 2C4RC1DG7HR654698 8BPX206
TRAILER . .
TOWING . .
*** VALUE STATED MUST NOT EXCEED MARKET VALUE ***
***VEHICLES THAT HAVE BEEN ACQUIRED AS SALVAGE, REBUILT, OR HAVE BEEN USED PREVIOUSLY AS A TAXI WILL BE CONSIDERED WITH A REDUCED VALUE OF 35% (thirty-five percent), TAKING
AS A BASE THE VALUE OF A SIMILAR NORMAL VEHICLE, THAT IS, ONE THAT HAS NOT BEEN ACQUIRED AS SALVAGE AND ITS PREVIOUS USE HAS NOT BEEN AS A TAXI OR REBUILT. IT WILL BE THE
SOLE OBLIGATION AND RESPONSIBILITY OF THE INSURED TO DECLARATE THIS WHEN ACQUIRING THE POLICY.
SECTION SPECIFICATION OF RISKS LIMIT OF LIABILITY
MATERIAL DAMAGE WITH MANDATORY DEDUCTIBLE COVERED/EXCLUDED VEHICLE 8,000.00 DLLS.
1 DEDUCTIBLE: WITH MINIMUM OF $500.00 ON AUTOS
TRAILER
(SEDANS, COUPES, CONVERTIBLES AND STATION WAGONS) COVERED
AND $500.00 ON ALL OTHERS (PICK UPS, VANS, SUV´s AND MOTOR HOMES). 0.00 DLLS.
TOTAL THEFT WITH MANDATORY DEDUCTIBLE COVERED/EXCLUDED TOWING
2 DEDUCTIBLE: WITH MINIMUM OF $1,000.00 ON AUTOS 0.00 DLLS.
(SEDANS, COUPES, CONVERTIBLES AND STATION WAGONS) COVERED
AND $1,000.00 ON ALL OTHERS (PICK UPS, VANS, SUV´s AND MOTOR HOMES).
LIABILITY FOR PROPERTY DAMAGE TO THIRD PARTIES
3 100,000.00 DLLS.
BODILY INJURY LIABILITY PER PER
4 PERSON 100,000.00 ACCIDENT 200,000.00 DLLS.
MEDICAL EXPENSES PER PER
5 PERSON 5,000.00 ACCIDENT 25,000.00 DLLS.
COVERED/EXCLUDED PREMIUM
6 A.N.A.'s LEGAL AID
COVERED 40.00
COVERED/EXCLUDED PREMIUM
7 A.N.A.'s ROADSIDE ASSISTANCE
COVERED 40.00
CATASTROPHIC LIABILITY FOR DEATH OF THIRD PREMIUM
8 EXCLUDED
PARTIES DLLS. 0.00
ELITE OR ELITE PLUS WITH MANDATORY DEDUCTIBLE COVERED/EXCLUDED
9 PARTIAL THEFT (LIMIT 0.00 DLLS.WITH DEDUCTIBLE: 0.00 DLLS. PER EVENT) 0.00
VANDALISM (LIMIT 0.00 DLLS.WITH DEDUCTIBLE: 0.00 DLLS. PER EVENT) EXCLUDED
ISSUED ONLINE`);
const ANA_AUTO_RC_DIAS = page(`A.N.A. COMPAÑIA DE SEGUROS SA DE CV
LUIS CABRERA #2033 INT. 201, Col. ZONA URBANA RIO TIJUANA
C.P. 22010 MUNICIPIO DE TIJUANA, BAJA CALIFORNIA
www.anaseguros.com.mx
AUTOMOBILE
ALL CLAIMS MUST BE REPORTED BEFORE LEAVING MEXICO
U.S. CELL PHONES TRY + 011-52-55-5322-82-66 MEXICAN CELL PHONES 800-911-911-9 SPECIAL POLICY FOR TOURISTS
TOLL-FREE FROM THE U.S.A. 888-335-7072 BELIZE CELL PHONES 00-52-55-5322-8266
WHATSAPP + 52-55-80-50-3633
No. 700487807
ISSUED BY: DATE ISSUED TERM OF INSURANCE
DAYS
JORGE HUMBERTO CUADROS DIARIA DAY MONTH YEAR DAY MONTH YEAR TIME
BENITO JUAREZ 25 NO50 INT 38 COL CENTRO
22 07 2026 FROM 23 07 2026 12:01
3
ROSARITO BAJA CALIFORNIA 22710
(661) 612 12 55 70175 TO 26 07 2026 12:01
DISCOUNT PREMIUM POLICY FEE TAX LOCAL TAX TOTAL
- 10.77 25.00 2.86 0.00 38.63
INSURED STEPHEN RUPAN SHATAFIAN
LICENSE C1394198
ADDRESS 13181 CROSSROADS PARKWAY NORTH STE 300 EMAIL sshatafian@lee-associates.com
CITY & STATE CITY OF INDUSTRY, CA 91746 TELEPHONE 7143221072
PAYMENT DEADLINE
INSURANCE COMPANY LIEN HOLDER
IMMEDIATE
ITEM YEAR MAKE BODY SERIAL No. PLATES
VEHICLE 2022 FORD TRANSIT 1FBAX2CG3NKA69091 EC46T99
TRAILER . .
TOWING . .
*** VALUE STATED MUST NOT EXCEED MARKET VALUE ***
***VEHICLES THAT HAVE BEEN ACQUIRED AS SALVAGE, REBUILT, OR HAVE BEEN USED PREVIOUSLY AS A TAXI WILL BE CONSIDERED WITH A REDUCED VALUE OF 35% (thirty-five percent), TAKING
AS A BASE THE VALUE OF A SIMILAR NORMAL VEHICLE, THAT IS, ONE THAT HAS NOT BEEN ACQUIRED AS SALVAGE AND ITS PREVIOUS USE HAS NOT BEEN AS A TAXI OR REBUILT. IT WILL BE THE
SOLE OBLIGATION AND RESPONSIBILITY OF THE INSURED TO DECLARATE THIS WHEN ACQUIRING THE POLICY.
SECTION SPECIFICATION OF RISKS LIMIT OF LIABILITY
MATERIAL DAMAGE WITH MANDATORY DEDUCTIBLE COVERED/EXCLUDED VEHICLE 0.00 DLLS.
1 DEDUCTIBLE: ON AUTOS (SEDANS, COUPES, CONVERTIBLES AND
TRAILER
STATION WAGONS) AND OTHERS (PICK UPS, VANS, EXCLUDED
SUV´s AND MOTOR HOMES). 0.00 DLLS.
TOTAL THEFT WITH MANDATORY DEDUCTIBLE COVERED/EXCLUDED TOWING
2 DEDUCTIBLE: ON AUTOS (SEDANS, COUPES, CONVERTIBLES AND 0.00 DLLS.
STATION WAGONS) AND OTHERS (PICK UPS, VANS, EXCLUDED
SUV´s AND MOTOR HOMES).
LIABILITY FOR PROPERTY DAMAGE TO THIRD PARTIES
3 100,000.00 DLLS.
BODILY INJURY LIABILITY PER PER
4 PERSON 100,000.00 ACCIDENT 200,000.00 DLLS.
MEDICAL EXPENSES PER PER
5 PERSON 5,000.00 ACCIDENT 25,000.00 DLLS.
COVERED/EXCLUDED PREMIUM
6 A.N.A.'s LEGAL AID
COVERED 2.25
COVERED/EXCLUDED PREMIUM
7 A.N.A.'s ROADSIDE ASSISTANCE
COVERED 2.25
CATASTROPHIC LIABILITY FOR DEATH OF THIRD PREMIUM
8 EXCLUDED
PARTIES DLLS. 0.00
ELITE OR ELITE PLUS WITH MANDATORY DEDUCTIBLE COVERED/EXCLUDED
9 PARTIAL THEFT (LIMIT 0.00 DLLS.WITH DEDUCTIBLE: 0.00 DLLS. PER EVENT) 0.00
VANDALISM (LIMIT 0.00 DLLS.WITH DEDUCTIBLE: 0.00 DLLS. PER EVENT) EXCLUDED
ISSUED ONLINE`);
const ANA_LICENCIA = page(`A.N.A. COMPAÑIA DE SEGUROS SA DE CV
LUIS CABRERA #2033 INT. 201, Col.4 ZONA URBANA RIO TIJUANA
C.P. 22010 MUNICIPIO DE TIJUANA, BAJA CALIFORNIA
www.anaseguros.com.mx
DRIVER´S POLICY FOR AUTOMOBILE
ALL CLAIMS MUST BE REPORTED BEFORE LEAVING MEXICO
U.S. CELL PHONES TRY + 011-52-55-5322-82-66 MEXICAN CELL PHONES 800-911-911-9
SPECIAL POLICY FOR TOURISTS
TOLL-FREE FROM THE U.S.A. 888-335-7072 BELIZE CELL PHONES 00-52-55-5322-8266
WHATSAPP + 52-55-80-50-3633 No. 700489616
ISSUED BY: DATE ISSUED & TIME TERM OF INSURANCE
JORGE HUMBERTO CUADROS
DAYS
DAY MONTH YEAR DAY MONTH YEAR TIME
BENITO JUAREZ 25 No.50 INT 38 CENTRO 04 08 2026 FROM 06 08 2026 12:01
365
ROSARITO, BAJA CALIFORNIA 22710 TO 06 08 2027 12:01
. 70175
DISCOUNT PREMIUM POLICY FEE TAX LOCAL TAX TOTAL
- 142.78 30.00 13.82 0.00 186.60
LICENSE N0017668 EMAIL PWAGONER49@AOL.COM TELEPHONE 3102001538
POLICY HOLDER
1. NAME : PAMELA DENISE WAGONER Ph.3102001538
ADDRESS : 49305 HIGHWAY 74 SPC 10, PALM DESERT, CA, 92260,
DRIVER LICENSE : N0017668
2. NAME :
ADDRESS :
DRIVER LICENSE :
NONE
3. NAME :
ADDRESS :
DRIVER LICENSE :
NONE
4. NAME :
ADDRESS :
DRIVER LICENSE : NONE
5. NAME :
ADDRESS :
DRIVER LICENSE : NONE
SPECIFICATION OF RISKS SUM INSURED PREMIUM
LIABILITY FOR PROPERTY DAMAGE TO THIRD PARTIES 100,000.00 usd. 18.70 usd.
BODILY INJURY LIABILITY ( EXCLUDING OCCUPANTS OF THE VEHICLE ) 100,000.00 usd. Per Person
54.27 usd.
200,000.00 usd. Per Accident
CATASTROPHIC LIABILITY FOR DEATH OF THIRD PARTIES 0.00 usd. 0.00 usd.
MEDICAL EXPENSES 4,000.00 usd. Per Person
9.81 usd.
20,000.00 usd. Per Accident
COVERED/EXCLUDED PREMIUM
LEGAL AID
COVERED 30.00 usd.
COVERED/EXCLUDED PREMIUM
AUTOMOBILE ASSISTANCE
COVERED 30.00 usd.
The following risks are excluded Collision, overtuning and glass breakage, fire, total theft and natural disasters, partial theft and vandalism.`);
describe("detectPolicyProvider / ANA", () => {
it("claims ANA from the letterhead", () => {
expect(
detectPolicyProvider("A.N.A. COMPAÑIA DE SEGUROS SA DE CV\nwww.anaseguros.com.mx"),
).toBe("ANA");
});
it("does not let GMX's layout rules claim an ANA page", () => {
// Both books print "MATERIAL DAMAGE"-ish headings; the brand pass runs
// before any layout rule precisely so this can't go the other way.
expect(detectPolicyProvider(ANA_AUTO_AMPLIA.text)).toBe("ANA");
expect(detectPolicyProvider(ANA_LICENCIA.text)).toBe("ANA");
});
});
describe("parsePolicy / ANA automobile", () => {
const p = parsePolicy(ANA_AUTO_AMPLIA);
it("reads the header band", () => {
expect(p.provider).toBe("ANA");
expect(p.policyNumber).toBe("700489651");
expect(p.insuredName).toBe("RAY DEAN II AND SUSAN ROCKHOLD");
expect(p.agentName).toBe("JORGE HUMBERTO CUADROS");
expect(p.legalAddress).toBe("10308 DONNA AVE, NORTHRIDGE, CA 91326");
expect(p.zip).toBe("91326");
expect(p.currency).toBe("USD");
expect(p.premiumPayment).toBe("IMMEDIATE");
});
it("reads DD MM YYYY out of the three date column cells", () => {
expect(p.policyDate?.toISOString().slice(0, 10)).toBe("2026-08-04");
expect(p.policyFrom?.toISOString().slice(0, 10)).toBe("2026-08-07");
expect(p.policyTo?.toISOString().slice(0, 10)).toBe("2027-08-07");
});
it("maps the six money cells positionally, not by finding six amounts", () => {
// DISCOUNT prints as a bare "-" here. A "take the amounts in order"
// reading would shift every value one column left.
expect(p.netPremium).toBe(298.61);
expect(p.policyFee).toBe(30);
expect(p.tax).toBe(26.29);
expect(p.total).toBe(354.9);
});
it("reads a TAX that reconciles against the rest of the row", () => {
// 298.61 + 30.00 = 328.61, taxed at 8% -> 26.29, totalling 354.90. The
// whole row agreeing is what proves the positional mapping landed on the
// right cells rather than merely on six numbers.
const base = p.netPremium! + p.policyFee!;
expect(Math.round(base * 0.08 * 100) / 100).toBe(p.tax);
expect(Math.round((base + p.tax!) * 100) / 100).toBe(p.total);
});
it("does not fold LOCAL TAX into the IVA", () => {
// It prints 0.00 here, so nothing to fold — but the guard is that a
// non-zero one would surface as a note instead of inflating `tax`.
expect(p.notes.join(" | ")).not.toMatch(/impuesto local/);
});
it("reads the vehicle by token role, not by column", () => {
expect(p.vehicles).toHaveLength(1);
expect(p.vehicles[0]).toEqual({
item: "VEHICLE",
modelYear: "2017",
make: "CHRYSLER",
bodyType: "PACIFICA",
vinNumber: "2C4RC1DG7HR654698",
licensePlate: "8BPX206",
});
});
it("reads a two-word BODY cell without losing the VIN", () => {
// "GENESIS SEDAN" is two tokens where "PACIFICA" is one — the VIN shape
// is the anchor, not the token count.
const v = parsePolicy(ANA_AUTO_RC_DIAS).vehicles[0];
expect(v.make).toBe("FORD");
expect(v.vinNumber).toBe("1FBAX2CG3NKA69091");
expect(v.licensePlate).toBe("EC46T99");
});
it("skips the empty TRAILER and TOWING slots", () => {
// Both print a "." per cell rather than being absent.
expect(p.vehicles.map((v) => v.item)).toEqual(["VEHICLE"]);
});
it("records the insured as a named driver with their licence", () => {
expect(p.drivers).toHaveLength(1);
expect(p.drivers[0].fullName).toBe("RAY DEAN II AND SUSAN ROCKHOLD");
expect(p.drivers[0].licenseNumber).toBe("P0066762");
expect(p.drivers[0].email).toBe("PROLABSALE@AOL.COM");
});
it("does not read the agent's own street number as the policy number", () => {
// "BENITO JUAREZ 25 No.50 INT 38" sits three lines above the No. cell.
expect(p.policyNumber).not.toBe("50");
expect(p.notes.join(" | ")).not.toMatch(/formas/);
});
it("reads the agent clave without picking up their postal code", () => {
// "ROSARITO, BAJA CALIFORNIA 22710" is five digits in the same band.
expect(p.notes.join(" | ")).toMatch(/clave de agente: 70175/);
expect(p.notes.join(" | ")).not.toMatch(/22710/);
});
it("labels the declared values by their printed item slot", () => {
const c = byRisk(p);
expect(c["MATERIAL DAMAGE — VEHICLE"]?.insuredAmount).toBe(8000);
expect(c["MATERIAL DAMAGE — TRAILER"]?.insuredAmount).toBe(0);
expect(c["TOTAL THEFT — TOWING"]?.insuredAmount).toBe(0);
});
it("keeps the deductible sentence out of the value columns", () => {
const c = byRisk(p);
expect(c["MATERIAL DAMAGE — VEHICLE"]?.deductible).toBe(
"WITH MINIMUM OF $500.00 ON AUTOS (SEDANS, COUPES, CONVERTIBLES AND " +
"STATION WAGONS) AND $500.00 ON ALL OTHERS (PICK UPS, VANS, SUV´s AND " +
"MOTOR HOMES).",
);
});
it("does not mistake the $500.00 inside the deductible for a sum insured", () => {
// It is the one amount in the block not suffixed "DLLS.".
const amounts = p.coverages.map((c) => c.insuredAmount);
expect(amounts).not.toContain(500);
});
it("splits the per-person and per-accident limits", () => {
const c = byRisk(p);
expect(c["BODILY INJURY LIABILITY — POR PERSONA"]?.insuredAmount).toBe(100000);
expect(c["BODILY INJURY LIABILITY — POR EVENTO"]?.insuredAmount).toBe(200000);
expect(c["MEDICAL EXPENSES — POR PERSONA"]?.insuredAmount).toBe(5000);
expect(c["MEDICAL EXPENSES — POR EVENTO"]?.insuredAmount).toBe(25000);
});
it("records an add-on's figure as a premium, never as a sum insured", () => {
// $40 is what legal aid COST. As `insuredAmount` it would read on the
// review screen as a $40 liability limit.
const c = byRisk(p);
expect(c["LEGAL AID"]?.premium).toBe(40);
expect(c["LEGAL AID"]?.insuredAmount).toBeNull();
expect(c["ROADSIDE ASSISTANCE"]?.premium).toBe(40);
});
it("unpacks section 9's parenthesised limit and deductible", () => {
const c = byRisk(p);
const theft = c["ELITE / ELITE PLUS — PARTIAL THEFT: EXCLUDED"];
expect(theft?.insuredAmount).toBe(0);
expect(theft?.deductible).toBe("0.00 DLLS. POR EVENTO");
expect(c["ELITE / ELITE PLUS — VANDALISM: EXCLUDED"]).toBeDefined();
});
it("emits each coverage once even though the PDF prints the face twice", () => {
// The real upload is ORIGINAL + AGENT COPY + receipt + three travel
// cards, all concatenated into one string before parsing.
const doubled = page(ANA_AUTO_AMPLIA.text + "\n\n" + ANA_AUTO_AMPLIA.text);
expect(parsePolicy(doubled).coverages).toHaveLength(p.coverages.length);
expect(parsePolicy(doubled).vehicles).toHaveLength(1);
});
});
describe("policy type, as a name for confirm to resolve", () => {
it("names ANA's two faces after the legacy tables they belong to", () => {
expect(parsePolicy(ANA_AUTO_AMPLIA).policyTypeName).toBe("AUTO");
expect(parsePolicy(ANA_AUTO_RC_DIAS).policyTypeName).toBe("AUTO");
expect(parsePolicy(ANA_LICENCIA).policyTypeName).toBe("LICENCIAS");
});
it("emits a NAME, never an id — the parser must not need a database", () => {
// Anything id-shaped here would mean the parser had reached for the DB.
for (const p of [ANA_AUTO_AMPLIA, ANA_AUTO_RC_DIAS, ANA_LICENCIA]) {
expect(parsePolicy(p).policyTypeName).toMatch(/^[A-Z_]+$/);
}
});
it("leaves the type unnamed when no parser claimed the page", () => {
expect(parsePolicy(page("a laundry receipt")).policyTypeName).toBeNull();
});
});
describe("parsePolicy / ANA responsabilidad civil por días", () => {
const p = parsePolicy(ANA_AUTO_RC_DIAS);
it("reads a by-the-day term rather than defaulting to a year", () => {
// Left at the schema's 365 default this weekend policy would sit in the
// renewals window a year out.
expect(p.policyFrom?.toISOString().slice(0, 10)).toBe("2026-07-23");
expect(p.policyTo?.toISOString().slice(0, 10)).toBe("2026-07-26");
expect(p.coveragePeriodDays).toBe(3);
});
it("reads the clave when the agent's phone occupies the left cell", () => {
// The by-the-day products print "(661) 612 12 55" ahead of the clave, so
// it is no longer the first thing on its line.
expect(p.notes.join(" | ")).toMatch(/clave de agente: 70175/);
});
it("marks the excluded sections as excluded, not as insured for zero", () => {
const risks = p.coverages.map((c) => c.risk);
expect(risks).toContain("MATERIAL DAMAGE — VEHICLE: EXCLUDED");
expect(risks).toContain("TOTAL THEFT — TOWING: EXCLUDED");
// The liability sections are what this product actually sells, and they
// are NOT excluded.
expect(risks).toContain("LIABILITY FOR PROPERTY DAMAGE TO THIRD PARTIES");
});
});
describe("parsePolicy / ANA driver's policy (licencia)", () => {
const p = parsePolicy(ANA_LICENCIA);
it("reads the holder off the numbered POLICY HOLDER list", () => {
expect(p.policyNumber).toBe("700489616");
expect(p.insuredName).toBe("PAMELA DENISE WAGONER");
expect(p.legalAddress).toBe("49305 HIGHWAY 74 SPC 10, PALM DESERT, CA, 92260");
expect(p.zip).toBe("92260");
});
it("lists one driver, not one per printed copy of the page", () => {
// The face renders three times in the real PDF; an unbounded walk
// returns the same person three times, which reads as a three-driver
// policy rather than as a parse bug.
const tripled = page([ANA_LICENCIA.text, ANA_LICENCIA.text, ANA_LICENCIA.text].join("\n\n"));
expect(p.drivers).toHaveLength(1);
expect(parsePolicy(tripled).drivers).toHaveLength(1);
});
it("splits the phone off the name even without the printed column gap", () => {
// The phone shares the name cell, and the only thing marking it off is
// white space — which the OCR seam is free to collapse. Depending on the
// gap surviving is what put "PAMELA DENISE WAGONER Ph.3102001538" in the
// insured field, where it matched no customer.
const collapsed = page(ANA_LICENCIA.text.replace(/ {2,}/g, " "));
expect(parsePolicy(collapsed).insuredName).toBe("PAMELA DENISE WAGONER");
});
it("drops the four empty driver slots", () => {
// Slots 2-5 print an empty NAME and a bare "NONE" licence.
expect(p.drivers.map((d) => d.fullName)).toEqual(["PAMELA DENISE WAGONER"]);
expect(p.drivers[0].licenseNumber).toBe("N0017668");
expect(p.drivers[0].phone).toBe("3102001538");
});
it("insures no vehicle", () => {
expect(p.vehicles).toEqual([]);
expect(p.notes.join(" | ")).toMatch(/no ampara un veh[íi]culo determinado/);
});
it("separates the SUM INSURED and PREMIUM columns by position", () => {
// Both columns print the same shape ("100,000.00 usd." / "18.70 usd.")
// and neither is labelled per row — only the offset tells them apart.
const c = byRisk(p);
const pd = c["LIABILITY FOR PROPERTY DAMAGE TO THIRD PARTIES"];
expect(pd?.insuredAmount).toBe(100000);
expect(pd?.premium).toBe(18.7);
});
it("reads the trailing Per Person / Per Accident labels on this layout", () => {
// They FOLLOW their amount here and PRECEDE it on the automobile face.
const c = byRisk(p);
expect(c["BODILY INJURY LIABILITY — POR PERSONA"]?.insuredAmount).toBe(100000);
expect(c["BODILY INJURY LIABILITY — POR EVENTO"]?.insuredAmount).toBe(200000);
expect(c["MEDICAL EXPENSES — POR PERSONA"]?.insuredAmount).toBe(4000);
expect(c["MEDICAL EXPENSES — POR EVENTO"]?.insuredAmount).toBe(20000);
});
it("charges a section's premium once, not once per limit", () => {
const c = byRisk(p);
expect(c["BODILY INJURY LIABILITY — POR PERSONA"]?.premium).toBe(54.27);
expect(c["BODILY INJURY LIABILITY — POR EVENTO"]?.premium).toBeNull();
});
it("handles the section order this layout uses", () => {
// CATASTROPHIC LIABILITY prints ABOVE MEDICAL EXPENSES here and below it
// on the automobile face; blocks are keyed by where the labels land.
const c = byRisk(p);
expect(c["CATASTROPHIC LIABILITY FOR DEATH OF THIRD PARTIES"]?.insuredAmount).toBe(0);
expect(c["LEGAL AID"]?.premium).toBe(30);
expect(c["ROADSIDE ASSISTANCE"]?.premium).toBe(30);
});
it("carries the excluded-risk sentence that defines the product", () => {
expect(p.notes.join(" | ")).toMatch(/riesgos excluidos: Collision, overtuning/);
});
});
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,100 @@
import { PolicyMatcherService } from "./policy-matcher.service";
import type { PrismaService } from "../prisma/prisma.service";
import type { ParsedPolicy } from "./parsers/policy-parser";
function parsed(over: Partial<ParsedPolicy> = {}): ParsedPolicy {
return {
provider: "GMX",
policyNumber: null,
insuredName: null,
notes: [],
coverages: [],
vehicles: [],
drivers: [],
...over,
} as unknown as ParsedPolicy;
}
function prismaStub(policies: unknown[], customers: { id: string; name: string }[]) {
const findManyPolicy = jest.fn().mockResolvedValue(policies);
const findManyCustomer = jest.fn().mockResolvedValue(customers);
return {
prisma: {
policy: { findMany: findManyPolicy },
customer: { findMany: findManyCustomer },
} as unknown as PrismaService,
findManyPolicy,
findManyCustomer,
};
}
const BOOK = [
{ id: "cust-1", name: "WAGONER, PAMELA" },
{ id: "cust-2", name: "SMITH, JOHN" },
];
describe("PolicyMatcherService name suggestions", () => {
it("suggests a customer when the policy number is new", async () => {
const { prisma } = prismaStub([], BOOK);
const svc = new PolicyMatcherService(prisma);
const r = await svc.match(
parsed({ policyNumber: "P-999", insuredName: "PAMELA DENISE WAGONER" } as never),
);
expect(r.customerSuggestions).toEqual([
expect.objectContaining({ customerId: "cust-1", tier: "PARTIAL" }),
]);
// The suggestion is surfaced, never applied.
expect(r.customerId).toBeNull();
expect(r.confident).toBe(false);
expect(r.note).toContain("posibles clientes por nombre: WAGONER, PAMELA");
});
it("suggests when the policy number could not be read at all", async () => {
const { prisma } = prismaStub([], BOOK);
const svc = new PolicyMatcherService(prisma);
const r = await svc.match(parsed({ insuredName: "PAMELA WAGONER" } as never));
expect(r.customerSuggestions[0]).toMatchObject({ customerId: "cust-1", tier: "EXACT" });
expect(r.customerId).toBeNull();
expect(r.note).toBe(
"no se pudo leer el número de póliza; posible cliente por nombre: WAGONER, PAMELA",
);
});
it("does not touch the book when the policy number hits", async () => {
const { prisma, findManyCustomer } = prismaStub(
[
{
id: "pol-1",
policyNumber: "P-1",
customerId: "cust-2",
customer: { name: "SMITH, JOHN" },
},
],
BOOK,
);
const svc = new PolicyMatcherService(prisma);
const r = await svc.match(
parsed({ policyNumber: "P-1", insuredName: "PAMELA WAGONER" } as never),
);
expect(r.confident).toBe(true);
expect(r.customerId).toBe("cust-2");
expect(r.customerSuggestions).toEqual([]);
expect(findManyCustomer).not.toHaveBeenCalled();
});
it("reads the customer book once across a batch", async () => {
const { prisma, findManyCustomer } = prismaStub([], BOOK);
const svc = new PolicyMatcherService(prisma);
await svc.match(parsed({ policyNumber: "A", insuredName: "PAMELA WAGONER" } as never));
await svc.match(parsed({ policyNumber: "B", insuredName: "JOHN SMITH" } as never));
expect(findManyCustomer).toHaveBeenCalledTimes(1);
});
});
@@ -1,6 +1,12 @@
import { Injectable } from "@nestjs/common";
import { PrismaService } from "../prisma/prisma.service";
import type { ParsedPolicy } from "./parsers/policy-parser";
import {
suggestCustomersByName,
suggestionNote,
type CustomerNameRow,
type CustomerNameSuggestion,
} from "./name-matcher";
export interface MatchResult {
policyId: string | null;
@@ -14,8 +20,24 @@ export interface MatchResult {
* the policy number is shared across customers and a human must pick.
*/
candidates: { policyId: string; customerId: string; customerName: string; policyNumber: string }[];
/**
* Customers whose name resembles the printed insured name. Populated only
* when the policy number resolved to nothing, and never used to set
* `customerId` or `confident` — see the class comment.
*/
customerSuggestions: CustomerNameSuggestion[];
}
/**
* How long the customer book is reused across documents in a batch.
*
* A twenty-page batch would otherwise read all 1536 rows twenty times. The
* only cost of the staleness is that a customer created in the last minute
* is not suggested — the picker still finds them, so nothing is lost that a
* reviewer cannot do in one click.
*/
const BOOK_TTL_MS = 60_000;
/**
* Resolves a parsed policy page to an existing Policy (and its customer) the
* office already holds.
@@ -33,14 +55,29 @@ export interface MatchResult {
* policy numbers across customers do occur (same group policy bound by two
* related parties), and picking one arbitrarily would silently book the
* wrong coverage.
*
* On that zero-hit path only, the printed name is used to *rank the picker*
* — see `name-matcher.ts`. That is not a walk-back of the rule above: the
* suggestion never reaches `customerId` or `confident`, a human still picks,
* and the ranking exists because the office writes names surname-first
* ("WAGONER, PAMELA") while carriers print them given-name-first ("PAMELA
* DENISE WAGONER"), so the reviewer is retyping a name the machine could
* have offered.
*/
@Injectable()
export class PolicyMatcherService {
private book: { rows: CustomerNameRow[]; loadedAt: number } | null = null;
constructor(private readonly prisma: PrismaService) {}
async match(parsed: ParsedPolicy): Promise<MatchResult> {
if (!parsed.policyNumber) {
return this.unmatched("no se pudo leer el número de póliza");
// No number to search on, so the page goes to review with a picker —
// the same place the name suggestions help.
return this.unmatched(
"no se pudo leer el número de póliza",
await this.suggestByName(parsed.insuredName),
);
}
const rows = await this.prisma.policy.findMany({
@@ -61,22 +98,33 @@ export class PolicyMatcherService {
}));
if (rows.length === 0) {
const suggestions = await this.suggestByName(parsed.insuredName);
const hint = suggestionNote(suggestions);
return {
policyId: null,
customerId: null,
note: `no se encontró ninguna póliza con el número ${parsed.policyNumber}`,
note: [
`no se encontró ninguna póliza con el número ${parsed.policyNumber}`,
hint,
]
.filter(Boolean)
.join("; "),
confident: false,
candidates: [],
customerSuggestions: suggestions,
};
}
if (rows.length > 1) {
// The policy number did find rows; the reviewer picks among those, and
// adding name guesses on top would only add noise.
return {
policyId: null,
customerId: null,
note: `${rows.length} pólizas comparten el número ${parsed.policyNumber}`,
confident: false,
candidates,
customerSuggestions: [],
};
}
@@ -86,16 +134,47 @@ export class PolicyMatcherService {
note: `coincidencia exacta por número de póliza ${parsed.policyNumber}`,
confident: true,
candidates,
customerSuggestions: [],
};
}
private unmatched(note: string): MatchResult {
private async suggestByName(
insuredName: string | null | undefined,
): Promise<CustomerNameSuggestion[]> {
if (!insuredName) return [];
return suggestCustomersByName(insuredName, await this.customerBook());
}
/**
* The whole customer book, held briefly. 1536 rows of `{id, name}` is a
* few hundred kilobytes and the comparison is pure token-set work, so
* scanning it beats any SQL approximation — and a `LIKE` search would in
* any case have to guess which token is the surname, which is the one
* thing the office's own data does not agree on.
*/
private async customerBook(): Promise<CustomerNameRow[]> {
if (this.book && Date.now() - this.book.loadedAt < BOOK_TTL_MS) {
return this.book.rows;
}
const rows = await this.prisma.customer.findMany({
select: { id: true, name: true },
});
this.book = { rows, loadedAt: Date.now() };
return rows;
}
private unmatched(
note: string,
customerSuggestions: CustomerNameSuggestion[] = [],
): MatchResult {
const hint = suggestionNote(customerSuggestions);
return {
policyId: null,
customerId: null,
note,
note: [note, hint].filter(Boolean).join("; "),
confident: false,
candidates: [],
customerSuggestions,
};
}
}
+14
View File
@@ -3,10 +3,13 @@ import {
IsArray,
IsDateString,
IsEnum,
IsInt,
IsNumber,
IsObject,
IsOptional,
IsString,
Max,
Min,
MinLength,
ValidateNested,
} from "class-validator";
@@ -19,6 +22,11 @@ export class ConfirmPolicyDocumentDto {
/** Required when creating a new Policy; ignored if `policyId` is set. */
@IsOptional() @IsString() customerId?: string;
/** Reviewer's explicit lookup picks. Both beat the parsed name; omitted,
* the service resolves `policy_types` / `insurance_providers` by name and
* leaves the FK null when there is no such row. */
@IsOptional() @IsString() policyTypeId?: string;
@IsOptional() @IsString() insuranceProviderId?: string;
/** Set when the document matched an existing Policy. */
@IsOptional() @IsString() policyId?: string;
@@ -35,8 +43,12 @@ export class ConfirmPolicyDocumentDto {
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() brokerFee?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsString() premiumPayment?: string;
/** Printed term in days. Omitted leaves the parsed value (or the schema's
* 365 default) in place; ANA sells 3- and 4-day tourist policies. */
@IsOptional() @IsInt() @Min(1) @Max(3660) coveragePeriodDays?: number;
/** Coverages parsed off the PDF, passed through verbatim to Policy.coveragesJson. */
@IsOptional() @IsObject() coveragesJson?: unknown;
@@ -68,8 +80,10 @@ export class ReviewPolicyDocumentDto {
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() brokerFee?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsString() premiumPayment?: string;
@IsOptional() @IsInt() @Min(1) @Max(3660) coveragePeriodDays?: number;
@IsOptional() @IsObject() coveragesJson?: unknown;
/** Set by the reviewer when the document matched an existing Policy. */
+256 -13
View File
@@ -10,7 +10,7 @@ import { PrismaService } from "../prisma/prisma.service";
import { StorageService } from "../storage/storage.service";
import type { UploadedFileLike } from "../storage/upload-file";
import { OCR_PROVIDER, type OcrPage, type OcrProvider } from "../statements/ocr/ocr.provider";
import { parsePolicy } from "./parsers/policy-parser";
import { parsePolicy, type ParsedDriver, type ParsedVehicle } from "./parsers/policy-parser";
import { PolicyMatcherService } from "./policy-matcher.service";
import type {
ConfirmPolicyBatchDto,
@@ -69,8 +69,11 @@ export class PolicyOcrService {
);
}
// The provider is not asked of the uploader and not assumed: `process`
// sets it from what the parsers actually claimed, so the batch label can
// never contradict its own documents. Until then it says so.
const batch = await this.prisma.policyOcrBatch.create({
data: { provider: "GMX", uploadedById, label, fileCount: files.length },
data: { provider: "por detectar", uploadedById, label, fileCount: files.length },
});
const copies = files.map((f) => ({ buffer: f.buffer, name: f.originalname }));
@@ -112,6 +115,7 @@ export class PolicyOcrService {
let fileOrdinal = 0;
let globalPageOrdinal = 0;
const providersSeen = new Set<string>();
for (const file of files) {
fileOrdinal += 1;
const sourceKey = `policy-ocr/${batchId}/source-${fileOrdinal}.pdf`;
@@ -155,6 +159,7 @@ export class PolicyOcrService {
if (parsed.provider === "") {
throw new Error("no se reconoció el proveedor");
}
providersSeen.add(parsed.provider);
const match = await this.matcher.match(parsed);
const notes = [...parsed.notes, match.note].filter(Boolean);
// Confident when exactly one Policy carries the printed number —
@@ -187,18 +192,31 @@ export class PolicyOcrService {
parsed.policyFee != null ? new Prisma.Decimal(parsed.policyFee) : null,
extractedBrokerFee:
parsed.brokerFee != null ? new Prisma.Decimal(parsed.brokerFee) : null,
extractedTax:
parsed.tax != null ? new Prisma.Decimal(parsed.tax) : null,
extractedTotal:
parsed.total != null ? new Prisma.Decimal(parsed.total) : null,
extractedCoveragesJson: parsed.coverages.length
? (parsed.coverages as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
extractedPremiumPayment: parsed.premiumPayment,
extractedCoveragePeriodDays: parsed.coveragePeriodDays,
extractedVehiclesJson: parsed.vehicles.length
? (parsed.vehicles as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
extractedDriversJson: parsed.drivers.length
? (parsed.drivers as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
extractedPolicyTypeName: parsed.policyTypeName,
matchedPolicyId: match.policyId,
matchedCustomerId: match.customerId,
matchCandidates: match.candidates.length
? (match.candidates as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
matchNote: notes.join("; ").slice(0, 190),
customerSuggestions: match.customerSuggestions.length
? (match.customerSuggestions as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
matchNote: notes.join("; "),
},
});
} catch (err) {
@@ -211,7 +229,7 @@ export class PolicyOcrService {
pageNumber: fileOrdinal,
storageKey: sourceKey,
status: "OCR_FAILED",
matchNote: (err as Error).message.slice(0, 190),
matchNote: (err as Error).message,
},
});
}
@@ -219,7 +237,13 @@ export class PolicyOcrService {
await this.prisma.policyOcrBatch.update({
where: { id: batchId },
data: { status: "READY_FOR_REVIEW" },
data: {
status: "READY_FOR_REVIEW",
// Whatever the parsers claimed. A mixed upload is labelled as mixed
// rather than as whichever provider happened to come first — the
// review header is the only place staff see what they dropped in.
provider: [...providersSeen].sort().join(" + ") || "desconocido",
},
});
}
@@ -343,12 +367,14 @@ export class PolicyOcrService {
dto.policyFee != null ? new Prisma.Decimal(dto.policyFee) : undefined,
extractedBrokerFee:
dto.brokerFee != null ? new Prisma.Decimal(dto.brokerFee) : undefined,
extractedTax: dto.tax != null ? new Prisma.Decimal(dto.tax) : undefined,
extractedTotal:
dto.total != null ? new Prisma.Decimal(dto.total) : undefined,
extractedCoveragesJson: dto.coveragesJson
? (dto.coveragesJson as Prisma.InputJsonValue)
: undefined,
extractedPremiumPayment: dto.premiumPayment ?? undefined,
extractedCoveragePeriodDays: dto.coveragePeriodDays ?? undefined,
matchedPolicyId,
matchedCustomerId,
status: dto.forceConfirm ? "CONFIRMED" : "MATCHED",
@@ -447,14 +473,18 @@ export class PolicyOcrService {
);
}
// 1. Resolve target Policy (create or update). Field selection: every
// 1. Resolve the lookup rows the parser can only name. The reviewer's
// explicit pick always wins; the parsed name is the fallback.
const lookups = await this.resolveLookups(item, doc);
// 2. Resolve target Policy (create or update). Field selection: every
// non-null `extracted*` on the doc (post-review) is written. Null is
// preserved — never overwrite an existing Policy's `netPremium` with
// null because the certificate page didn't carry one.
let policyId = item.policyId ?? null;
if (policyId) {
const updateData = buildPolicyUpdateFromDoc(item, doc);
const updateData = buildPolicyUpdateFromDoc(item, doc, lookups);
await this.prisma.policy.update({
where: { id: policyId },
data: updateData,
@@ -467,21 +497,25 @@ export class PolicyOcrService {
`Documento página ${doc.pageNumber}: falta número de póliza.`,
);
}
const createData = buildPolicyCreateFromDoc(item, doc, item.customerId!);
const createData = buildPolicyCreateFromDoc(item, doc, item.customerId!, lookups);
const created = await this.prisma.policy.create({
data: createData,
});
policyId = created.id;
}
// 2. Attach the source PDF as a PolicyDocument. `doc.storageKey`
// 3. Vehicles and named drivers, for the providers whose face carries
// them (ANA's automobile and driver's policies; never GMX Hogar).
await this.applyVehiclesAndDrivers(doc, policyId);
// 4. Attach the source PDF as a PolicyDocument. `doc.storageKey`
// already points at the exact upload (`policy-ocr/{batchId}/source-N.pdf`)
// so the attach is just a stream copy into the policy's namespace —
// the previous per-page "which file did this page come from" walk is
// gone because one PDF = one doc now.
await this.attachSourcePdf(doc.storageKey, policyId);
await this.attachSourcePdf(doc.storageKey, policyId, doc.provider);
// 3. Optionally post the premium to the ledger. Only when staff
// 5. Optionally post the premium to the ledger. Only when staff
// explicitly asked (`postPremium` true) and netPremium parses — without
// that gate a missing premium would silently book $0.
let postedTransactionId: string | null = null;
@@ -539,13 +573,149 @@ export class PolicyOcrService {
};
}
/**
* Turn the two things the parser can only NAME into foreign keys.
*
* The parser is a pure function over text and never touches the database,
* so it emits `policyTypeName` ("AUTO") and `provider` ("ANA"). Resolving
* them here keeps that boundary and means a renamed lookup row is a data
* change rather than a parser change.
*
* **Resolve, never create.** A missing `policy_types` row is a signal that
* a human deleted it (that is exactly how M_EMPR disappeared), and silently
* recreating it would undo that decision with no record. The field stays
* null and the reviewer can add the row through the lookups screen.
*
* An explicit pick from the reviewer always beats the parsed name.
*/
private async resolveLookups(
item: ConfirmPolicyDocumentDto,
doc: { extractedPolicyTypeName: string | null; provider: string | null },
): Promise<{ policyTypeId?: string; insuranceProviderId?: string }> {
const out: { policyTypeId?: string; insuranceProviderId?: string } = {};
if (item.policyTypeId) {
out.policyTypeId = item.policyTypeId;
} else if (doc.extractedPolicyTypeName) {
const row = await this.prisma.policyType.findUnique({
where: { name: doc.extractedPolicyTypeName },
select: { id: true },
});
if (row) out.policyTypeId = row.id;
}
if (item.insuranceProviderId) {
out.insuranceProviderId = item.insuranceProviderId;
} else if (doc.provider) {
const name = PROVIDER_ROW_NAME[doc.provider] ?? doc.provider;
const row = await this.prisma.insuranceProvider.findFirst({
where: { name },
select: { id: true },
});
if (row) out.insuranceProviderId = row.id;
}
return out;
}
/**
* Write the parsed `Vehicle` and `InsuredDriver` rows onto the policy.
*
* Both inserts are skipped when an equivalent row is already on the policy.
* The reason is `confirmBatch` applying to an EXISTING policy: the office
* uploads a renewal for a car already on file, and a blind insert would
* leave the customer with the same VIN listed twice with no way to tell
* which row the renewal belongs to. Matching is on the identifier the
* document actually prints — the VIN for a vehicle (falling back to the
* plate, since ANA's TRAILER/TOWING slots have no VIN), the licence number
* for a driver (falling back to the name).
*
* Nothing is ever updated or deleted here. A vehicle whose plate changed
* lands as a second row for a human to reconcile, which is the safe half
* of the mistake: an over-write would destroy the only record of what was
* insured last term.
*/
private async applyVehiclesAndDrivers(
doc: { extractedVehiclesJson: Prisma.JsonValue | null; extractedDriversJson: Prisma.JsonValue | null },
policyId: string,
): Promise<void> {
const vehicles = asArray<ParsedVehicle>(doc.extractedVehiclesJson);
const drivers = asArray<ParsedDriver>(doc.extractedDriversJson);
if (vehicles.length === 0 && drivers.length === 0) return;
const policy = await this.prisma.policy.findUnique({
where: { id: policyId },
select: { customerId: true },
});
if (!policy) return;
if (vehicles.length) {
const existing = await this.prisma.vehicle.findMany({
where: { policyId },
select: { vinNumber: true, licensePlate: true },
});
const seen = new Set(
existing.flatMap((v) =>
[v.vinNumber, v.licensePlate].filter((k): k is string => !!k).map(norm),
),
);
for (const v of vehicles) {
const key = norm(v.vinNumber ?? v.licensePlate ?? "");
if (!key || seen.has(key)) continue;
seen.add(key);
await this.prisma.vehicle.create({
data: {
policyId,
customerId: policy.customerId,
make: v.make,
// ANA prints one BODY cell, not separate model/body columns, so
// it lands on `bodyType`; `model` stays null rather than being
// guessed out of the same string.
bodyType: v.bodyType,
modelYear: v.modelYear,
vinNumber: v.vinNumber,
licensePlate: v.licensePlate,
// "VEHICLE" / "TRAILER" / "TOWING" — the printed slot, which is
// the difference between the insured car and the trailer behind
// it and has no column of its own.
notes: v.item && v.item !== "VEHICLE" ? v.item : null,
},
});
}
}
if (drivers.length) {
const existing = await this.prisma.insuredDriver.findMany({
where: { policyId },
select: { licenseNumber: true, fullName: true },
});
const seen = new Set(
existing.flatMap((d) =>
[d.licenseNumber, d.fullName].filter((k): k is string => !!k).map(norm),
),
);
for (const d of drivers) {
const key = norm(d.licenseNumber ?? d.fullName ?? "");
if (!key || seen.has(key)) continue;
seen.add(key);
await this.prisma.insuredDriver.create({
data: { policyId, fullName: d.fullName, licenseNumber: d.licenseNumber },
});
}
}
}
/**
* Stream the source PDF (`sourceKey`, set by `process` on the doc row)
* into the policy's storage namespace and create a `PolicyDocument`
* pointer. Trivial now that the doc row holds the exact source key —
* the old per-page "which file did this page come from" walk is gone.
*/
private async attachSourcePdf(sourceKey: string, policyId: string): Promise<void> {
private async attachSourcePdf(
sourceKey: string,
policyId: string,
provider: string | null,
): Promise<void> {
const got = await this.storage.getStream(sourceKey);
const chunks: Buffer[] = [];
for await (const c of got.stream) chunks.push(c as Buffer);
@@ -556,7 +726,10 @@ export class PolicyOcrService {
await this.prisma.policyDocument.create({
data: {
policyId,
documentType: "GMX_POLICY",
// Named after whichever parser claimed the page. Was hardcoded
// `GMX_POLICY`, which mislabelled every ANA upload as a GMX
// document in the policy's file list.
documentType: `${provider ?? "OCR"}_POLICY`,
storageKey: newKey,
},
});
@@ -588,6 +761,27 @@ export class PolicyOcrService {
}
}
/**
* The parser's provider code is not the carrier's row name in
* `insurance_providers`, and the two namespaces are allowed to differ.
*
* ANA is the case that forces this: the office's book is filed under
* "ANA SEGUROS" (738 policies). A bare "ANA" row also existed with 1 policy
* and is merged away by `20260815160000_policy_type_repair`, so an exact-name
* lookup on the parser's "ANA" would find nothing at all after that migration.
*
* Anything not listed resolves by its own name.
*/
const PROVIDER_ROW_NAME: Record<string, string> = {
ANA: "ANA SEGUROS",
};
/** The lookup FKs resolved for one document, absent when unresolvable. */
interface ResolvedLookups {
policyTypeId?: string;
insuranceProviderId?: string;
}
/** Map a (post-review) doc + final confirmed fields onto a `Policy.update`
* payload. Every field that is null in both inputs is omitted so we never
* write null over a value the Policy already carries (the GMX certificate
@@ -608,10 +802,13 @@ function buildPolicyUpdateFromDoc(
extractedNetPremium: Prisma.Decimal | null;
extractedPolicyFee: Prisma.Decimal | null;
extractedBrokerFee: Prisma.Decimal | null;
extractedTax: Prisma.Decimal | null;
extractedTotal: Prisma.Decimal | null;
extractedCoveragesJson: Prisma.JsonValue | null;
extractedPremiumPayment: string | null;
extractedCoveragePeriodDays: number | null;
},
lookups: ResolvedLookups,
): Prisma.PolicyUpdateInput {
const numOrUndef = (a: number | undefined, b: Prisma.Decimal | null): Prisma.Decimal | undefined => {
if (a != null) return new Prisma.Decimal(a);
@@ -631,14 +828,31 @@ function buildPolicyUpdateFromDoc(
return {
policyNumber: strOrUndef(item.policyNumber, doc.extractedPolicyNumber),
// `connect` rather than a raw id: this is the CHECKED update input. Left
// undefined when unresolved, so an existing Policy never loses a type or
// carrier it already had because this document could not name one.
policyType: lookups.policyTypeId ? { connect: { id: lookups.policyTypeId } } : undefined,
insuranceProvider: lookups.insuranceProviderId
? { connect: { id: lookups.insuranceProviderId } }
: undefined,
agentName: strOrUndef(item.agentName, doc.extractedAgentName),
policyFrom: dateOrUndef(item.policyFrom, doc.extractedPolicyFrom),
policyTo: dateOrUndef(item.policyTo, doc.extractedPolicyTo),
policyDate: dateOrUndef(item.policyDate, doc.extractedPolicyDate),
// Left undefined when the document didn't print a term, so the schema
// default (365) stands for GMX. ANA's by-the-day policies DO print one,
// and the default would otherwise turn a 4-day tourist policy into an
// annual one on the renewals screen.
coveragePeriodDays:
item.coveragePeriodDays ?? doc.extractedCoveragePeriodDays ?? undefined,
currency: strOrUndef(item.currency, doc.extractedCurrency) as Currency | undefined,
netPremium: numOrUndef(item.netPremium, doc.extractedNetPremium),
policyFee: numOrUndef(item.policyFee, doc.extractedPolicyFee),
brokerFee: numOrUndef(item.brokerFee, doc.extractedBrokerFee),
// `taxRate` is deliberately left alone. A.N.A. prints the IVA amount, not
// the rate, and back-dividing it would mint a rate the document never
// stated — the policy form resolves one from the line of business instead.
tax: numOrUndef(item.tax, doc.extractedTax),
total: numOrUndef(item.total, doc.extractedTotal),
// coveragesJson / observations: freeform, keep the GMX data when present.
coveragesJson:
@@ -680,11 +894,14 @@ function buildPolicyCreateFromDoc(
extractedNetPremium: Prisma.Decimal | null;
extractedPolicyFee: Prisma.Decimal | null;
extractedBrokerFee: Prisma.Decimal | null;
extractedTax: Prisma.Decimal | null;
extractedTotal: Prisma.Decimal | null;
extractedCoveragesJson: Prisma.JsonValue | null;
extractedPremiumPayment: string | null;
extractedCoveragePeriodDays: number | null;
},
customerId: string,
lookups: ResolvedLookups,
): Prisma.PolicyUncheckedCreateInput {
const numOrUndef = (a: number | undefined, b: Prisma.Decimal | null): Prisma.Decimal | undefined => {
if (a != null) return new Prisma.Decimal(a);
@@ -712,14 +929,26 @@ function buildPolicyCreateFromDoc(
return {
policyNumber,
customerId,
policyTypeId: lookups.policyTypeId,
insuranceProviderId: lookups.insuranceProviderId,
agentName: strOrUndef(item.agentName, doc.extractedAgentName),
policyFrom: dateOrUndef(item.policyFrom, doc.extractedPolicyFrom),
policyTo: dateOrUndef(item.policyTo, doc.extractedPolicyTo),
policyDate: dateOrUndef(item.policyDate, doc.extractedPolicyDate),
// Left undefined when the document didn't print a term, so the schema
// default (365) stands for GMX. ANA's by-the-day policies DO print one,
// and the default would otherwise turn a 4-day tourist policy into an
// annual one on the renewals screen.
coveragePeriodDays:
item.coveragePeriodDays ?? doc.extractedCoveragePeriodDays ?? undefined,
currency: strOrUndef(item.currency, doc.extractedCurrency) as Currency | undefined,
netPremium: numOrUndef(item.netPremium, doc.extractedNetPremium),
policyFee: numOrUndef(item.policyFee, doc.extractedPolicyFee),
brokerFee: numOrUndef(item.brokerFee, doc.extractedBrokerFee),
// `taxRate` is deliberately left alone. A.N.A. prints the IVA amount, not
// the rate, and back-dividing it would mint a rate the document never
// stated — the policy form resolves one from the line of business instead.
tax: numOrUndef(item.tax, doc.extractedTax),
total: numOrUndef(item.total, doc.extractedTotal),
coveragesJson:
item.coveragesJson !== undefined
@@ -765,3 +994,17 @@ function strOrUndefDb(a: string | undefined, b: string | null): string | undefin
if (b != null && b !== "") return b;
return undefined;
}
/** A JSON column the parser wrote as an array, read back as one. Anything
* else (null, DbNull, a legacy object shape) is an empty list rather than a
* crash — these columns are only ever populated by the parser, so a
* surprise shape means old data, not a caller to reject. */
function asArray<T>(value: Prisma.JsonValue | null): T[] {
return Array.isArray(value) ? (value as unknown as T[]) : [];
}
/** Compare identifiers the way a person would: case- and space-insensitive.
* VINs and plates are printed inconsistently ("8BPX206" vs "8BPX 206"). */
function norm(s: string): string {
return s.replace(/\s+/g, "").toUpperCase();
}
+120 -20
View File
@@ -15,6 +15,11 @@
*/
import { Prisma } from "@jorgecuadros/database";
import {
BALANCE_FORWARD_TYPE,
notCashJournal,
periodSourceTable,
} from "../billing/billing.service";
import {
intParam,
NOT_VOIDED,
@@ -751,13 +756,22 @@ const edoCuentaDatos: ReportDef = {
title: "Estado de cuenta",
description:
"Estado de cuenta de un cliente: saldos por moneda, desglose por " +
"ramo y concepto, y el historial completo de movimientos con saldo " +
"corrido. El reporte del cliente final.",
"ramo y concepto, y los movimientos del año en curso con saldo " +
"corrido, abriendo con el saldo anterior. El reporte del cliente final.",
domain: "estado-cuenta",
legacyName: "EDO CUENTA DATOS",
format: "statement",
params: [
{ key: "customerId", label: "Cliente", kind: "customer-picker" },
// Which period to print. Blank means the year in progress; an earlier year
// prints from its imported archive, the same source the on-screen
// statement reads.
{
key: "year",
label: "Periodo (año)",
kind: "number",
placeholder: "año en curso",
},
],
columns: [
// Statement rows carry synthetic `__kind` discriminators instead of
@@ -789,22 +803,63 @@ const edoCuentaDatos: ReportDef = {
});
if (!customer) return { rows: [], subtitle: "Cliente no encontrado" };
// Reuse the same NOT_VOIDED + STATEMENT_EXCLUDED_SOURCE_TABLES filter
// as BillingService.statement so the numbers match what the customer
// already sees in /estado-cuenta/[id].
// The source-table exclusion, the balance floor and the year scope below
// are BillingService.statement's, because this report and
// /estado-cuenta/[id] are the same statement — one printable, one on
// screen — and a customer holding both must not read two balances.
const floor = await prisma.transaction.findFirst({
where: {
customerId,
voidedAt: null,
type: { nameEn: BALANCE_FORWARD_TYPE },
},
orderBy: { transactionDate: "desc" },
select: { transactionDate: true },
});
// Which period to print. An earlier year comes from its imported archive,
// tagged rather than dated, exactly as the on-screen statement reads it.
const thisYear = new Date().getUTCFullYear();
const askedYear = Number(p.year);
const requestedYear =
Number.isInteger(askedYear) && askedYear > 0 ? askedYear : thisYear;
const isArchive = requestedYear !== thisYear;
const rows = await prisma.transaction.findMany({
where: {
customerId,
voidedAt: null,
legacySourceTable: {
notIn: [
"EFECTIVO",
"EFECTIVO_BACKUP",
"EFECTIVO FM3",
"CHEQUE FM3",
"IVA 2015",
],
},
...(isArchive
? // The archive is one period's ledger already, so the tag is the
// whole filter and the balance floor must not apply — the floor
// hides exactly the history this period is asking for.
{ legacySourceTable: periodSourceTable(requestedYear) }
: {
...(floor ? { transactionDate: { gte: floor.transactionDate } } : {}),
// Archive rows count as history below the year start (that is
// what `opening` is for, and for a customer floored by an archive
// it is the only carry there is) and are dropped at or above it.
// Same rule as the on-screen twin — see BillingService.statement.
AND: [
{
OR: [
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: "datos2@" } } },
{
transactionDate: {
lt: new Date(Date.UTC(requestedYear, 0, 1)),
},
},
],
},
// The cash receipt book, which the ledger already carries as
// its own `C<folio>` postings. Taken from the shared helper
// rather than restated, so the printed statement and the screen
// cannot drift apart — and so this keeps the database
// qualifier that spares the insurance line's own EFECTIVO.
notCashJournal(),
],
}),
},
orderBy: [{ transactionDate: "asc" }, { id: "asc" }],
select: {
@@ -822,12 +877,32 @@ const edoCuentaDatos: ReportDef = {
},
});
// Compute running balance per currency, then return newest-first.
// Scoped to the calendar year and listed oldest-first, the way the legacy
// EDO CUENTA sheet reads. Rows from earlier years still move the running
// balance — they are folded into `opening` and printed as a single "saldo
// anterior" line, which is what a BALANCE FORWARD row is.
// An archive needs no fold: it *is* the period, and its own Jan-1 BALANCE
// FORWARD row is the carry, printed like legacy printed it.
const yearStart = isArchive
? new Date(0)
: new Date(Date.UTC(requestedYear, 0, 1));
const year = requestedYear;
const running = new Map<string, Prisma.Decimal>();
const movements = rows.map((r) => {
const opening = new Map<string, Prisma.Decimal>();
const visible: typeof rows = [];
const movements = rows.flatMap((r) => {
const prev = running.get(r.currency) ?? new Prisma.Decimal(0);
const next = prev.plus(r.amount);
running.set(r.currency, next);
if (r.transactionDate < yearStart) {
opening.set(r.currency, next);
return [];
}
visible.push(r);
return {
date: r.transactionDate.toISOString().slice(0, 10),
domain: r.domain,
@@ -840,14 +915,38 @@ const edoCuentaDatos: ReportDef = {
balanceAfter: next.toFixed(2),
};
});
movements.reverse();
// Per-currency summary + per-domain breakdown.
// The carried balance, printed as the statement's first line — same shape
// as a movement row so it needs nothing special from the renderer.
const carried = [...opening.entries()]
.filter(([, amount]) => !amount.isZero())
.map(([currency, amount]) => ({
date: yearStart.toISOString().slice(0, 10),
domain: "UTILITY",
currency,
reference: "",
period: `Al cierre de ${year - 1}`,
checkNumber: "",
concept: "SALDO ANTERIOR",
amount: amount.toFixed(2),
balanceAfter: amount.toFixed(2),
}));
// Per-currency summary, seeded with the carried balance so it reconciles
// against the last running balance printed below.
const perCurrency = new Map<
string,
{ currency: string; charges: Prisma.Decimal; credits: Prisma.Decimal; count: number }
>();
for (const r of rows) {
for (const [currency, amount] of opening) {
perCurrency.set(currency, {
currency,
charges: amount.lessThan(0) ? amount : new Prisma.Decimal(0),
credits: amount.lessThan(0) ? new Prisma.Decimal(0) : amount,
count: 0,
});
}
for (const r of visible) {
const c =
perCurrency.get(r.currency) ??
{
@@ -881,9 +980,10 @@ const edoCuentaDatos: ReportDef = {
count: c.count,
})),
{ __kind: "movements-header" },
...carried,
...movements,
],
subtitle: `${nameOf(customer)} · ${rows.length} movimientos`,
subtitle: `${nameOf(customer)} · ${year} · ${visible.length} movimientos`,
};
},
};
@@ -7,8 +7,13 @@ import { parseBboxLayout } from "./tesseract.provider";
* that grouping is what left `PERIODO FACTURADO` with no value next to it and
* every period field empty on a batch whose text was perfectly readable.
*/
/**
* Boxes are sized from the text, at 6 units a character: the reassembler now
* reads the space BETWEEN two boxes, so a fixed width would put a fabricated
* gap after every short word and every row would come back column-padded.
*/
function word(x: number, y: number, text: string): string {
return `<word xMin="${x}" yMin="${y}" xMax="${x + 20}" yMax="${y + 8}">${text}</word>`;
return `<word xMin="${x}" yMin="${y}" xMax="${x + text.length * 6}" yMax="${y + 8}">${text}</word>`;
}
function doc(...lines: string[]): string {
@@ -26,23 +31,57 @@ describe("parseBboxLayout", () => {
it("rejoins a label with the value printed beside it in another flow", () => {
const [page] = parseBboxLayout(
doc(
word(20, 100, "PERIODO") + word(45, 100, "FACTURADO:"),
word(20, 100, "PERIODO") + word(68, 100, "FACTURADO:"),
word(300, 100.4, "20260630-20260630"),
padding(),
),
1,
);
expect(page).not.toBeNull();
expect(page!.text).toContain("PERIODO FACTURADO: 20260630-20260630");
expect(page!.text).toMatch(/PERIODO FACTURADO:\s+20260630-20260630/);
});
it("keeps genuinely separate lines apart", () => {
const [page] = parseBboxLayout(
doc(word(20, 100, "Cuenta:") + word(80, 100, "0900003463"), word(20, 130, "Nombre:"), padding()),
doc(word(20, 100, "Cuenta:") + word(68, 100, "0900003463"), word(20, 130, "Nombre:"), padding()),
1,
);
expect(page!.text.split("\n")).toContain("Cuenta: 0900003463");
expect(page!.text.split("\n")).toContain("Nombre:");
const lines = page!.text.split("\n").map((l) => l.trim());
expect(lines).toContain("Cuenta: 0900003463");
expect(lines).toContain("Nombre:");
});
/**
* The layout is data. A borderless table separates its cells with nothing
* but white space, so the parsers read a run of spaces as a cell boundary
* (`INSURED\s{2,}`) and a column offset as a column (`SUM INSURED` vs
* `PREMIUM`). Both regressed to nothing when this collapsed every gap to a
* single space, and the fixtures — taken from `pdftotext -layout`, which
* prints the gaps — could not see it.
*/
it("preserves the gap between two cells of a borderless table", () => {
const [page] = parseBboxLayout(
doc(word(20, 100, "INSURED") + word(300, 100, "PAMELA") + word(340, 100, "WAGONER"), padding()),
1,
);
const line = page!.text.split("\n").find((l) => l.includes("INSURED"))!;
expect(line).toMatch(/INSURED\s{2,}PAMELA WAGONER/);
});
it("preserves the blank line between two blocks", () => {
const [page] = parseBboxLayout(
doc(word(20, 100, "Insured"), word(20, 112, "wraps"), word(20, 200, "Next"), padding()),
1,
);
const lines = page!.text.split("\n").map((l) => l.trim());
// The wrapped continuation stays attached; the next block is cut off from
// it, which is what stops a "join until the cell ends" walk running away.
expect(lines.slice(lines.indexOf("Insured"), lines.indexOf("Next") + 1)).toEqual([
"Insured",
"wraps",
"",
"Next",
]);
});
it("scales point coordinates into the render's pixel space", () => {
@@ -254,6 +254,23 @@ export function parseBboxLayout(xhtml: string, scale: number): (OcrPage | null)[
* Rows are cut when a word's vertical centre leaves the band established by
* the row's first word, which tolerates the sub-pixel baseline differences
* between fonts on one line without merging two genuinely separate lines.
*
* Vertical WHITE SPACE is preserved as a blank line. Rows alone are not the
* whole layout: on a form, the blank between two blocks is what says where a
* cell's wrapped value stops, and dropping it leaves parsers that walk a
* block ("keep joining until the cell ends") running to the end of the page.
* That is not hypothetical — the GMX PVL especificación read its whole first
* page as the insured's name, because the fixtures were taken from
* `pdftotext -layout` (which prints the blanks) while the runtime fed it this
* function's output (which did not).
*
* Horizontal white space is preserved the same way, by padding each word out
* to its own column. The same fixture mismatch bit here: a run of spaces is
* the ONLY thing separating two cells of a borderless table, so ANA's
* `INSURED\s{2,}` label matches and its `SUM INSURED` / `PREMIUM` column
* split (taken from `head.search()` offsets) both need real offsets. Joining
* on one space put every driver's-policy premium in the sum-insured column
* and left the phone glued to the insured's name.
*/
function toVisualRows(words: OcrWord[]): string {
const centre = (w: OcrWord) => w.top + w.height / 2;
@@ -281,14 +298,85 @@ function toVisualRows(words: OcrWord[]): string {
}
if (current.length) rows.push(current);
return rows
.map((r) =>
[...r]
.sort((a, b) => a.left - b.left)
.map((w) => w.text)
.join(" "),
)
.join("\n");
const charWidth = estimateCharWidth(words);
const out: string[] = [];
rows.forEach((r, i) => {
if (i > 0 && isBlankBetween(rows[i - 1], r)) out.push("");
out.push(layoutRow(r, charWidth));
});
return out.join("\n");
}
/**
* One row rendered at its printed column offsets.
*
* Words that merely follow one another inside the same cell are separated by
* exactly one space, whatever the column arithmetic says: one `charWidth` for
* a page that mixes fonts leaves a rounding error on every word, and letting
* that accumulate sprinkles `\s{2,}` runs through ordinary prose — which is
* the very thing the parsers read as a cell boundary. Only a gap wide enough
* to be deliberate (more than one blank character) is rendered as one, and
* only there is the word re-anchored to its true column, so the offsets a
* column split depends on stay honest while values stay clean.
*/
function layoutRow(row: OcrWord[], charWidth: number): string {
let line = "";
let right = 0;
for (const w of [...row].sort((a, b) => a.left - b.left)) {
const col = Math.round(w.left / charWidth);
if (!line.length) {
line = " ".repeat(Math.max(0, col));
} else if (w.left - right > charWidth * 1.5) {
line += " ".repeat(Math.max(2, col - line.length));
} else {
line += " ";
}
line += w.text;
right = w.left + w.width;
}
return line.trimEnd();
}
/**
* Width of one character, in the same units the word boxes use.
*
* The median of each word's own width-per-character: robust to the handful of
* oversized headings and to the wide-tracked letterhead, both of which would
* drag a mean. Only words of 3+ characters vote, since a one-character box is
* mostly side bearing. Falls back to a value derived from line height when a
* page has nothing long enough to measure.
*/
function estimateCharWidth(words: OcrWord[]): number {
const samples = words
.filter((w) => w.text.length >= 3 && w.width > 0)
.map((w) => w.width / w.text.length)
.sort((a, b) => a - b);
if (samples.length) return samples[Math.floor(samples.length / 2)];
const heights = words.map((w) => w.height).filter((h) => h > 0);
return heights.length ? Math.max(...heights) / 2 : 1;
}
/**
* Does the space between two consecutive rows read as an empty line?
*
* Measured against the taller of the two rows so a heading and its body text
* are judged on their own scale. On the real documents the two populations do
* not overlap: consecutive lines of one paragraph sit at 0.31.1 line heights
* apart, and anything the reader sees as blank-separated starts at 2.1. The
* threshold is placed in that empty middle, biased high — a missed blank only
* restores today's behaviour, while a spurious one would cut a wrapped value
* short.
*/
function isBlankBetween(prev: OcrWord[], row: OcrWord[]): boolean {
const bottom = Math.max(...prev.map((w) => w.top + w.height));
const top = Math.min(...row.map((w) => w.top));
const unit = Math.max(
...prev.map((w) => w.height),
...row.map((w) => w.height),
);
return unit > 0 && top - bottom > unit * 1.6;
}
/**
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@jorgecuadros/web",
"version": "1.0.18",
"version": "1.0.26",
"private": true,
"scripts": {
"dev": "next dev -p 4500",
+4
View File
@@ -18,6 +18,10 @@ const TYPE: ChildConfig = {
fields: [
{ key: "name", label: "Nombre" },
{ key: "shortDescription", label: "Descripción" },
// The rate is stored as a fraction, not a percentage, and the label has to
// say so: 8 typed here would tax a $600 premium $4,800. The API rejects
// anything above 1 rather than trusting the label alone.
{ key: "taxRate", label: "IVA (0.08 = 8%)", type: "number", step: "0.0001" },
],
};
const ADJUSTER: ChildConfig = {
+10 -7
View File
@@ -128,6 +128,7 @@ function Detail({ id }: { id: string }) {
customerId={data.id}
summary={data.transactionSummary}
transactions={data.transactions}
year={data.transactionYear}
/>
<DocumentosSection data={data} />
</div>
@@ -746,16 +747,18 @@ function EstadoCuentaSection({
customerId,
summary,
transactions,
year,
}: {
customerId: string;
summary: TransactionSummaryRow[];
transactions: Transaction[];
year: number;
}) {
return (
<section className="section">
<SectionHead
rule="cuenta"
title="Estado de cuenta"
title={`Estado de cuenta ${year}`}
count={transactions.length}
countSuffix="movimientos"
/>
@@ -782,7 +785,7 @@ function EstadoCuentaSection({
<div className="card">
{transactions.length === 0 ? (
<div className="empty-inline">Sin movimientos registrados.</div>
<div className="empty-inline">Sin movimientos en {year}.</div>
) : (
<div className="tx-scroll">
<table className="tx-table">
@@ -804,11 +807,11 @@ function EstadoCuentaSection({
</table>
</div>
)}
{transactions.length >= 100 && (
<div className="section-note" style={{ padding: "0 16px 14px" }}>
Mostrando los 100 movimientos más recientes.
</div>
)}
<div className="section-note" style={{ padding: "0 16px 14px" }}>
Movimientos de {year}, del más antiguo al más reciente. Los saldos de
arriba son el saldo actual por línea de negocio los mismos del
estado de cuenta, no la suma del año.
</div>
</div>
{transactions.length > 0 && (
<p className="section-note">
+100 -10
View File
@@ -36,10 +36,14 @@ import type {
* charge and an insurance payment finally sit on the same page, under the same
* person, with a running balance.
*
* The running balance is per currency (the API accumulates it chronologically
* before handing the list back newest-first), so the movement table is scoped
* to one currency at a time — a column that alternated between pesos and
* dollars would be a meaningless number.
* The running balance is per currency, so the movement table is scoped to one
* currency at a time — a column that alternated between pesos and dollars would
* be a meaningless number.
*
* Like the legacy EDO CUENTA report, the table covers one calendar year and runs
* oldest-first, opening on the balance carried in from before it. The period
* selector switches years; earlier ones are served from the imported archive of
* that year, which is how legacy kept them — one table per closed year.
*/
export default function EstadoCuentaDetailPage({
params,
@@ -65,19 +69,27 @@ function StatementView({ id }: { id: string }) {
const [currency, setCurrency] = useState<LedgerCurrency | null>(null);
const [domain, setDomain] = useState<TransactionDomain | "">("");
/** null = the current period; the API decides what that is. */
const [year, setYear] = useState<number | null>(null);
function reload() {
let alive = true;
setLoading(true);
setError(null);
getStatement(id)
getStatement(id, year ?? undefined)
.then((d) => {
if (!alive) return;
setData(d);
// Default to the currency the customer actually moves the most in;
// preserve a previously-chosen currency across reloads.
// preserve a previously-chosen currency across reloads — but only if
// the loaded period still has it. Switching to a year the customer
// never moved dollars in would otherwise leave the picker on USD with
// no matching option, showing an empty table for a year that has rows.
const busiest = [...d.summary].sort((a, b) => b.count - a.count)[0];
setCurrency((prev) => prev ?? busiest?.currency ?? "MXN");
const fallback = busiest?.currency ?? "MXN";
setCurrency((prev) =>
prev && d.summary.some((s) => s.currency === prev) ? prev : fallback,
);
setLoading(false);
})
.catch((e) => {
@@ -99,7 +111,7 @@ function StatementView({ id }: { id: string }) {
getBillingFacets().then(setFacets).catch(() => setFacets(null));
return cleanup;
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [id]);
}, [id, year]);
const movements = useMemo(() => {
if (!data || !currency) return [];
@@ -194,7 +206,7 @@ function StatementView({ id }: { id: string }) {
<section className="section">
<SectionHead
rule="cuenta"
title="Movimientos"
title={`Movimientos ${data.year}`}
count={movements.length}
countSuffix={movements.length === 1 ? "movimiento" : "movimientos"}
right={
@@ -227,6 +239,26 @@ function StatementView({ id }: { id: string }) {
)}
<div className="filter-row">
{/* Only the periods this customer has. A year with no archive would
render an empty table that reads as "no hubo movimientos" when the
truth is that the year was never imported. */}
{data.availableYears.length > 1 && (
<label className="filter-field">
<span className="filter-label">Periodo</span>
<select
className="input select"
value={data.year}
onChange={(e) => setYear(Number(e.target.value))}
>
{data.availableYears.map((y) => (
<option key={y} value={y}>
{y}
{y === data.availableYears[0] ? " (en curso)" : ""}
</option>
))}
</select>
</label>
)}
<label className="filter-field">
<span className="filter-label">Moneda</span>
<select
@@ -260,7 +292,7 @@ function StatementView({ id }: { id: string }) {
<div className="card">
{movements.length === 0 ? (
<div className="empty-inline">
Sin movimientos en {currency}
Sin movimientos de {data.year} en {currency}
{domain ? ` para ${domainLabel(domain)}` : ""}.
</div>
) : (
@@ -282,6 +314,26 @@ function StatementView({ id }: { id: string }) {
</tr>
</thead>
<tbody>
{/*
The carried balance, shown the way the legacy report shows
it: a BALANCE FORWARD line above the year's movements. It
only appears when there is something to carry — when the
customer's opening-balance row is itself dated inside this
year (the usual case) it is listed as an ordinary movement
and this row is zero, so it is left out.
Suppressed under a business-line filter: the carried balance
is the customer's, across both lines, and printing it above
one line's rows would read as that line's opening balance.
*/}
{!domain && Number(active?.opening ?? 0) !== 0 && (
<OpeningRow
opening={active!.opening}
currency={currency}
year={data.year}
canVoid={canVoid}
/>
)}
{movements.map((m) => (
<StatementRow
key={m.id}
@@ -481,6 +533,44 @@ function ConceptosSection({
);
}
/** The balance carried into the statement year — legacy's BALANCE FORWARD. */
function OpeningRow({
opening,
currency,
year,
canVoid,
}: {
opening: string;
currency: LedgerCurrency;
year: number;
canVoid: boolean;
}) {
return (
<tr>
<td className="mono" style={{ whiteSpace: "nowrap" }}>
{formatDate(`${year}-01-01T00:00:00.000Z`)}
</td>
<td className="tx-domain-cell">Ambas líneas</td>
<td>
Saldo anterior
<div className="tx-concept">Al cierre de {year - 1}</div>
</td>
<td className="tx-ref"></td>
<td className="num">
<span className={`tx-amount ${Number(opening) < 0 ? "neg" : "pos"}`}>
{formatMoney(opening, currency)}
</span>
</td>
<td className="num">
<span className={`bal-running ${balanceTone(opening)}`}>
{formatMoney(opening, currency)}
</span>
</td>
{canVoid && <td />}
</tr>
);
}
function StatementRow({
m,
canVoid,
+73
View File
@@ -926,6 +926,15 @@ button {
color: var(--ink-soft);
margin-bottom: 0.4375rem;
}
/* Sub-label under an input: the computed figure behind an override field, or
why a field is disabled. Quiet enough not to compete with .field-label. */
.field-hint {
display: block;
font-size: 0.75rem;
line-height: 1.35;
color: var(--muted-2);
margin-top: 0.3125rem;
}
.input {
width: 100%;
font-family: inherit;
@@ -940,6 +949,12 @@ button {
.input::placeholder {
color: var(--muted-2);
}
.input:disabled,
.select:disabled {
background: var(--surface-2, var(--surface));
color: var(--muted-2);
cursor: not-allowed;
}
.input:focus {
outline: none;
border-color: var(--brand-600);
@@ -3114,3 +3129,61 @@ button {
border-color: var(--brand-500);
color: var(--brand-700);
}
/* ============================================================================
Layout + text utilities the screens already assumed
Several components were written against these names before any rule
defined them, so they rendered as bare inline spans. The visible symptom
was the policy OCR review header running together —
"Para revisarPágina 1700489616· PAMELA DENISE WAGONERLICENCIASANA" —
because JSX drops the newline between sibling elements and the `gap` those
call sites pass does nothing without a flex container.
========================================================================== */
.row {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.5rem;
}
.stack {
display: flex;
flex-direction: column;
gap: 1rem;
}
/* The muted line under a page title, and the same voice reused inline. Only
the block form takes a margin — as a flex child it would shift the item
off the row's centre line. */
.page-sub {
color: var(--muted);
font-size: 0.875rem;
}
p.page-sub {
margin: 0.25rem 0 0;
}
/* A neutral chip. Same shape as `.badge` so the OCR statuses, policy type and
carrier read as the labels they are rather than as running prose. */
.tag {
display: inline-flex;
align-items: center;
gap: 0.375rem;
padding: 0.1875rem 0.5625rem;
border-radius: 999px;
font-size: 0.75rem;
font-weight: 600;
letter-spacing: 0.01em;
line-height: 1.4;
white-space: nowrap;
background: var(--paper-2);
color: var(--muted);
border: 1px solid var(--line-strong);
}
/* The warning sibling of `.state-error`, used where a page needs a human to
choose between candidates rather than reporting a failure. */
.state-warn {
background: var(--servicios-tint);
border: 1px solid rgba(154, 106, 18, 0.25);
color: var(--servicios-ink);
border-radius: var(--radius);
padding: 1rem 1.125rem;
font-size: 0.875rem;
}
+58 -4
View File
@@ -68,6 +68,7 @@ function Operaciones() {
const [starting, setStarting] = useState(false);
const fileInputs = useRef<Record<string, HTMLInputElement | null>>({});
const periodInput = useRef<HTMLInputElement | null>(null);
const refreshLists = useCallback(() => {
listIngest().then(setIngest).catch(() => setIngest([]));
@@ -143,6 +144,20 @@ function Operaciones() {
}
}
/** Upload a prior-period archive under its own filename. */
async function handleUploadPeriod(file: File | undefined) {
if (!file) return;
if (!/^\d{4}\.accdb$/i.test(file.name)) {
setError(
`"${file.name}" no es un archivo de periodo. Debe llamarse AAAA.accdb, por ejemplo 2025.accdb.`,
);
if (periodInput.current) periodInput.current.value = "";
return;
}
await handleUpload(file.name, file);
if (periodInput.current) periodInput.current.value = "";
}
async function handleDeleteIngest(name: string) {
setError(null);
try {
@@ -197,7 +212,11 @@ function Operaciones() {
else await start("RESTORE", c.file);
}
const ingestReady = (ingest ?? []).every((f) => f.present);
// Only the four fixed Access sources gate a run. Period archives are
// optional extras — having none simply means no prior years are available.
const ingestReady = (ingest ?? [])
.filter((f) => f.periodYear === null)
.every((f) => f.present);
return (
<>
@@ -244,8 +263,9 @@ function Operaciones() {
<div className="card" style={{ padding: 20, marginBottom: 20 }}>
<h2 className="section-title">Carpeta de ingesta</h2>
<p className="inline-form-note">
Los cuatro archivos originales de Access. La reimportación y la
sincronización leen de aquí. Tamaño máximo por archivo: {formatBytes(INGEST_MAX_BYTES)}.
Los cuatro archivos originales de Access, más los archivos de periodos
anteriores. La reimportación y la sincronización leen de aquí. Tamaño
máximo por archivo: {formatBytes(INGEST_MAX_BYTES)}.
</p>
<div className="tx-scroll">
<table className="tx-table">
@@ -262,7 +282,14 @@ function Operaciones() {
{(ingest ?? []).map((f) => (
<Fragment key={f.name}>
<tr>
<td className="mono">{f.name}</td>
<td className="mono">
{f.name}
{f.periodYear !== null && (
<span className="badge" style={{ marginLeft: 8 }}>
periodo {f.periodYear}
</span>
)}
</td>
<td>
<span className={`badge ${f.present ? "badge-positive" : "badge-negative"}`}>
{f.present ? "Presente" : "Falta"}
@@ -313,6 +340,33 @@ function Operaciones() {
</tbody>
</table>
</div>
{/* A period archive that has never been uploaded has no row to click,
so it needs its own entry point. The file names itself: the archive
IS `2025.accdb`, and that name is what declares the period, so the
control reads it off the chosen file rather than asking twice. */}
<div className="row-actions" style={{ marginTop: 16 }}>
<input
ref={periodInput}
type="file"
accept=".accdb"
style={{ display: "none" }}
onChange={(e) => handleUploadPeriod(e.target.files?.[0])}
/>
<button
className="btn btn-outline"
type="button"
disabled={uploading !== null}
onClick={() => periodInput.current?.click()}
>
Agregar periodo anterior
</button>
<span className="inline-form-note">
Un archivo de Access por año cerrado, nombrado con su periodo:{" "}
<span className="mono">2025.accdb</span>. Aporta el año anterior al
estado de cuenta; no altera el saldo actual.
</span>
</div>
</div>
{/* Operations */}
+117 -7
View File
@@ -26,7 +26,13 @@ import {
premiumHeadline,
SIN_NOMBRE,
} from "@/lib/labels";
import type { AdjusterRow, Installment, PolicyDetail } from "@/lib/types";
import {
PAYMENT_FREQUENCY_LABELS,
type AdjusterRow,
type Installment,
type PolicyDetail,
} from "@/lib/types";
import { formatRate } from "@/lib/premium";
export default function PolizaDetailPage({
params,
@@ -186,6 +192,10 @@ function ChildrenEditor({
const INSTALLMENTS: ChildConfig = {
apiKind: "installments",
title: "Pagos",
// A policy paid in several exhibiciones prices each payment on its own, so
// the whole premium breakdown repeats per row — that is the two-row money
// block on the Access form. `amount` stays what was actually collected and
// is deliberately separate from `total`; they differ by rounding.
fields: [
{ key: "sequence", label: "Sec.", type: "number" },
{ key: "amount", label: "Monto", type: "number" },
@@ -195,6 +205,12 @@ function ChildrenEditor({
{ key: "paidDate", label: "Pagado", type: "date" },
{ key: "checkNumber", label: "Cheque" },
{ key: "isCash", label: "Efectivo", type: "checkbox" },
{ key: "netPremium", label: "Prima neta", type: "number" },
{ key: "surcharge", label: "Recargo", type: "number" },
{ key: "policyFee", label: "Derecho", type: "number" },
{ key: "tax", label: "IVA", type: "number" },
{ key: "total", label: "Prima total", type: "number" },
{ key: "commission", label: "Comisión", type: "number" },
],
};
const VEHICLES: ChildConfig = {
@@ -412,15 +428,40 @@ function CondicionesSection({ data }: { data: PolicyDetail }) {
data.coveragePeriodDays ? `${data.coveragePeriodDays} días` : null
}
/>
<KV
label="Forma de pago"
value={
data.paymentFrequency
? PAYMENT_FREQUENCY_LABELS[data.paymentFrequency]
: null
}
/>
<KV label="Prima neta" value={formatMoney(data.netPremium, cur)} />
{/* Only ever set on a policy paid in installments, so showing an
empty row on the other 98% would be noise. */}
{data.surcharge != null && Number(data.surcharge) !== 0 && (
<KV label="Recargo" value={formatMoney(data.surcharge, cur)} />
)}
<KV label="Derecho de póliza" value={formatMoney(data.policyFee, cur)} />
<KV label="Comisión" value={formatMoney(data.commission, cur)} />
<KV label="Honorarios" value={formatMoney(data.brokerFee, cur)} />
{/* Access never stored IVA — it was a calculated control on the form
— so every migrated policy reads null here until it is edited. */}
{data.tax != null && (
<KV
label={
data.taxRate != null
? `IVA (${formatRate(Number(data.taxRate))})`
: "IVA"
}
value={formatMoney(data.tax, cur)}
/>
)}
{/* The legacy `total` is 0 or null on all but 2 of 2378 policies —
only show it when it actually carries a figure. */}
{data.total != null && Number(data.total) > 0 && (
<KV label="Total" value={formatMoney(data.total, cur)} />
<KV label="Prima total" value={formatMoney(data.total, cur)} />
)}
<KV label="Comisión" value={formatMoney(data.commission, cur)} />
<KV label="Honorarios" value={formatMoney(data.brokerFee, cur)} />
<KV
label="Liquidación"
value={
@@ -648,10 +689,79 @@ function SiniestrosSection({ data }: { data: PolicyDetail }) {
}
/* -------------------------------------------------------- Coberturas */
/** The legacy tables carry per-line coverage columns the target schema does
* not model; the migration preserved them verbatim in `coveragesJson`. */
/**
* `coveragesJson` holds two unrelated shapes and the section renders each on
* its own terms:
*
* - **A Spanish-keyed object** — the legacy per-line coverage columns the
* target schema does not model, preserved verbatim by the migration. Every
* policy imported from Access carries this one.
* - **A `ParsedCoverage[]` array** — written by the policy OCR confirm step
* (GMX's coverage table, ANA's numbered risk sections).
*
* Running the object renderer over the array is what used to happen, and it
* produced a row per array index labelled "0", "1", "2" with `[object
* Object]` as its value — not a crash, so nothing surfaced it.
*/
interface StoredCoverage {
risk?: string;
insuredAmount?: number | null;
deductible?: string | null;
lossParticipation?: string | null;
premium?: number | null;
}
function CoberturasSection({ data }: { data: PolicyDetail }) {
const entries = Object.entries(data.coveragesJson ?? {}).filter(
const raw = data.coveragesJson ?? null;
if (Array.isArray(raw)) {
const rows = (raw as StoredCoverage[]).filter((c) => c && c.risk);
if (rows.length === 0) return null;
return (
<section className="section">
<SectionHead rule="seguros" title="Coberturas" count={rows.length} />
<div className="card">
<div className="tx-scroll">
<table className="tx-table">
<thead>
<tr>
<th>Riesgo</th>
<th className="num">Suma asegurada</th>
<th className="num">Prima</th>
<th>Deducible</th>
<th>Participación</th>
</tr>
</thead>
<tbody>
{rows.map((c, i) => (
<tr key={i}>
<td>{c.risk}</td>
<td className="num">
{c.insuredAmount == null
? "—"
: formatMoney(c.insuredAmount.toString(), data.currency)}
</td>
<td className="num">
{c.premium == null
? "—"
: formatMoney(c.premium.toString(), data.currency)}
</td>
<td>{c.deductible ?? "—"}</td>
<td>{c.lossParticipation ?? "—"}</td>
</tr>
))}
</tbody>
</table>
</div>
<div className="section-note" style={{ padding: "0 22px 18px" }}>
Coberturas leídas del PDF de la aseguradora.
</div>
</div>
</section>
);
}
const entries = Object.entries(raw ?? {}).filter(
([, v]) => v !== null && v !== "" && v !== 0,
);
if (entries.length === 0) return null;
+1 -1
View File
@@ -4,7 +4,7 @@ import { AppShell } from "@/components/AppShell";
import { PolicyCaptura } from "@/components/PolicyCaptura";
/**
* OCR mode of the policy intake screen. Drops the GMX PDF, walks through
* OCR mode of the policy intake screen. Drops the GMX or A.N.A. PDF, walks through
* per-page review, confirms. Same wrapper as `/polizas/nuevo` (manual)
* with `initialMode="auto"`, so the tab strip is identical and swapping
* modes doesn't drop state.
+4 -1
View File
@@ -9,6 +9,9 @@ export type FieldDef = {
type?: "text" | "number" | "date" | "checkbox" | "select";
options?: { value: string; label: string }[];
width?: number;
/** Numeric granularity. Defaults to money (0.01); a tax rate stored as a
* fraction needs finer, or the browser rejects 0.0825 as off-step. */
step?: string;
};
export type ChildConfig = {
@@ -158,7 +161,7 @@ export function ChildCollection({
<input
className="input"
type={f.type === "number" ? "number" : f.type === "date" ? "date" : "text"}
step={f.type === "number" ? "0.01" : undefined}
step={f.type === "number" ? f.step ?? "0.01" : undefined}
value={String(values[f.key] ?? "")}
onChange={(e) => setValues({ ...values, [f.key]: e.target.value })}
/>
+2 -2
View File
@@ -12,7 +12,7 @@ import { useCan } from "@/lib/abilities";
* two ways in:
*
* - **manual** — `PolicyForm` keys every field by hand.
* - **auto** — `PolicyOcrIntake` uploads a GMX PDF, OCR proposes the
* - **auto** — `PolicyOcrIntake` uploads a GMX or A.N.A. PDF, OCR proposes the
* policy, a human still confirms.
*
* Both end at the same place (a `Policy` row on a customer's file) so they
@@ -28,7 +28,7 @@ export type PolicyCaptureMode = "manual" | "auto";
const MODE_HINT: Record<PolicyCaptureMode, string> = {
manual:
"Captura cada campo a mano. Use esta opción cuando la póliza llega en papel, en un correo sin PDF legible, o cuando hay que revisar cada dato.",
auto: "Suelte el PDF descargado del portal de GMX y el sistema propondrá los campos. Nada se registra sin tu confirmación.",
auto: "Suelte el PDF descargado del portal de GMX o de A.N.A. y el sistema propondrá los campos. Nada se registra sin tu confirmación.",
};
export function PolicyCaptura({ initialMode = "manual" }: { initialMode?: PolicyCaptureMode }) {
+119 -6
View File
@@ -4,12 +4,22 @@ import { useEffect, useState } from "react";
import { useRouter } from "next/navigation";
import { CustomerPicker } from "@/components/CustomerPicker";
import { createPolicy, getLookups, updatePolicy } from "@/lib/api";
import type {
Currency,
LookupsResponse,
PolicyDetail,
PolicyInput,
import {
PAYMENT_FREQUENCY_LABELS,
type Currency,
type LookupsResponse,
type PaymentFrequency,
type PolicyDetail,
type PolicyInput,
} from "@/lib/types";
import {
computeTax,
computeTotal,
formatRate,
resolveTaxRate,
surchargeApplies,
taxableBase,
} from "@/lib/premium";
function toDateInput(v: string | null | undefined): string {
if (!v) return "";
@@ -36,9 +46,16 @@ type V = {
policyFrom: string;
policyTo: string;
netPremium: string;
surcharge: string;
policyFee: string;
brokerFee: string;
commission: string;
/** Blank means "use the computed figure". Only ever holds a value once the
* operator overrides it, so a later change to prima neta keeps flowing
* through instead of being frozen by a value the form itself wrote. */
tax: string;
total: string;
paymentFrequency: PaymentFrequency | "";
currency: Currency;
liquidated: boolean;
liquidationNumber: string;
@@ -58,9 +75,13 @@ function initial(p?: PolicyDetail): V {
policyFrom: toDateInput(p?.policyFrom),
policyTo: toDateInput(p?.policyTo),
netPremium: p?.netPremium != null ? String(p.netPremium) : "",
surcharge: p?.surcharge != null ? String(p.surcharge) : "",
policyFee: p?.policyFee != null ? String(p.policyFee) : "",
brokerFee: p?.brokerFee != null ? String(p.brokerFee) : "",
commission: p?.commission != null ? String(p.commission) : "",
tax: p?.tax != null ? String(p.tax) : "",
total: p?.total != null ? String(p.total) : "",
paymentFrequency: p?.paymentFrequency ?? "",
currency: (p?.currency as Currency) ?? "MXN",
liquidated: p?.liquidated ?? false,
liquidationNumber: p?.liquidationNumber ?? "",
@@ -101,6 +122,27 @@ export function PolicyForm({
setV((p) => ({ ...p, [k]: val }));
}
// IVA and Total are the only two figures the form derives. Everything else,
// the recargo included, is keyed by hand — the carrier quotes the financing
// charge, we do not compute it.
const selectedType = lookups?.types.find((t) => t.id === v.policyTypeId);
const taxRate = resolveTaxRate(policy?.taxRate, selectedType?.taxRate);
const parts = {
netPremium: v.netPremium,
// A recargo on an annual policy is a data-entry mistake, so it is dropped
// from the arithmetic as well as disabled in the UI. Otherwise switching
// ANNUAL after typing one would leave it silently inflating the IVA.
surcharge: surchargeApplies(v.paymentFrequency || null) ? v.surcharge : "",
policyFee: v.policyFee,
};
const computedTax = computeTax(parts, taxRate);
const computedTotal = computeTotal(parts, taxRate);
// Blank field = take the computed figure. A typed one wins, so staff can key
// the carrier's rounding verbatim when it disagrees with ours by a centavo.
const effectiveTax = v.tax.trim() === "" ? computedTax : Number(v.tax);
const effectiveTotal = v.total.trim() === "" ? computedTotal : Number(v.total);
const showSurcharge = surchargeApplies(v.paymentFrequency || null);
async function submit(e: React.FormEvent) {
e.preventDefault();
if (!customerId) {
@@ -118,9 +160,17 @@ export function PolicyForm({
policyFrom: s(v.policyFrom),
policyTo: s(v.policyTo),
netPremium: numOrUndef(v.netPremium),
surcharge: showSurcharge ? numOrUndef(v.surcharge) : undefined,
policyFee: numOrUndef(v.policyFee),
brokerFee: numOrUndef(v.brokerFee),
commission: numOrUndef(v.commission),
// The derived figures are persisted, not recomputed on read: the printed
// policy is the record of truth and a later rate change must not silently
// restate what was issued. `taxRate` rides along for the same reason.
tax: Number.isFinite(effectiveTax) ? effectiveTax : undefined,
taxRate,
total: Number.isFinite(effectiveTotal) ? effectiveTotal : undefined,
paymentFrequency: v.paymentFrequency || undefined,
currency: v.currency,
liquidated: v.liquidated,
liquidationNumber: s(v.liquidationNumber),
@@ -208,7 +258,7 @@ export function PolicyForm({
</div>
<div className="card" style={{ padding: 20, marginBottom: 16 }}>
<h2 className="section-title" style={{ marginBottom: 14 }}>Vigencia y prima</h2>
<h2 className="section-title" style={{ marginBottom: 14 }}>Vigencia</h2>
<div className="form-grid">
<label className="field">
<span className="field-label">Emisión</span>
@@ -225,21 +275,84 @@ export function PolicyForm({
<input className="input" type="date" value={v.policyTo}
onChange={(e) => set("policyTo", e.target.value)} />
</label>
<label className="field">
<span className="field-label">Forma de pago</span>
<select className="select" value={v.paymentFrequency}
onChange={(e) =>
set("paymentFrequency", e.target.value as PaymentFrequency | "")
}>
<option value=""></option>
{(
Object.keys(PAYMENT_FREQUENCY_LABELS) as PaymentFrequency[]
).map((f) => (
<option key={f} value={f}>{PAYMENT_FREQUENCY_LABELS[f]}</option>
))}
</select>
</label>
</div>
</div>
<div className="card" style={{ padding: 20, marginBottom: 16 }}>
<h2 className="section-title" style={{ marginBottom: 4 }}>Primas</h2>
<p className="muted" style={{ fontSize: 12, marginBottom: 14 }}>
IVA y prima total se calculan solos sobre (prima neta + recargo +
derecho de póliza). Puede sobrescribirlos si la póliza impresa
redondea distinto.
</p>
<div className="form-grid">
<label className="field">
<span className="field-label">Prima neta</span>
<input className="input" type="number" step="0.01" value={v.netPremium}
onChange={(e) => set("netPremium", e.target.value)} />
</label>
<label className="field">
<span className="field-label">Recargo</span>
<input className="input" type="number" step="0.01" value={v.surcharge}
disabled={!showSurcharge}
onChange={(e) => set("surcharge", e.target.value)} />
<span className="field-hint">
{showSurcharge
? "Lo cotiza la aseguradora — se captura a mano."
: "No aplica en pago anual ni de contado."}
</span>
</label>
<label className="field">
<span className="field-label">Derecho de póliza</span>
<input className="input" type="number" step="0.01" value={v.policyFee}
onChange={(e) => set("policyFee", e.target.value)} />
</label>
<label className="field">
<span className="field-label">IVA ({formatRate(taxRate)})</span>
<input className="input" type="number" step="0.01"
placeholder={computedTax.toFixed(2)} value={v.tax}
onChange={(e) => set("tax", e.target.value)} />
<span className="field-hint">
Calculado: {computedTax.toFixed(2)} sobre base{" "}
{taxableBase(parts).toFixed(2)}
{selectedType?.taxRate == null &&
policy?.taxRate == null &&
" · tasa por omisión, configúrela en Catálogos"}
</span>
</label>
<label className="field">
<span className="field-label">Prima total</span>
<input className="input" type="number" step="0.01"
placeholder={computedTotal.toFixed(2)} value={v.total}
onChange={(e) => set("total", e.target.value)} />
<span className="field-hint">
Calculado: {computedTotal.toFixed(2)}
</span>
</label>
<label className="field">
<span className="field-label">Comisión</span>
<input className="input" type="number" step="0.01" value={v.commission}
onChange={(e) => set("commission", e.target.value)} />
</label>
<label className="field">
<span className="field-label">Honorarios</span>
<input className="input" type="number" step="0.01" value={v.brokerFee}
onChange={(e) => set("brokerFee", e.target.value)} />
</label>
</div>
</div>
+10 -7
View File
@@ -13,9 +13,12 @@ import type { PolicyOcrBatch, PolicyOcrBatchStatus } from "@/lib/types";
/**
* Insurance OCR intake — mirror of StatementIntake, scoped to the insurance
* side. Today the only provider is GMX; the parser dispatches on a brand
* wordmark (`Grupo Mexicano de Seguros` / `gmx.com.mx` / the GMX letterhead)
* and a new portal only needs a new BRAND entry plus a parser file.
* side. GMX and A.N.A. today; the parser dispatches on a brand wordmark
* (`Grupo Mexicano de Seguros` / `gmx.com.mx`, `A.N.A. Compañía de Seguros` /
* `anaseguros.com.mx`) and a new portal only needs a new BRAND entry plus a
* parser file. The uploader is never asked which provider a file came from —
* a batch may mix them, and the pipeline labels the batch from what the
* parsers actually claimed.
*
* Lives inside the `Pólizas` page rather than a top-level route because it
* is one mode of one job (staff uploading whatever PDFs the office has on
@@ -102,8 +105,8 @@ export function PolicyOcrIntake() {
<div className="state-box">Cargando</div>
) : batches.length === 0 ? (
<div className="state-box">
Todavía no hay lotes de pólizas. Descargue el certificado del portal
de GMX y suéltelo arriba.
Todavía no hay lotes de pólizas. Descargue la póliza del portal de
GMX o de A.N.A. y suéltela arriba.
</div>
) : (
<div className="tx-scroll">
@@ -183,14 +186,14 @@ function UploadCard({ onDone }: { onDone: () => void }) {
return (
<section className="card" style={{ padding: 16 }}>
<h2 className="section-title" style={{ marginTop: 0 }}>
Subir PDFs de pólizas (GMX)
Subir PDFs de pólizas (GMX / A.N.A.)
</h2>
<div className="inline-form" style={{ flexWrap: "wrap", gap: 12 }}>
<label>
<span className="page-sub">Referencia (opcional)</span>
<input
className="input"
placeholder="ej. GMX julio 2026"
placeholder="ej. ANA agosto 2026"
value={label}
onChange={(e) => setLabel(e.target.value)}
/>
+155 -7
View File
@@ -21,6 +21,7 @@ import type {
PolicyOcrBatchDetail,
PolicyOcrConfirmDocument,
PolicyOcrCoverage,
PolicyOcrCustomerSuggestion,
PolicyOcrDocument,
PolicyOcrReviewInput,
} from "@/lib/types";
@@ -180,8 +181,11 @@ export function PolicyOcrReview({ id }: { id: string }) {
{STATUS_LABEL[batch.status] ?? batch.status}
</p>
</div>
<Link className="btn btn-ghost" href="/polizas">
Volver a pólizas
{/* Back to the capture screen this batch was uploaded from, not to
the policy list — same as the statement review screen, which
returns to /recibos. */}
<Link className="btn btn-ghost" href="/polizas/captura">
Volver a captura
</Link>
</header>
@@ -272,8 +276,11 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
policyDate: doc.extractedPolicyDate?.slice(0, 10) ?? "",
currency: doc.extractedCurrency ?? "USD",
netPremium: doc.extractedNetPremium ?? "",
policyFee: doc.extractedPolicyFee ?? "",
tax: doc.extractedTax ?? "",
total: doc.extractedTotal ?? "",
premiumPayment: doc.extractedPremiumPayment ?? "",
coveragePeriodDays: doc.extractedCoveragePeriodDays?.toString() ?? "",
postPremium: doc.extractedNetPremium != null && Number(doc.extractedNetPremium) > 0,
});
const [customerId, setCustomerId] = useState(
@@ -309,8 +316,11 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
policyDate: v.policyDate || undefined,
currency,
netPremium: numOrUndef(v.netPremium),
policyFee: numOrUndef(v.policyFee),
tax: numOrUndef(v.tax),
total: numOrUndef(v.total),
premiumPayment: trimOrUndef(v.premiumPayment),
coveragePeriodDays: numOrUndef(v.coveragePeriodDays),
matchedPolicyId: policyId || undefined,
matchedCustomerId: !policyId && customerId ? customerId : undefined,
forceConfirm: true,
@@ -330,8 +340,11 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
policyDate: reviewInput.policyDate,
currency: (currency as "MXN" | "USD" | "EUR" | undefined) ?? undefined,
netPremium: reviewInput.netPremium,
policyFee: reviewInput.policyFee,
tax: reviewInput.tax,
total: reviewInput.total,
premiumPayment: reviewInput.premiumPayment,
coveragePeriodDays: reviewInput.coveragePeriodDays,
coveragesJson: (doc.extractedCoveragesJson ?? undefined) as
| PolicyOcrCoverage[]
| undefined,
@@ -348,18 +361,27 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
const locked = doc.status === "POSTED" || doc.status === "REJECTED";
const matchedExisting = !!doc.matchedPolicy;
const candidates = doc.matchCandidates ?? [];
const suggestions: PolicyOcrCustomerSuggestion[] = doc.customerSuggestions ?? [];
return (
<article className="card" style={{ padding: 16 }}>
<header className="row" style={{ gap: 12, alignItems: "center" }}>
<span className="tag">{STATUS_LABEL[doc.status] ?? doc.status}</span>
<span className="page-sub">Página {doc.pageNumber}</span>
{doc.extractedPolicyNumber && (
<strong style={{ marginLeft: 8 }}>{doc.extractedPolicyNumber}</strong>
)}
{/* No hand-rolled separators or margins here: `.row` is a flex
container and its gap does the spacing. A literal "· " would leave
a dot floating in that gap. */}
{doc.extractedPolicyNumber && <strong>{doc.extractedPolicyNumber}</strong>}
{doc.extractedInsuredName && (
<span className="page-sub">· {doc.extractedInsuredName}</span>
<span className="page-sub">{doc.extractedInsuredName}</span>
)}
{/* Read-only: the parser names the type, the confirm step resolves it
to a policy_types row. Reassigning it is the policy screen's job,
where the full picker already lives. */}
{doc.extractedPolicyTypeName && (
<span className="tag">{doc.extractedPolicyTypeName}</span>
)}
{doc.provider && <span className="tag">{doc.provider}</span>}
</header>
<div className="doc-detail">
@@ -454,6 +476,17 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
onChange={(e) => set("policyDate", e.target.value)}
/>
</Field>
{/* ANA sells 3- and 4-day tourist policies; left blank the
póliza keeps the 365-day default. */}
<Field label="Días de vigencia">
<input
className="input"
type="number"
min={1}
value={v.coveragePeriodDays}
onChange={(e) => set("coveragePeriodDays", e.target.value)}
/>
</Field>
<Field label="Moneda">
<select
className="input select"
@@ -474,7 +507,25 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
onChange={(e) => set("netPremium", e.target.value)}
/>
</Field>
<Field label="Total">
<Field label="Derecho de póliza">
<input
className="input"
type="number"
step="0.01"
value={v.policyFee}
onChange={(e) => set("policyFee", e.target.value)}
/>
</Field>
<Field label="IVA">
<input
className="input"
type="number"
step="0.01"
value={v.tax}
onChange={(e) => set("tax", e.target.value)}
/>
</Field>
<Field label="Prima total">
<input
className="input"
type="number"
@@ -523,6 +574,7 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
<tr>
<th>Riesgo</th>
<th className="num">Suma</th>
<th className="num">Prima</th>
<th>Deducible</th>
<th>Participación</th>
</tr>
@@ -534,6 +586,14 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
<td className="num">
{formatMoney(c.insuredAmount?.toString() ?? null, v.currency)}
</td>
{/* ANA's add-on sections print what the coverage COST
where the others print what it pays. Kept in its own
column so the two are never added together. */}
<td className="num">
{c.premium == null
? "—"
: formatMoney(c.premium.toString(), v.currency)}
</td>
<td>{c.deductible ?? "—"}</td>
<td>{c.lossParticipation ?? "—"}</td>
</tr>
@@ -543,6 +603,66 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
</details>
)}
{/*
* Vehicles and named drivers are read-only here: they are written
* as their own Vehicle / InsuredDriver rows on confirm, and the
* policy screen is where they get edited. Showing them is what
* lets a reviewer catch a misread VIN before it is applied.
*/}
{doc.extractedVehiclesJson && doc.extractedVehiclesJson.length > 0 && (
<details>
<summary>Unidades ({doc.extractedVehiclesJson.length})</summary>
<table className="tx-table" style={{ marginTop: 8 }}>
<thead>
<tr>
<th>Tipo</th>
<th>Año</th>
<th>Marca</th>
<th>Carrocería</th>
<th>Serie</th>
<th>Placas</th>
</tr>
</thead>
<tbody>
{doc.extractedVehiclesJson.map((veh, i) => (
<tr key={i}>
<td>{veh.item}</td>
<td>{veh.modelYear ?? "—"}</td>
<td>{veh.make ?? "—"}</td>
<td>{veh.bodyType ?? "—"}</td>
<td>{veh.vinNumber ?? "—"}</td>
<td>{veh.licensePlate ?? "—"}</td>
</tr>
))}
</tbody>
</table>
</details>
)}
{doc.extractedDriversJson && doc.extractedDriversJson.length > 0 && (
<details>
<summary>Conductores ({doc.extractedDriversJson.length})</summary>
<table className="tx-table" style={{ marginTop: 8 }}>
<thead>
<tr>
<th>Nombre</th>
<th>Licencia</th>
<th>Teléfono</th>
</tr>
</thead>
<tbody>
{doc.extractedDriversJson.map((d, i) => (
<tr key={i}>
<td>{d.fullName}</td>
<td>{d.licenseNumber ?? "—"}</td>
<td>{d.phone ?? "—"}</td>
</tr>
))}
</tbody>
</table>
</details>
)}
{candidates.length > 1 && (
<Field label="Póliza destino">
<select
@@ -580,6 +700,34 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
</Field>
)}
{/*
* Name suggestions, never a preselection. The office writes
* customers surname-first and carriers print them given-name-first,
* so without this the reviewer retypes a name the parser already
* read. One click fills the picker above; nothing is chosen until
* they click. Kept outside the <Field> label — a label must not
* wrap other interactive controls.
*/}
{!policyId && !customerId && !locked && canReview && suggestions.length > 0 && (
<div className="row" style={{ gap: 8, flexWrap: "wrap" }}>
<span className="page-sub">Sugerencias por nombre:</span>
{suggestions.map((s) => (
<button
key={s.customerId}
type="button"
className="btn btn-ghost btn-sm"
onClick={() => {
setCustomerId(s.customerId);
setCustomerName(s.customerName);
}}
>
{s.customerName}
{s.tier === "PARTIAL" && <span className="page-sub"> · parcial</span>}
</button>
))}
</div>
)}
<label className="field">
<input
type="checkbox"
+8 -3
View File
@@ -615,8 +615,13 @@ export function getBillingFacets(): Promise<BillingFacets> {
return apiFetch<BillingFacets>("/billing/facets");
}
export function getStatement(customerId: string): Promise<Statement> {
return apiFetch<Statement>(`/billing/customers/${customerId}`);
/** `year` omitted reads the current period; earlier years come from an archive. */
export function getStatement(
customerId: string,
year?: number,
): Promise<Statement> {
const q = year === undefined ? "" : `?year=${year}`;
return apiFetch<Statement>(`/billing/customers/${customerId}${q}`);
}
/** Append a new ledger movement. Booked movements are never edited — fix
@@ -1432,7 +1437,7 @@ export function statementPageUrl(documentId: string): string {
return `${API_ORIGIN}/statements/documents/${documentId}/page`;
}
/* ----------------------------------------------------- Policy OCR (GMX) */
/* ----------------------------------------------- Policy OCR (GMX / ANA) */
export function getPolicyOcrStatus(): Promise<{
ocrAvailable: boolean;
+79
View File
@@ -0,0 +1,79 @@
import type { PaymentFrequency } from "./types";
/**
* Client-side twin of apps/api/src/policies/premium.ts. Duplicated rather than
* shared because the API and the web app do not share a package today, and
* both need it: the form computes IVA and Total live as the operator types,
* the API stores what it is sent.
*
* base = prima neta + recargo + derecho de póliza
* IVA = round(base * tasa)
* Total = base + IVA
*
* The recargo is inside the taxable base — that is what reconciles the Access
* books (policy 7006785 prints IVA 52.03 on 610.86 + 8.55 + 31.00; leaving the
* recargo out gives 51.35, which matches nothing on the page).
*/
/** Applied when neither the policy nor its type carries a rate. The single
* row both legacy IMPUESTOS tables held. */
export const DEFAULT_TAX_RATE = 0.08;
/** Paying in more than one exhibición is what earns a recargo. A null
* frequency (every migrated policy) is treated as "unknown, allow it": the
* legacy recargo figures are real and hiding the field would hide them. */
export function surchargeApplies(
frequency: PaymentFrequency | null | undefined,
): boolean {
return frequency !== "ANNUAL" && frequency !== "SINGLE";
}
export function num(v: string | number | null | undefined): number {
if (v === null || v === undefined || v === "") return 0;
const n = typeof v === "number" ? v : Number(String(v).trim());
return Number.isFinite(n) ? n : 0;
}
/** Half-up to cents, matching how the printed policy rounds. */
export function round2(n: number): number {
return Math.round((n + Number.EPSILON) * 100) / 100;
}
export interface PremiumParts {
netPremium: string | number | null | undefined;
surcharge: string | number | null | undefined;
policyFee: string | number | null | undefined;
}
export function taxableBase(p: PremiumParts): number {
return round2(num(p.netPremium) + num(p.surcharge) + num(p.policyFee));
}
export function computeTax(p: PremiumParts, rate: number): number {
return round2(taxableBase(p) * rate);
}
export function computeTotal(p: PremiumParts, rate: number): number {
return round2(taxableBase(p) + computeTax(p, rate));
}
/** Rate ladder: what the policy was issued at, else its line of business, else
* the default. Keeps an old policy reading back at its original rate after
* somebody edits the catalog. */
export function resolveTaxRate(
policyRate: string | number | null | undefined,
policyTypeRate: string | number | null | undefined,
): number {
for (const candidate of [policyRate, policyTypeRate]) {
if (candidate === null || candidate === undefined || candidate === "") continue;
const n = Number(candidate);
if (Number.isFinite(n) && n >= 0) return n;
}
return DEFAULT_TAX_RATE;
}
/** 0.08 -> "8%". Rates are stored as fractions but read as percentages. */
export function formatRate(rate: number): string {
const pct = round2(rate * 100);
return `${pct}%`;
}
+114 -1
View File
@@ -3,6 +3,24 @@
export type Currency = "USD" | "MXN";
/** How the premium is split into payments. Anything other than ANNUAL/SINGLE
* is what earns a recargo. Null on every migrated policy — the original ETL
* dropped Access's FORMA PAGO column entirely. */
export type PaymentFrequency =
| "ANNUAL"
| "SEMIANNUAL"
| "QUARTERLY"
| "MONTHLY"
| "SINGLE";
export const PAYMENT_FREQUENCY_LABELS: Record<PaymentFrequency, string> = {
ANNUAL: "Anual",
SEMIANNUAL: "Semestral",
QUARTERLY: "Trimestral",
MONTHLY: "Mensual",
SINGLE: "Contado",
};
export type Role = "ADMIN" | "MANAGER" | "STAFF" | "VIEWER";
export type Ability =
@@ -174,6 +192,8 @@ export interface IngestFile {
present: boolean;
size: number | null;
modifiedAt: string | null;
/** Year of a prior-period archive (`2025.accdb`); null on the four fixed sources. */
periodYear: number | null;
}
export interface BackupFile {
@@ -289,6 +309,16 @@ export interface Installment {
paidDate: string | null;
checkNumber: string | null;
isCash: boolean;
/** Per-payment premium breakdown — a policy paid in several exhibiciones
* prices each one separately. `amount` is what was actually collected and
* can differ from `total` by rounding; it is not derived from these. */
netPremium: string | null;
surcharge: string | null;
policyFee: string | null;
tax: string | null;
taxRate: string | null;
total: string | null;
commission: string | null;
}
export interface Vehicle {
@@ -398,10 +428,14 @@ export interface PolicyInput {
policyTo?: string;
coveragePeriodDays?: number;
netPremium?: number;
surcharge?: number;
policyFee?: number;
brokerFee?: number;
commission?: number;
tax?: number;
taxRate?: number;
total?: number;
paymentFrequency?: PaymentFrequency;
currency?: Currency;
observations?: string;
notes?: string;
@@ -419,6 +453,13 @@ export interface InstallmentInput {
paidDate?: string;
checkNumber?: string;
isCash?: boolean;
netPremium?: number;
surcharge?: number;
policyFee?: number;
tax?: number;
taxRate?: number;
total?: number;
commission?: number;
}
export interface VehicleInput {
make?: string;
@@ -469,6 +510,10 @@ export interface PolicyTypeRow {
id: string;
name: string;
shortDescription: string | null;
/** IVA fraction for this line of business, 0.08 = 8%. Null means "not
* configured" and the form falls back to DEFAULT_TAX_RATE — it does NOT
* mean the line is untaxed. Serialized as a decimal string by Prisma. */
taxRate: string | null;
_count?: { policies: number };
}
export interface AdjusterRow {
@@ -567,10 +612,14 @@ export interface PolicyDetail {
policyTo: string | null;
coveragePeriodDays: number | null;
netPremium: string | null;
surcharge: string | null;
policyFee: string | null;
brokerFee: string | null;
commission: string | null;
tax: string | null;
taxRate: string | null;
total: string | null;
paymentFrequency: PaymentFrequency | null;
currency: string | null;
observations: string | null;
notes: string | null;
@@ -994,6 +1043,8 @@ export interface BillingFacets {
export interface StatementSummary {
currency: LedgerCurrency;
/** Balance carried in from before the statement year — legacy's BALANCE FORWARD. */
opening: string;
charges: string;
credits: string;
balance: string;
@@ -1007,6 +1058,7 @@ export interface StatementSummary {
export interface StatementDomainRow {
domain: TransactionDomain;
currency: LedgerCurrency;
opening: string;
charges: string;
credits: string;
balance: string;
@@ -1042,6 +1094,15 @@ export interface Statement {
propertyCount: number;
policyCount: number;
};
/** Calendar year the statement covers; movements are scoped to it. */
year: number;
/**
* Periods this customer actually has, newest first. The current year is
* always present; each earlier year comes from an imported archive. Offering
* anything outside this list would render an empty statement that reads as
* "no activity" rather than "not imported".
*/
availableYears: number[];
summary: StatementSummary[];
byDomain: StatementDomainRow[];
byType: StatementTypeRow[];
@@ -1077,6 +1138,8 @@ export interface CustomerDetail {
properties: Property[];
policies: Policy[];
transactions: Transaction[];
/** Calendar year `transactions` covers. */
transactionYear: number;
transactionSummary: TransactionSummaryRow[];
}
@@ -1405,7 +1468,7 @@ export interface DiscardBatchResult {
rejected: number;
}
/* ------------------------------------------ Policy OCR intake (GMX) */
/* ------------------------------------- Policy OCR intake (GMX / ANA) */
export type PolicyOcrBatchStatus =
| "UPLOADED"
@@ -1447,6 +1510,27 @@ export interface PolicyOcrCoverage {
insuredAmount: number | null;
deductible: string | null;
lossParticipation: string | null;
/** What the coverage COST, where the layout prints it separately from what
* it pays out (ANA's add-on sections). GMX never prints one. */
premium?: number | null;
}
/** A row of ANA's `ITEM / YEAR / MAKE / BODY / SERIAL No. / PLATES` table. */
export interface PolicyOcrVehicle {
item: string;
modelYear: string | null;
make: string | null;
bodyType: string | null;
vinNumber: string | null;
licensePlate: string | null;
}
export interface PolicyOcrDriver {
fullName: string;
licenseNumber: string | null;
address: string | null;
phone: string | null;
email: string | null;
}
export interface PolicyOcrMatchCandidate {
@@ -1456,6 +1540,18 @@ export interface PolicyOcrMatchCandidate {
policyNumber: string;
}
/**
* A customer whose name resembles the printed insured name. A suggestion,
* not a match — the API never preselects one.
*/
export interface PolicyOcrCustomerSuggestion {
customerId: string;
customerName: string;
/** `EXACT` = same name tokens in any order; `PARTIAL` = one contains the other. */
tier: "EXACT" | "PARTIAL";
score: number;
}
export interface PolicyOcrDocument {
id: string;
pageNumber: number;
@@ -1475,9 +1571,18 @@ export interface PolicyOcrDocument {
extractedNetPremium: string | null;
extractedPolicyFee: string | null;
extractedBrokerFee: string | null;
/** IVA off A.N.A.'s `TAX` cell. Null on GMX — its certificate carries no
* premium, so no tax either. `LOCAL TAX` is not folded in; a non-zero one
* shows up in `matchNote`. */
extractedTax: string | null;
extractedTotal: string | null;
extractedCoveragesJson: PolicyOcrCoverage[] | null;
extractedPremiumPayment: string | null;
extractedCoveragePeriodDays: number | null;
extractedVehiclesJson: PolicyOcrVehicle[] | null;
extractedDriversJson: PolicyOcrDriver[] | null;
/** `PolicyType.name` the parser read, resolved to an id only at confirm. */
extractedPolicyTypeName: string | null;
matchedPolicy: {
id: string;
policyNumber: string | null;
@@ -1486,6 +1591,7 @@ export interface PolicyOcrDocument {
} | null;
matchedCustomer: { id: string; name: string } | null;
matchCandidates: PolicyOcrMatchCandidate[] | null;
customerSuggestions: PolicyOcrCustomerSuggestion[] | null;
matchNote: string | null;
}
@@ -1503,8 +1609,10 @@ export interface PolicyOcrReviewInput {
netPremium?: number;
policyFee?: number;
brokerFee?: number;
tax?: number;
total?: number;
premiumPayment?: string;
coveragePeriodDays?: number;
coveragesJson?: PolicyOcrCoverage[];
matchedPolicyId?: string;
matchedCustomerId?: string;
@@ -1528,9 +1636,14 @@ export interface PolicyOcrConfirmDocument {
netPremium?: number;
policyFee?: number;
brokerFee?: number;
tax?: number;
total?: number;
premiumPayment?: string;
coveragePeriodDays?: number;
coveragesJson?: PolicyOcrCoverage[];
/** Explicit lookup picks; both beat the name the parser read. */
policyTypeId?: string;
insuranceProviderId?: string;
postPremium?: boolean;
}
@@ -61,6 +61,11 @@ services:
INGEST_DIR: /data/ingest
BACKUP_DIR: /data/backups
MIGRATION_ENV: prod
# The API applies pending Prisma migrations at container start, before
# Nest listens, and refuses to start if they fail (docker/api-entrypoint.sh).
# Set false ONLY when the schema is being moved by hand — the app will
# then boot against whatever schema it finds.
RUN_MIGRATIONS: ${RUN_MIGRATIONS:-true}
# Credentials the "Operaciones" screen runs mysqldump/mysql as. NOT the
# application user: --single-transaction needs the global RELOAD privilege
# and the app user has only ALL ON jorgecuadros.*, so every backup, sync
+5
View File
@@ -41,6 +41,11 @@ services:
INGEST_DIR: /data/ingest
BACKUP_DIR: /data/backups
MIGRATION_ENV: prod
# The API applies pending Prisma migrations at container start, before
# Nest listens, and refuses to start if they fail (docker/api-entrypoint.sh).
# Set false ONLY when the schema is being moved by hand — the app will
# then boot against whatever schema it finds.
RUN_MIGRATIONS: ${RUN_MIGRATIONS:-true}
# Credentials the "Operaciones" screen runs mysqldump/mysql as. NOT the
# application user: --single-transaction needs the global RELOAD privilege
# and the app user has only ALL ON jorgecuadros.*, so every backup, sync
+102
View File
@@ -0,0 +1,102 @@
#!/bin/sh
# Apply pending Prisma migrations, then hand off to the API.
#
# WHY THE CONTAINER AND NOT THE DEPLOY WORKFLOW
#
# The workflow still has its own `prisma migrate deploy` step and that is not
# redundant: it runs BEFORE the new images are pulled, i.e. while the OLD code
# is still serving, which is the order expand/contract migrations are designed
# around (see docs/DEPLOY_AND_MIGRATIONS.md). Doing it here as well closes the
# gaps that step cannot:
#
# - The runner has to reach MySQL directly. When it cannot, the deploy is run
# with `skip_migrate=true` and the schema silently does not move — the app
# then boots against a schema that is one release behind, which surfaces
# later as a column-not-found at runtime rather than as a failed deploy.
# - A container restarted by `restart: unless-stopped` after a host reboot,
# or a stack re-applied by hand in Portainer, never goes through the
# workflow at all.
#
# `migrate deploy` is idempotent, so running it in both places costs one
# no-op query on the normal path.
#
# THE API DOES NOT START IF THE MIGRATION FAILS. That is deliberate: serving
# against a schema that does not match the code is worse than being down,
# because the failures it produces are partial and silent (a write to a column
# that does not exist yet fails for one feature while the rest of the app looks
# healthy). The container exits non-zero and Docker's restart policy retries.
set -e
SCHEMA=/repo/packages/database/prisma/schema.prisma
log() { echo "[entrypoint] $*"; }
if [ "${RUN_MIGRATIONS:-true}" != "true" ]; then
log "RUN_MIGRATIONS=${RUN_MIGRATIONS} — skipping migrations, starting the API"
exec "$@"
fi
if [ -z "${DATABASE_URL}" ]; then
log "DATABASE_URL is unset; cannot migrate." >&2
log "Set it, or set RUN_MIGRATIONS=false if you migrate out of band." >&2
exit 1
fi
# pnpm's hoisted linker normally puts the CLI in the root .bin, but the
# workspace package keeps its own link too. Accept either rather than pinning
# a layout detail of the installer — the Dockerfile asserts at build time that
# one of these exists, so a missing CLI breaks the image build, not a deploy.
PRISMA=""
for candidate in /repo/node_modules/.bin/prisma \
/repo/packages/database/node_modules/.bin/prisma; do
if [ -x "$candidate" ]; then
PRISMA="$candidate"
break
fi
done
if [ -z "$PRISMA" ]; then
log "prisma CLI not found in this image; cannot migrate." >&2
exit 1
fi
# Retry ONLY a connection failure (P1001). On a full bring-up the database
# container can still be starting, and on galactus the API additionally has to
# resolve the host's MagicDNS name — a lookup that is unreliable for the first
# moments after a host reboot (see the dns block in the app compose file, and
# docs/DEPLOY_AND_MIGRATIONS.md).
#
# Every other failure exits immediately. Retrying a migration that is actually
# broken just delays the same error behind a minute of noise, and P3005 in
# particular needs a human.
attempt=1
max="${MIGRATE_MAX_ATTEMPTS:-20}"
delay="${MIGRATE_RETRY_SECONDS:-3}"
while : ; do
log "prisma migrate deploy (attempt ${attempt}/${max})"
if output=$("$PRISMA" migrate deploy --schema "$SCHEMA" 2>&1); then
printf '%s\n' "$output"
log "migrations up to date"
break
fi
printf '%s\n' "$output" >&2
if ! printf '%s' "$output" | grep -q 'P1001'; then
log "migrate deploy FAILED — refusing to start the API." >&2
if printf '%s' "$output" | grep -q 'P3005'; then
log "P3005: the database has tables but no migration history. This is a" >&2
log "database that predates Prisma migrations. Baseline it ONCE with:" >&2
log " npx prisma@5 migrate resolve --applied 0000_init --schema $SCHEMA" >&2
fi
exit 1
fi
if [ "$attempt" -ge "$max" ]; then
log "database unreachable after ${max} attempts — giving up." >&2
exit 1
fi
attempt=$((attempt + 1))
sleep "$delay"
done
exec "$@"
+16
View File
@@ -113,5 +113,21 @@ ENV APP_VERSION=$APP_VERSION \
GIT_SHA=$GIT_SHA \
BUILD_DATE=$BUILD_DATE
# Pending migrations are applied at container start, before Nest listens —
# see the header of the script for why this is done here as well as in the
# deploy workflow. Asserted at BUILD time so a missing prisma CLI breaks the
# image build rather than a production boot: the runtime layer copies
# /repo/node_modules wholesale, and which of these two paths carries the bin
# is an implementation detail of pnpm's hoisted linker.
COPY docker/api-entrypoint.sh /usr/local/bin/api-entrypoint.sh
RUN chmod +x /usr/local/bin/api-entrypoint.sh
RUN for c in /repo/node_modules/.bin/prisma \
/repo/packages/database/node_modules/.bin/prisma; do \
if [ -x "$c" ]; then echo "prisma CLI found at $c"; exit 0; fi; \
done; \
echo "FATAL: prisma CLI is not in the runtime layer; api-entrypoint.sh cannot migrate" >&2; \
exit 1
EXPOSE 3001
ENTRYPOINT ["/usr/local/bin/api-entrypoint.sh"]
CMD ["node", "apps/api/dist/main.js"]
+56 -13
View File
@@ -12,8 +12,8 @@ carries the reasoning. Close an item *there* as well as here, or the two drift.
**Verified against dev at compile time** (re-run before trusting the numbers):
```
policy_types: AUTO, LICENCIAS, MULT
policies NULL policyTypeId: 5
policy_types: AUTO, LICENCIAS, MULT (+ M_EMPR after 20260815160000)
policies NULL policyTypeId: 5 (0 after 20260815160000)
policies pending liquidación: 226
customers: 1536
last tag: v1.0.6 (2026-08-02 02:06 UTC) — 14 commits, 5 migrations behind HEAD
@@ -98,16 +98,31 @@ in the same phone call — (55) 5480-4000.
## 2. Live data defects — open, and confirmed open today
### 2.1 `policy_types` is missing `INCENDIO` and `M_EMPR`, and 5 policies are orphaned
### 2.1 ~~`policy_types` missing rows + 5 orphaned policies~~ — FIXED 2026-08-15
`policyTypeId` is `String?` with a plain relation, so Prisma's default is
`SetNull`. The spec's recommended `onDelete: Restrict` was **never applied**.
Five `m_empr` policies lost their ramo; four of them are pending liquidación
and are invisible to every ramo-filtered query — including the pending report
§2 is supposed to produce.
`SetNull`, and `removePolicyType()` had no in-use guard — deleting a lookup row
returned 200 and silently blanked the ramo on every policy using it. That is
what happened to `M_EMPR` and its 5 `m_empr` policies.
Fix alongside the liquidación work (3.1), since it distorts that feature's own
report. Source: INSURANCE "Two defects found while verifying this spec".
Closed by `20260815160000_policy_type_repair` plus the guard in
`policies.service.ts`:
- `M_EMPR` restored and the 5 policies re-pointed at it, scoped to
`policyTypeId IS NULL AND legacySourceTable = 'm_empr'` so it cannot claim a
policy blanked for some other reason. Idempotent; verified against dev inside
a rolled-back transaction.
- **`INCENDIO` deliberately not recreated.** The legacy `INCENDIO` table has
1 row and it never loaded, so the type has zero policies — restoring it would
only add a dead option to the type picker.
- Deleting an in-use policy type, carrier or adjuster now **refuses** with the
name and the count. `claims.adjusterId` had the identical `SET NULL` trap and
is guarded too. `onDelete: Restrict` at the schema level was not applied —
the application guard gives a Spanish message the operator can act on, where
a raw FK error would not.
- The duplicate `ANA` carrier row (1 policy) was merged into `ANA SEGUROS`
(738), since OCR now assigns the carrier automatically and two rows would
keep splitting the book.
### 2.2 ≤41 MULT second settlements were dropped in migration
@@ -164,13 +179,41 @@ Each of these is a known, deliberate stopping point rather than a bug.
be added.
- No `SKIPPED_NO_EMAIL` worklist (see 1.9).
**Captura de pólizas — desglose de primas** (built 2026-08-18)
- **IVA y prima total no existen en los datos legacy.** En Access eran
controles calculados sin campo, así que las 2,378 pólizas migradas leen
`tax` y `total` en null hasta que alguien las edite. No es recuperable: no
hay de dónde.
- **`LOCAL TAX` de A.N.A. no se captura.** El IVA (`TAX`) sí — se guarda desde
2026-08-18 — pero `LOCAL TAX` es un gravamen distinto sin columna destino y
**no** se suma al IVA: sumarlo daría una cifra que ya no divide de vuelta a
una tasa. Imprime 0.00 en todas las pólizas vistas hasta hoy; una distinta
de cero levanta la nota *"impuesto local N no capturado"* y significa que
`total` no cuadra contra `netPremium + policyFee + tax`.
- **`Policy.taxRate` queda en null por la ruta OCR.** A.N.A. imprime el monto
del IVA, no la tasa, y despejarla a la inversa inventaría una tasa que el
documento nunca declaró. El formulario resuelve una desde el ramo.
- **El recargo no se valida contra la forma de pago en datos migrados.** El
formulario lo deshabilita en ANUAL/CONTADO, pero
`backfill_policy_premium_breakdown.py` solo advierte cuando encuentra una
póliza anual con recargo; no la corrige.
- **Las parcialidades 3 y 4 no llevan desglose.** Access solo dibujó la fila
de dinero dos veces, así que una póliza trimestral capturada hoy sí puede
llenar las cuatro a mano, pero no hay nada legacy que migrar a las dos
últimas.
**Policy OCR** — [`POLICY_OCR.md`](POLICY_OCR.md)
- **GMX only.** The dispatcher is a `[provider, pattern]` table plus a parser
map, so a second carrier is one function and two entries — but no other
layout has been seen, and guessing produces a parser nobody can verify.
- **GMX and A.N.A. only.** The dispatcher is a `[provider, pattern]` table plus
a parser map, so a third carrier is one function and two entries — but no
other layout has been seen, and guessing produces a parser nobody can verify.
- **The `recibo` PDF is unread.** The GMX certificate carries no premium at
all; reading the separate receipt and pairing it to its certificate is what
would let `postPremium` stop being a manual tick.
would let `postPremium` stop being a manual tick. A.N.A. prints its premium
on the face, so this is a GMX-only gap.
- **No `insuranceProviderId` beyond the two OCR carriers.** Confirm resolves
the parser's provider to an `insurance_providers` row by name, so GMX and
A.N.A. land correctly; a policy typed in by hand still gets whatever the
operator picks.
- **No versioning.** A re-issued policy arrives as a new certificate with the
same number and confirm updates the existing row. Nothing records that this
is the 2027 issue of that policy.
+62 -6
View File
@@ -56,9 +56,11 @@ workflow's last step fails if the API does not report the tag you dispatched.
`pre-migrate-<tag>-<timestamp>.sql.gz`, which is exactly what the
**Operaciones** admin screen lists and can restore. A dump taken on the CI
runner would be unreachable by the only restore path the platform has.
3. **`prisma migrate deploy`** — as a workflow *step*, never the container
`CMD`. If it were the CMD, N replicas would race each other applying the
same migration.
3. **`prisma migrate deploy`** — as a workflow *step*, so the schema moves
while the OLD code is still serving, which is the order expand/contract is
designed around. **The api container repeats this at start** (below); the
command is idempotent, so on the normal path the container's run is a
no-op.
4. **app** — the new api + web images.
5. **Verify**`GET /version` on the running API must report the dispatched
tag.
@@ -141,6 +143,59 @@ Commit the generated `migrations/<timestamp>_add_foo/` directory. `db push` is
now a local-scratch tool only — using it against a database with history
desynchronises it from `_prisma_migrations`.
If you hand-write a migration instead of generating one, check it against what
Prisma would have produced before committing — a hand-written file that drifts
from `schema.prisma` fails on the *next* deploy, not this one:
```bash
prisma migrate diff \
--from-schema-datamodel <schema.prisma at the previous commit> \
--to-schema-datamodel packages/database/prisma/schema.prisma --script
```
## Migrations also run at container start
`docker/api-entrypoint.sh` is the api image's `ENTRYPOINT`. It runs
`prisma migrate deploy` and only then `exec`s the API. **If the migration
fails the container exits non-zero and the API never listens.**
That is the point. Serving against a schema that does not match the code is
worse than being down, because the failures are partial and silent — a write
to a column that does not exist yet breaks one feature while the rest of the
app looks healthy.
This does not replace the workflow step, which still runs first and against
the old code. It covers what that step cannot:
- **`skip_migrate: true`.** Previously that left the schema behind with no
further safety net, and the mismatch surfaced later as a runtime error. Now
it just moves the migration into the container, so it is a safe choice when
the runner cannot reach MySQL.
- **Restarts that never touch the workflow** — `restart: unless-stopped`
bringing the stack back after a host reboot, or a stack re-applied by hand
in Portainer.
Behaviour worth knowing:
| | |
|---|---|
| `RUN_MIGRATIONS=false` | Skip and start anyway. Plumbed through both app stack files. For when the schema is being moved by hand. |
| `DATABASE_URL` unset | Refuses to start (it would have failed at Nest boot anyway, but this says why). |
| Cannot reach the database (**P1001**) | Retries, default 20 × 3s. Covers a cold `db` container and galactus's MagicDNS lookup right after a reboot. `MIGRATE_MAX_ATTEMPTS` / `MIGRATE_RETRY_SECONDS` tune it. |
| Any other failure | Exits at once. Retrying a broken migration only delays the same error; **P3005** additionally prints the `migrate resolve --applied 0000_init` hint. |
**On replicas.** Both stacks are `replicas: 1` and must stay that way for an
unrelated reason (the servicios email sweep has no DB lock — see the caveats
below). If that ever changes, concurrent `migrate deploy` runs are safe on
their own: Prisma takes a database advisory lock, so the others block and then
find nothing pending. They would each pay the wait at startup, not corrupt
anything.
The prisma CLI has to be present in the runtime layer for any of this. The
image copies `/repo/node_modules` wholesale so it already is, and the
Dockerfile **asserts it at build time** — a missing CLI breaks the image
build rather than a production boot.
## galactus vs cubex
`galactus` is standalone Docker (Portainer endpoint **3**), `cubex` is a 3-node
@@ -319,9 +374,10 @@ backup does them (see `deploy/scripts/pre-migrate-backup.mjs`):
Portainer serves a self-signed certificate. It is scoped to that one step,
which talks to nothing but Portainer. Replacing the certificate and dropping
the flag is the real fix.
- The runner lives on cubex and must reach the target host's Portainer (9443)
**and** MySQL (3306). If it cannot reach 3306, run the migration by hand from
a host that can and dispatch with `skip_migrate: true`.
- The runner lives on cubex and must reach the target host's Portainer (9443).
It should also reach MySQL (3306) for the migrate step, but that is no longer
load-bearing — dispatch with `skip_migrate: true` and the api container
applies the migrations itself at start.
- `bootstrap: true` lets the pre-migrate backup be skipped when no API container
exists yet. Use it for a first-ever deploy only — it is the one switch that
lets a migration run with no restore point.
+6
View File
@@ -440,6 +440,12 @@ take a `policyType` select param. The workflow is *not* MULT-only: the legacy
Params: ramo (with an "todos" option), aseguradora, date range on `policyFrom`.
Columns: póliza, cliente, ramo, aseguradora, vigencia, prima neta, forma de pago.
️ `forma de pago` became a real column on 2026-08-18 (`Policy.paymentFrequency`).
It is **null on every policy migrated before that date** — the original ETL
marked Access's `FORMA PAGO` consumed and then never wrote it anywhere — so the
report must render null as "—" rather than assuming annual. Running
`backfill_policy_premium_breakdown.py` recovers it from the staged Parquet.
Totals: count + prima neta sum per currency (**never collapse MXN and USD** —
same constraint as the billing module).
+345 -18
View File
@@ -34,7 +34,7 @@ was **reused, not copied**.
|---|---|
| API module | `apps/api/src/policy-ocr/` (service, controller, DTOs, matcher, parser) |
| Shared OCR seam | `apps/api/src/ocr/ocr.module.ts` |
| Tables | `policy_ocr_batches`, `policy_ocr_documents` (`20260801000000_policy_ocr_intake`) |
| Tables | `policy_ocr_batches`, `policy_ocr_documents` (`20260801000000_policy_ocr_intake`, extended by `20260815120000_policy_ocr_ana` and `20260815160000_policy_type_repair`) |
| Web | `components/PolicyCaptura.tsx` (tab shell), `PolicyOcrIntake.tsx` (upload), `PolicyOcrReview.tsx` (review queue) |
| Abilities | `policy:ingest`, `policy:ocr-review` — both **STAFF** |
@@ -84,8 +84,9 @@ feature's core assumption.
Utility statements arrive **bundled, one customer per page** — so there, one
page is one document and the parser runs per page. A policy PDF is the
opposite: the GMX certificate is a 2-page document where page 1 carries the
contract header and page 2 carries the per-coverage table, and **both pages
describe the same policy**. So the pipeline concatenates every page's text
contract header and page 2 carries the per-coverage table (and the PVL
especificación runs to ten), and **every page describes the same policy**. So
the pipeline concatenates every page's text
(`\n\n` between pages, which also keeps `ocrRawText` readable for debugging)
and runs the parser and the matcher exactly **once per file**.
@@ -133,6 +134,203 @@ customers do occur (one group policy bound by two related parties), and
picking arbitrarily would silently book the wrong coverage against the wrong
person.
### Name suggestions on the zero-hit path
When the policy number finds nothing — the new-policy case, where a human has
to pick a customer anyway — `name-matcher.ts` ranks the customer book against
the printed insured name and the review screen offers the top three as
one-click buttons above the picker. They are written to
`policy_ocr_documents.customerSuggestions`, deliberately **not** to
`matchCandidates`, so a name hint can never be read as a policy-number hit.
Nothing sets `matchedCustomerId`; the rule above is unchanged.
The problem is only ordering: the office books customers surname-first
(`WAGONER, PAMELA`) and carriers print them given-name-first
(`PAMELA DENISE WAGONER`), so a string compare never hits while a **token-set**
compare does. Names are normalized (accents folded, so OCR's `MUNOZ` reaches
the book's `MUÑOZ`; initials, `DE`/`LA`/`Y`, `JR`, `S.A. DE C.V.` and any token
containing a digit dropped — ANA prints the phone hard against the name as
`Ph.3102001538`). Two tiers:
| Tier | Rule |
|---|---|
| `EXACT` | identical token sets, any order |
| `PARTIAL` | one set contains the other, ≥2 shared tokens, **and** the surname is present |
Both thresholds come from measuring the real book (1536 customers):
- 1487 distinct token sets, so `EXACT` cross-person collisions are ~0
- loosen to surname + first given name and 131 customers (8.5%) collide —
the book holds `MCWILLIAMS, BRIAN MICHAEL` *and* `MCWILLIAMS, BRIAN`
- 185 surnames are shared by 524 customers, so one token is never evidence;
hence the ≥2 floor and the explicit surname requirement, which is what stops
`JERRY MARILYN` reaching `ESTRADA, JERRY & MARILYN` on given names alone
Replaying every book row as a carrier would print it (given-name-first, joint
spouse dropped): 97.9% top-ranked correct, 0.9% no suggestion, 1.2% a
different row — and all but two of those are the same human on a duplicate or
variant row (`MOLNAR, JANOS` vs `MOLNAR, JANOS`, `IBARRA, ISMAEL &`). The
two genuine wrong-person cases are `CUADROS, JORGE JR` against three
`CUADROS, JORGE H.`, and they appear as a tie in the list rather than as a
single answer.
A blob is refused outright (>8 tokens or >80 characters): GMX's especificación
has no field labels and the parser has been seen handing its whole first page
over as `insuredName`, which would find a surname somewhere in the prose.
`(SIN NOMBRE)` — 14 rows the migration left — is skipped on both sides.
**Not used for utility statements.** There the registrant genuinely is not the
customer (the `CATT, RANDY` finding above), so the same trick would be wrong,
not merely noisy.
## GMX ships two unrelated documents for the same policy
The office downloads both from the same portal, and either can land in a
batch. They share only the brand and the policy number, so `parseGmx` is a
two-line dispatcher over two real parsers — both returning `provider: "GMX"`,
because the matcher keys on the policy number alone and must not care which
artifact was uploaded.
| | **Caratula** (`…_Traduccion.pdf`) | **Especificación** (`…-CondicionesParticulares.pdf`) |
|---|---|---|
| Language | English (free translation) | Spanish |
| Shape | boxed header table + 4-column coverage table | 10 pages of prose, no tables at all |
| Header fields | Policy / Insured / Broker / Term / From / To / Currency | insured name, risk location, property description |
| Dates, broker, currency field | yes | **none printed** |
| Coverages | one row per risk | section heading + `Límite Máximo de Responsabilidad:` |
| Parser | `parseGmxCaratula` | `parseGmxEspecificacion` |
Selected by `isEspecificacion` on the PVL page header
(`ESPECIFICACIÓN QUE SE ADHIERE`, `PVL Hogar`, `Nombre del asegurado`).
Three things about the especificación are worth knowing before touching it:
- **The policy number's group widths differ between the two.** The caratula
reads `007-037-07005947-0000-02` and the especificación
`07-037-07006957-00000-01` — 2 digits in the first group, 5 in the fourth.
The original parser pinned the widths, so it read one family and returned
null on the other. `POLICY_NUMBER_SHAPE` now matches the shape, and since
the especificación prints the number on all ten page headers, the ten
readings cross-check each other (disagreement is noted, not resolved — the
same rule the zona federal parser applies to its clave).
- **Coverages are found by anchoring on the limit label and walking backwards
for the heading.** There is no row shape to match. A heading is a short line
*preceded by a blank line* — that last condition is the whole trick, since
length alone cannot tell a heading from the wrapped tail of the paragraph
above it (`efectuados.`, `Y CADA PÉRDIDA.`), and without it coverages get
named after the last word of the preceding prose.
- **Vigencia, agente and prima are absent by design**, not unread. The parser
says so in a note, so a reviewer seeing three empty fields does not read it
as a broken parse.
**These three are keyed in by hand** — confirmed 2026-08-14 with Luz, who
handles GMX policies at the office. The review screen already has editable
inputs for all three, and `postPremium` enables off the *typed* premium, so
a hand-entered prima posts to the ledger exactly like a parsed one. No code
change was needed to support this; it is a process decision, recorded here
because the parser's own note now instructs the reviewer accordingly.
> **A blank vigencia is silently permanent.** `Policy.policyTo` is nullable
> and the renewals window query filters `policyTo: { gte, lte }`
> (`renewals.service.ts`), so a policy confirmed without one **never matches
> and never gets a renewal notice** — no error, no warning, and nothing
> later notices. This is why the parser's note names the consequence instead
> of just listing the missing fields.
An excluded catastrophic risk is recorded as excluded **in the risk label**
(`Terremoto o erupción volcánica — Sección Edificio: EXCLUIDO`) with a null
amount, never as `0`: a coverage insured for zero and an excluded coverage are
the same number and very different facts, and `ParsedCoverage` has no field
for the distinction.
## A.N.A. ships two unrelated faces too
`A.N.A. Compañía de Seguros` is the Rosarito office's tourist auto book. Same
split as GMX, different reason: GMX ships two *documents about one policy*,
A.N.A. ships two *products*.
| | **AUTOMOBILE** (`SPECIAL POLICY FOR TOURISTS`) | **DRIVER´S POLICY** (the office says *licencia*) |
|---|---|---|
| Insures | a specific car | up to five named drivers, whatever they drive |
| Vehicle table | `ITEM / YEAR / MAKE / BODY / SERIAL No. / PLATES` | **none** |
| Insured | one `INSURED` cell | numbered `POLICY HOLDER` list |
| Value columns | one (`LIMIT OF LIABILITY`) | two (`SUM INSURED`, `PREMIUM`) |
| Sections | 9, numbered | 6, unnumbered, **in a different order** |
| Parser | `parseAnaAutomobile` | `parseAnaDriverPolicy` |
Selected by `isAnaDriverPolicy` on the title band.
The **four automobile products** the office sells — amplia and responsabilidad
civil, each annual or by-the-day — are the **same layout with different
numbers**. "Amplia" prints a vehicle value and `COVERED` on sections 12;
"resp. civil" prints `0.00` and `EXCLUDED`. That is data, not a layout, so
there is one parser rather than four.
Things worth knowing before touching the ANA parsers:
- **These are born-digital portal PDFs**, so `pdftotext -layout` returns exact
glyphs and exact columns. The driver's policy parser uses that: `SUM
INSURED` and `PREMIUM` print the same shape (`100,000.00 usd.` /
`18.70 usd.`) with no per-row label, so **horizontal position is the only
thing that separates them**. The split is computed from the header's own
column offsets rather than hardcoded, because they shift between products.
If a scan ever arrives without column fidelity, every amount is reported as
a sum insured and the reviewer is told the split failed.
- **The money row is read positionally, not by finding six amounts.** An
unused `DISCOUNT` prints as a bare `-`, so an "amounts in order" reading
shifts every value one column left on a discounted policy. The parser
requires exactly six whitespace-separated cells or reports the row unread.
- **Each PDF prints its face two or three times** (ORIGINAL, AGENT COPY, then
a summary receipt and three travel ID cards), and the pipeline concatenates
every page before parsing. The coverage walk is bounded to the first copy
and the driver list to the first `POLICY HOLDER` block. Unbounded, the
licencia returns the same person three times — which reads as a
three-driver policy, not as a bug, so nothing downstream would catch it.
- **Two five-digit numbers sit in the header band** and only one is the agent
clave: the other is the agent's own postal code
(`ROSARITO, BAJA CALIFORNIA 22710`). Likewise the agent's street address
reads `BENITO JUAREZ 25 No.50 INT 38`, three lines above the `No.` cell that
holds the policy number — hence the two-space floor after `No.`.
- **Sections 68 print a PREMIUM where the others print a limit.** $40 is what
legal aid *cost*, not a $40 liability limit, so it lands on
`ParsedCoverage.premium` (a field GMX never fills) and gets its own column
on the review screen. Adding the two together would be meaningless.
- **`coveragePeriodDays` matters here and nowhere else.** A.N.A. sells 3- and
4-day policies. `Policy.coveragePeriodDays` defaults to 365, so a weekend
policy left at the default sits in the renewals window a year out. The term
is derived from the two dates and cross-checked against the printed `DAYS`
cell; a disagreement is noted rather than resolved.
Exclusions follow the GMX rule — recorded in the risk label
(`MATERIAL DAMAGE — VEHICLE: EXCLUDED`) with a null amount, never as `0`. It
matters more here: a responsabilidad-civil policy prints `0.00` for material
damage, so the two are visually identical on the page.
### Vehicles and drivers
A.N.A. is the first provider whose face carries either, so confirm now writes
`Vehicle` and `InsuredDriver` rows alongside the `Policy`
(`applyVehiclesAndDrivers`). The parsed values are stored on the document as
`extractedVehiclesJson` / `extractedDriversJson` and shown read-only in the
review queue, so a misread VIN is catchable before it is applied.
Both inserts skip a row that already exists on the policy, matched on the
identifier the document prints — VIN then plate for a vehicle (A.N.A.'s
TRAILER and TOWING slots have no VIN), licence number then name for a driver.
The case that forces this is confirming a **renewal** onto an existing policy:
a blind insert leaves the customer with the same VIN listed twice and no way
to tell which row the renewal belongs to.
Nothing is ever updated or deleted there. A vehicle whose plate changed lands
as a second row for a human to reconcile — the safe half of the mistake, since
an overwrite would destroy the only record of what was insured last term.
The vehicle table is parsed **by token role, not by column offset**, because
`BODY` is the cell that wraps: `PACIFICA` is one token and `GENESIS SEDAN` is
two, so a fixed token count reads the VIN out of the wrong slot on the second.
The 17-character VIN is the anchor and `BODY` is whatever sits between the
make and it.
## What the parser reads, and the field it cannot
`ParsedPolicy` fields are all nullable on purpose: each carrier prints a
@@ -145,6 +343,12 @@ insured, broker (→ `Policy.agentName`), legal address, ZIP, `policyFrom` /
per-coverage table (risk, insured amount, deductible, loss participation)
preserved verbatim.
Read from an A.N.A. face: all of the above except additional insured and
broker parens, **plus** the premium (A.N.A. prints it — see below), the policy
fee, the total, the term in days, the vehicle table, and the named drivers
with their US licence numbers. The tax and the agent clave have no column in
the schema and ride in the notes.
> **The GMX certificate carries no premium.** Not "sometimes missing" — the
> document does not have the figure. It lives on GMX's **separate `recibo`
> PDF**. The parser leaves `netPremium` / `policyFee` / `brokerFee` / `total`
@@ -157,20 +361,92 @@ This is also why confirm never overwrites an existing `Policy.netPremium`
with null: the certificate not carrying a premium is not evidence that the
premium is gone.
A.N.A.'s faces do print one — the `DISCOUNT / PREMIUM / POLICY FEE / TAX /
LOCAL TAX / TOTAL` row is on the same page — so an ANA document reaches the
review queue with `netPremium` populated and `postPremium` already ticked.
Four of those six cells are stored: `PREMIUM``netPremium`, `POLICY FEE`
`policyFee`, `TAX``tax` (`extractedTax` on the document, `Policy.tax` on
confirm), `TOTAL``total`. `DISCOUNT` and `LOCAL TAX` are reported as notes
instead:
- **`DISCOUNT`** has no column, and it prints as a bare `-` when unused, which
is what makes the row positional rather than "find six amounts".
- **`LOCAL TAX`** is a separate levy and is deliberately **not** summed into
`tax`. Folding it in would produce an IVA figure that no longer divides back
to a rate, which is the reason to store it at all. It reads 0.00 on every
A.N.A. policy seen so far; a non-zero one raises
*"impuesto local N no capturado"* and means `total` will not reconcile
against `netPremium + policyFee + tax`.
`Policy.taxRate` is left null by confirm. A.N.A. prints the IVA **amount**, not
the rate, and back-dividing one would mint a rate the document never stated —
the capture form resolves it from the line of business instead
(`PolicyType.taxRate`, see `apps/api/src/policies/premium.ts`). The figures do
agree: 298.61 + 30.00 taxed at 8% is 26.29, totalling 354.90, asserted in
`policy-parser.spec.ts`.
Deductible and loss participation are stored as **strings** (`"5%"`, `"20%"`,
`"USD 1,000"`) — they are printed as a mix of percentages, currency amounts
and free text, and normalising them would lose the distinction.
## Policy type and carrier
Confirm sets `Policy.policyTypeId` and `Policy.insuranceProviderId` from what
the parser read.
| document | `policyTypeName` |
|---|---|
| ANA `AUTOMOBILE` | `AUTO` |
| ANA `DRIVER´S POLICY` | `LICENCIAS` |
| GMX caratula **and** especificación | `MULT` |
The parser emits a **name**, never an id — it is a pure function over text and
must not reach for the database, so `resolveLookups()` in the service turns the
name into a foreign key. A renamed lookup row is then a data change rather than
a parser change.
**Resolve, never create.** A missing `policy_types` row means a human deleted
it, and silently recreating it would undo that with no record. The field stays
null and the reviewer adds the row through the lookups screen. An explicit
`policyTypeId` / `insuranceProviderId` on the confirm payload always wins.
Two judgement calls worth recording:
- **GMX is `MULT`, not `INCENDIO`.** The caratula's own header reads "Multiple
Policy / Home" and the especificación is "PVL Hogar" — one product, two
artifacts. `MULT` is the live row carrying 769 of them; `INCENDIO` is
fire-only and no policy in the book has ever used it.
- **The parser's provider code is not the carrier's row name.** The office's
book is filed under `ANA SEGUROS`, so `PROVIDER_ROW_NAME` maps `ANA` onto it.
A bare `ANA` row with 1 policy also existed and is merged away by
`20260815160000_policy_type_repair`.
> **Deleting a lookup row used to be silent data loss.** `policies.policyTypeId`,
> `policies.insuranceProviderId` and `claims.adjusterId` are all
> `ON DELETE SET NULL`, and the lookups screen deleted unconditionally — so the
> delete returned 200 and blanked the field on every row that used it. That is
> how `M_EMPR` vanished and left 5 policies with no ramo, found months later by
> querying. All three deletes now refuse while the row is in use, naming it and
> the count. See `assertLookupUnused` and BACKLOG §2.1.
## Confirm: what actually gets written
Per confirmed document, in order:
1. **The `Policy` row** — updated if a policy was matched, created under the
picked customer if not. Only non-null `extracted*` fields are written; null
never overwrites existing data.
2. **A `PolicyDocument`** — the source PDF is streamed into the policy's
storage namespace and attached, so the paperwork stays with the policy.
3. **Optionally a `Transaction`**`INSURANCE` domain, negative amount
never overwrites existing data. `policyTypeId` and `insuranceProviderId` are
resolved first (above) and left untouched when unresolvable, so an existing
policy never loses a type or carrier it already had.
2. **`Vehicle` and `InsuredDriver` rows** — for the providers whose face
carries them (A.N.A.; never GMX Hogar), skipping any that already exist on
the policy. See *Vehicles and drivers* above.
3. **A `PolicyDocument`** — the source PDF is streamed into the policy's
storage namespace and attached, so the paperwork stays with the policy. Its
`documentType` is named after whichever parser claimed the page
(`ANA_POLICY`, `GMX_POLICY`).
4. **Optionally a `Transaction`**`INSURANCE` domain, negative amount
(a charge), `captureSource: "OCR"`, `captureRef` = the document id.
The ledger write is **opt-in twice over**: staff must tick `postPremium`
@@ -210,20 +486,71 @@ feature is disabled.
## Tests
`apps/api/src/policy-ocr/parsers/policy-parser.spec.ts` — 8 cases, all
against verbatim text extracted from one real document,
`HC_Folio_000767_Traduccion.pdf`: provider detection from the wordmark and
from the footer URL, the header fields, every coverage row off the second
page, the deductible/loss-participation strings, the missing-premium note,
the broker line with the agent-number parens absent, and a page with no GMX
signal at all (which must yield no provider rather than a bad guess).
`apps/api/src/policy-ocr/parsers/policy-parser.spec.ts`58 cases against
verbatim text extracted from five real documents, indentation and blank lines
included (the column positions are what the parser reads, so a cleaned-up
fixture would test nothing — and on A.N.A.'s driver's policy the offsets are
literally the only thing separating two columns).
From `HC_Folio_000767_Traduccion.pdf` (caratula): provider detection from the
wordmark and from the footer URL, the header fields, every coverage row off
the second page, the deductible/loss-participation strings, the
missing-premium note, the broker line with the agent-number parens absent,
and a page with no GMX signal at all (which must yield no provider rather
than a bad guess).
From `007_LGS-HGMX_07006957_01_0-CondicionesParticulares.pdf`
(especificación): the differently-grouped policy number, the risk location
read across its wrapped line, the empty `Asegurado Adicional` cell that must
not capture the next line, the absent-by-design fields, currency taken from
the USD limits rather than the M.N. sublimits in the body prose, a limit
split under `Edificio` / `Contenidos` sub-labels, a limit printed on the
label's own line, a deductible stated as a sentence *above* its limit, a
sublimit block whose amount sits after both a blank line and a page break,
the excluded earthquake coverage, and the hydrometeorological deductible and
coinsurance pulled from their own per-zone block.
Plus four cases in `apps/api/src/policies/lookup-delete-guard.spec.ts` pinning
the refusal that stops a lookup delete from silently blanking the rows that use
it, and four in the parser suite on the policy-type NAME each document yields.
From the three A.N.A. PDFs: brand detection (and that GMX's layout rules
cannot claim an ANA page), the header band, DD MM YYYY read out of three
separate column cells, the six money cells with `DISCOUNT` printed as a bare
`-`, the vehicle row with a one-word and a two-word `BODY` cell, the empty
TRAILER/TOWING slots, the agent street number and postal code that must *not*
be read as the policy number and clave, the declared values labelled by item
slot, the `$500.00` inside the deductible sentence that is not a sum insured,
the per-person/per-accident split in both of its printed forms, an add-on's
figure recorded as a premium, section 9's parenthesised limit, the by-the-day
term, the excluded sections, the column-position split on the driver's policy,
its different section order, and — for both faces — that a doubled or tripled
input yields one set of coverages and one driver rather than one per copy.
`apps/api/src/policy-ocr/name-matcher.spec.ts` — 21 cases on the customer name
suggestions, every fixture name lifted from the real book: the reversed name,
the printed middle name, the exact row outranking the row that merely contains
it, the joint account reached from one spouse (and refused when only given
names are printed), the Spanish double surname with the comma in either place,
the 54 rows with no comma at all, the `(SIN NOMBRE)` placeholder, a bare shared
surname, and the page-sized blob. Four more in
`policy-matcher.service.spec.ts` pin the wiring: suggestions on the zero-hit
and unreadable-number paths, no book read at all when the policy number hits,
and one book read across a batch.
Four of the GMX cases are regression tests for ways the parser can silently attach
the *wrong* value rather than none — a neighbouring coverage's prose read as
a deductible, the page-level `DEDUCIBLES:` paragraph read as one, a coverage
named after a wrapped prose tail, and one section's per-zone deductible
adopted by the coverage above it. Each was a real defect caught by running
the parser against the full ten-page document.
## Not built
- **Only GMX.** The dispatcher (`detectPolicyProvider`) is a table of
`[provider, pattern]` pairs plus a `parsers` map, so adding ANA or Qualitas
is a parser function and two entries — but no other carrier's layout has
been seen yet, and guessing at one produces a parser nobody can verify.
- **GMX and A.N.A. only.** The dispatcher (`detectPolicyProvider`) is a table
of `[provider, pattern]` pairs plus a `parsers` map, so adding Qualitas is a
parser function and two entries — but no other carrier's layout has been
seen yet, and guessing at one produces a parser nobody can verify.
- **The `recibo` PDF.** Reading the premium off GMX's separate receipt
document, and pairing it to the certificate it belongs to, is the obvious
next piece. It is what would let `postPremium` stop being a manual tick.
@@ -0,0 +1,249 @@
"""
Recovers the premium breakdown the original policy transform dropped.
`transform_policies.py` modeled prima neta, derecho de póliza and comisión and
nothing else, which lost two things from every migrated policy:
1. RECARGO — the financing surcharge on a policy paid in more than one
exhibición, plus the whole second money row (`p_neta_2`, `recargo_2`,
`d_pol_2`, `com_2`) that a semestral policy carries because each payment
is priced separately. These were not deleted, they were swept into
`policies.coveragesJson` as loose strings alongside the real coverages —
unqueryable, and mislabeled as coverage amounts.
2. FORMA PAGO — dropped outright. The column was marked "consumed" by the
transform's coverage sweep but never written to any column, so it exists
nowhere in the platform database. It is the field that decides whether a
recargo is legitimate on a row at all, so it cannot be inferred back from
the money.
The transform has been fixed in the same commit, so a full `run_all.py` now
produces all of this directly. This script exists for a database that must not
be re-imported: it reads the same staged Parquet and patches in place.
What it does NOT do: invent IVA or the printed TOTAL. Those were never columns
in the home tables — they were unbound calculated controls on the Access form —
so there is genuinely nothing to recover, and both stay null until a human
edits the policy. The app computes them (apps/api/src/policies/premium.ts).
Idempotent, and never overwrites a non-null value: a figure a human has since
corrected in the app wins over the legacy one.
./.venv/bin/python backfill_policy_premium_breakdown.py --env dev
"""
from __future__ import annotations
import json
import sys
from decimal import Decimal, InvalidOperation
from pathlib import Path
import pandas as pd
from dbenv import connect
from sync import parse_mode
STG = Path(__file__).parent / "output" / "stg_seguros"
LEGACY_DB = "SEGUROS 16_be"
NULL = ""
# Legacy column -> what it is, per source table. Only the tables that actually
# carry a breakdown appear; the auto tables have no recargo and no second row.
HOME_TABLES = ("mult", "incendio", "m_empr")
# Keys the transform used to dump into coveragesJson that are now real columns.
# Stripped once migrated so the blob stops pretending they are coverages.
MIGRATED_COVERAGE_KEYS = (
"recargo", "recargo_2", "p_neta_2", "d_pol_2", "com_2",
)
_FREQ = {
"ANNUAL": "ANNUAL",
"ANUAL": "ANNUAL",
"SEMESTRAL": "SEMIANNUAL",
"TRIMESTRAL": "QUARTERLY",
"MENSUAL": "MONTHLY",
"CONTADO": "SINGLE",
}
def s(v):
if v is None or pd.isna(v):
return None
v = str(v).strip()
return None if v in ("", NULL) else v
def dec(v):
v = s(v)
if v is None:
return None
try:
return Decimal(v.replace(",", ""))
except (InvalidOperation, ValueError):
return None
def freq(v):
return _FREQ.get((s(v) or "").upper())
def load(name):
df = pd.read_parquet(STG / f"{name}.parquet").sort_values("_row_num").reset_index(drop=True)
for c in df.columns:
if c != "_row_num":
df[c] = df[c].astype("string").str.strip()
return df
def main():
env, _sync_mode = parse_mode()
# Fails closed rather than reporting a clean run over nothing: an empty
# staging directory and a policy set with no recargo look identical from
# the database side, and "0 rows updated" would read as success.
if not STG.exists():
print(f"[policy-premium] staged Parquet missing at {STG} — run extract/load first.")
return 3
conn = connect(env)
c = conn.cursor()
print(f"[policy-premium] target env: {env}")
# Every policy that came from the insurance ETL, keyed by provenance. The
# id is needed to reach the installments, coveragesJson to strip the keys.
c.execute(
"SELECT legacySourceTable, legacyId, id, coveragesJson "
"FROM policies WHERE legacySourceDb = %s AND legacyId IS NOT NULL",
(LEGACY_DB,),
)
by_key = {(t, lid): (pid, cov) for t, lid, pid, cov in c.fetchall()}
print(f" {len(by_key)} migrated polic(ies) in the target database")
pol_updates = [] # (surcharge, paymentFrequency, coveragesJson, policyId)
inst_updates = [] # (netPremium, surcharge, policyFee, commission, policyId, seq)
seen_tables = 0
for table in HOME_TABLES + (
"tabla_autos", "tabla_autos_ampl", "tabla_autos_limit",
"tabla_autos_ampl_r", "tabla_autos_rc_r", "mca2", "licencias",
):
path = STG / f"{table}.parquet"
if not path.exists():
continue
seen_tables += 1
df = load(table)
home = table in HOME_TABLES
for _, row in df.iterrows():
key = (table, str(int(row["_row_num"])))
hit = by_key.get(key)
if not hit:
continue
pid, cov_raw = hit
surcharge = dec(row.get("recargo")) if home else None
frequency = freq(row.get("forma_pago"))
# Strip the now-modeled keys out of the coverage blob. Rewritten
# only when something actually changes, so a policy whose blob a
# human has edited is left byte-identical.
cov_new = None
if cov_raw:
try:
cov = json.loads(cov_raw) if isinstance(cov_raw, str) else cov_raw
except (TypeError, ValueError):
cov = None
if isinstance(cov, dict):
kept = {k: v for k, v in cov.items() if k not in MIGRATED_COVERAGE_KEYS}
if len(kept) != len(cov):
cov_new = json.dumps(kept, ensure_ascii=False) if kept else None
if surcharge is not None or frequency is not None or cov_new is not None:
pol_updates.append((surcharge, frequency, cov_new, cov_new is not None, pid))
# Per-payment breakdown. Slot 1 is the unsuffixed money row, slot 2
# the _2 twin; the auto tables have a single slot and no recargo.
if home:
slots = [
(1, "p_neta", "recargo", "d_pol", "com"),
(2, "p_neta_2", "recargo_2", "d_pol_2", "com_2"),
]
else:
pn = "prima1" if table == "mca2" else "prima_neta"
dp = "d_poliza1" if table == "mca2" else "d_poliza"
slots = [(1, pn, None, dp, None)]
for seq, pn, rc, dp, cm in slots:
vals = (
dec(row.get(pn)) if pn else None,
dec(row.get(rc)) if rc else None,
dec(row.get(dp)) if dp else None,
dec(row.get(cm)) if cm else None,
)
if all(v is None for v in vals):
continue
inst_updates.append((*vals, pid, seq))
if not seen_tables:
print(f"[policy-premium] no policy tables staged under {STG} — nothing to do.")
return 3
# COALESCE on every target: a column a human has already filled in the app
# keeps its value, the legacy figure only lands where there is a hole.
for surcharge, frequency, cov_new, rewrite_cov, pid in pol_updates:
c.execute(
"UPDATE policies SET "
" surcharge = COALESCE(surcharge, %s), "
" paymentFrequency = COALESCE(paymentFrequency, %s), "
" coveragesJson = IF(%s, %s, coveragesJson) "
"WHERE id = %s",
(surcharge, frequency, 1 if rewrite_cov else 0, cov_new, pid),
)
for netp, surch, fee, comm, pid, seq in inst_updates:
c.execute(
"UPDATE policy_payment_installments SET "
" netPremium = COALESCE(netPremium, %s), "
" surcharge = COALESCE(surcharge, %s), "
" policyFee = COALESCE(policyFee, %s), "
" commission = COALESCE(commission, %s) "
"WHERE policyId = %s AND sequence = %s",
(netp, surch, fee, comm, pid, seq),
)
conn.commit()
print(f" policies : {len(pol_updates)} row(s) touched")
print(f" installments: {len(inst_updates)} row(s) touched")
# --- validation ---------------------------------------------------------
c.execute("SELECT COUNT(*) FROM policies WHERE surcharge IS NOT NULL AND surcharge <> 0")
n_surch = c.fetchone()[0]
c.execute("SELECT COUNT(*) FROM policies WHERE paymentFrequency IS NOT NULL")
n_freq = c.fetchone()[0]
c.execute(
"SELECT COUNT(*) FROM policies "
"WHERE paymentFrequency IN ('ANNUAL','SINGLE') AND surcharge IS NOT NULL AND surcharge <> 0"
)
n_bad = c.fetchone()[0]
c.execute(
"SELECT COUNT(*) FROM policies WHERE coveragesJson IS NOT NULL "
"AND JSON_EXTRACT(coveragesJson, '$.recargo') IS NOT NULL"
)
n_left = c.fetchone()[0]
print(f" -> policies with a recargo : {n_surch}")
print(f" -> policies with a forma pago : {n_freq}")
print(f" -> recargo still in coverages : {n_left}")
# A surcharge on an annual policy contradicts the rule the capture form
# enforces, so it is worth surfacing rather than leaving for someone to
# find in a total. It is a warning, not a failure: the books are the books.
if n_bad:
print(f" !! {n_bad} annual/contado polic(ies) carry a non-zero recargo — review by hand")
return 0
if __name__ == "__main__":
sys.exit(main() or 0)
+57
View File
@@ -18,6 +18,7 @@ the four Access source files.
"""
import os
import re
from pathlib import Path
# The folder holding the four Access source files. Overridable via INGEST_DIR so
@@ -95,3 +96,59 @@ SOURCES = {
},
},
}
# --- prior-period archives ----------------------------------------------
#
# Legacy ran a year-end *corte*: it summed the closing year, wrote that total
# back as each customer's Jan-1 BALANCE FORWARD, and started the next year
# clean. Access keeps the closed year as a whole-database snapshot named for
# the period it holds — `2025.accdb` is UTILITIES as it stood when 2025 was
# cut — and the office archives one per year.
#
# Only the ledger is staged out of a snapshot. Everything else in it (DATMEX,
# PROFILE, EFECTIVO, ...) is a year-stale copy of a table the live
# UTILITIES.accdb already provides, and staging all ~50 of them would triple
# the extract time to import data we would then have to ignore. DATGRAL comes
# along solely to check that a NUMid still means the same customer it did that
# year; see the recycle guard in transform_transactions.py.
#
# The cash side is deliberately NOT taken from the snapshot: `EFECTIVO` is a
# lifetime journal, so the snapshot's copy is a subset of the live one and
# importing it would double-book every prior-year receipt.
PERIOD_FILE_RE = re.compile(r"^(\d{4})\.accdb$", re.IGNORECASE)
PERIOD_TABLES = {"datos2", "DATGRAL"}
def period_schema(year: int) -> str:
return f"stg_period_{year}"
def discover_periods(root: Path) -> dict[str, dict]:
"""Find every `YYYY.accdb` archive sitting in the ingest folder.
Discovery is by filename because that is the whole upload contract: the
operator drops `2025.accdb` on the Operaciones page and the period is 2025.
Nothing inside the file names the year — a snapshot's `datos2` looks
identical to the live one — so the name is the only declaration of intent
we get, and it is what the allowlist on the upload endpoint enforces.
"""
found: dict[str, dict] = {}
if not root.is_dir():
return found
for path in sorted(root.iterdir()):
m = PERIOD_FILE_RE.match(path.name)
if not m:
continue
year = int(m.group(1))
found[f"period_{year}"] = {
"path": path,
"schema": period_schema(year),
"exclude": set(),
"include": set(PERIOD_TABLES),
"period_year": year,
}
return found
SOURCES.update(discover_periods(SOURCE_ROOT))
+11
View File
@@ -40,6 +40,17 @@ def stage_source(source_name: str, source_cfg: dict, sink) -> None:
tables = extract.list_tables(cnxn)
excluded = source_cfg["exclude"]
# A source may name the only tables it is worth staging. Prior-period
# archives do: they are whole-database snapshots, but everything in them
# except the ledger is a year-stale copy of a live table, so staging the
# rest costs minutes per file to produce data nothing reads.
include = source_cfg.get("include")
if include is not None:
missing = include - set(tables)
if missing:
print(f" [WARN] {source_name}: missing expected table(s) {sorted(missing)}", file=sys.stderr)
tables = [t for t in tables if t in include]
for table_name in tables:
if table_name in excluded:
print(f" [exclude] {table_name}")
+12
View File
@@ -59,6 +59,12 @@ STEPS = [
# touches.
"backfill_statement_match_fields.py",
"transform_policies.py",
# Premium breakdown (recargo, per-payment figures, forma de pago).
# transform_policies.py now writes these directly, so on a full rebuild
# this is a no-op that re-asserts them; on a database migrated before the
# breakdown existed it is what recovers them out of coveragesJson.
# Must follow transform_policies.py, which truncates the installments.
"backfill_policy_premium_breakdown.py",
"transform_transactions.py",
"prune_empty_customers.py",
# Seeds the Scotiabank chequera that every SCOTHIA movement is booked into;
@@ -78,6 +84,12 @@ SYNC_STEPS = [
# touches.
"backfill_statement_match_fields.py",
"transform_policies.py",
# Premium breakdown (recargo, per-payment figures, forma de pago).
# transform_policies.py now writes these directly, so on a full rebuild
# this is a no-op that re-asserts them; on a database migrated before the
# breakdown existed it is what recovers them out of coveragesJson.
# Must follow transform_policies.py, which truncates the installments.
"backfill_policy_premium_breakdown.py",
"transform_transactions.py",
# Manual-safe prune: drops legacy-owned empties that the customer upsert
# re-creates from Parquet, but leaves manually-added customers alone.
+72 -12
View File
@@ -18,6 +18,17 @@ Design (validated against staged data):
policies are skipped and counted (required FK).
- Payment slots: c_1er_pago is the first amount, pago_subsec the recurring
amount for slots 2-4; efectivo is a cash flag, no_cheque the check ref.
- Premium breakdown: a policy paid in more than one exhibicion prices EACH
payment separately, which is why the home tables carry the whole money row
twice (p_neta/recargo/d_pol/com and their _2 twins). The unsuffixed set is
the policy header, the suffixed one belongs to payment 2, and both are
written per installment as well. This used to be lost: `recargo` and the
_2 columns fell into coveragesJson as loose strings and forma_pago was
marked consumed but never written anywhere at all.
- IVA and the printed TOTAL are NOT in Access for the home tables. They were
unbound calculated controls on the form, so there is nothing to migrate;
the app computes them (apps/api/src/policies/premium.ts) from
(p_neta + recargo + d_pol) * rate.
- Any source column not explicitly modeled (coverage amounts: edificio,
contenidos, robo, cristales, ...) is preserved verbatim in coveragesJson,
so nothing is lost in consolidation.
@@ -93,20 +104,55 @@ def truthy(v):
return (s(v) or "0").lower() in {"1", "-1", "true", "si", "", "yes", "x"}
# Access FORMA PAGO -> PaymentFrequency. The whole staged corpus holds exactly
# four spellings (ANNUAL 1851, SEMESTRAL 29, semestral 2, CONTADO 1); anything
# else is left null rather than guessed, because the value decides whether a
# recargo is legitimate on the row.
_FREQ = {
"ANNUAL": "ANNUAL",
"ANUAL": "ANNUAL",
"SEMESTRAL": "SEMIANNUAL",
"TRIMESTRAL": "QUARTERLY",
"MENSUAL": "MONTHLY",
"CONTADO": "SINGLE",
}
def freq(v):
return _FREQ.get((s(v) or "").upper())
def slot_dec(row, slot, key):
"""One figure of a payment's premium breakdown, or None when the source
table has no such column. Deliberately not zero: a zero d_pol on the second
payment is a real figure in the books and must stay distinguishable from
'this table never had that column'."""
col = slot.get(key)
return dec(row.get(col)) if col else None
# --- per-table config ------------------------------------------------------- #
# fields: policy column -> source column. installments: list of slot dicts.
# vehicles: 'trip_underscore' | 'single' | 'mca2' | None. drivers: 'mca2' |
# 'licencias' | None.
# `pneta`/`recarg`/`dpol`/`com` on a slot are that payment's own share of the
# premium. Only the first two payments have one in Access — the form only ever
# drew the money row twice — so slots 3 and 4 carry none and keep just the
# amount actually collected.
HOME_INST = [
dict(seq=1, amt="c_1er_pago", cu="moned", d="fecha_pago", ck="no_cheque", cash="efectivo"),
dict(seq=2, amt="pago_subsec", cu="moned_2", d="fecha_pago_2", ck="no_cheque_2", cash="efectivo_2"),
dict(seq=1, amt="c_1er_pago", cu="moned", d="fecha_pago", ck="no_cheque", cash="efectivo",
pneta="p_neta", recarg="recargo", dpol="d_pol", com="com"),
dict(seq=2, amt="pago_subsec", cu="moned_2", d="fecha_pago_2", ck="no_cheque_2", cash="efectivo_2",
pneta="p_neta_2", recarg="recargo_2", dpol="d_pol_2", com="com_2"),
dict(seq=3, amt="pago_subsec", cu="moned_3", d="fecha_pago_3", ck="no_cheque_3", cash="efectivo_3"),
dict(seq=4, amt="pago_subsec", cu="moned_4", d="fecha_pago_4", ck="no_cheque_4", cash="efectivo_4"),
]
HOME_FIELDS = dict(polno="no_poliza", agent="agent", comp="comp", desde="desde", hasta="hasta",
forma="forma_pago", curcol="moned", pneta="p_neta", dpol="d_pol", com="com",
forma="forma_pago", curcol="moned", pneta="p_neta", recarg="recargo",
dpol="d_pol", com="com",
liquidada="liquidada", numliq="num_liquidacion", fliq="f_liquida1", renov="renovacion")
AUTO_SINGLE_INST = [dict(seq=1, amt="total", cu="moneda", d="fecha_pago", ck="no_cheque", cash="efectivo")]
AUTO_SINGLE_INST = [dict(seq=1, amt="total", cu="moneda", d="fecha_pago", ck="no_cheque", cash="efectivo",
pneta="prima_neta", dpol="d_poliza")]
CONFIGS = {
"incendio": dict(ptype="INCENDIO", idcol="num_id", fields={**HOME_FIELDS, "curcol": "moneda"},
@@ -157,7 +203,8 @@ CONFIGS = {
forma="forma_pago", curcol="moneda", pneta="prima_neta", dpol="d_poliza",
total="total", liquidada="liquidada", numliq="num_liquidacion",
fliq="f_liquida1", renov="renovacion"),
inst=[dict(seq=1, amt="total", cu="moneda", d="fecha_pago", ck="no_cheque", cash="efectivo")],
inst=[dict(seq=1, amt="total", cu="moneda", d="fecha_pago", ck="no_cheque",
cash="efectivo", pneta="prima_neta", dpol="d_poliza")],
veh=None, drv="licencias"),
}
@@ -200,6 +247,10 @@ def main():
consumed = {cfg["idcol"], *F.values()}
for slot in cfg["inst"]:
consumed |= {slot["amt"], slot["cu"], slot["d"], slot["ck"], slot["cash"]}
# The per-payment premium columns are now modeled, so they must
# leave the coveragesJson sweep — otherwise every recargo would be
# written twice, once as a column and once as a fake coverage.
consumed |= {slot[k] for k in ("pneta", "recarg", "dpol", "com") if slot.get(k)}
for _, row in df.iterrows():
cid = cust.get(norm_id(row[cfg["idcol"]]))
@@ -237,9 +288,11 @@ def main():
dt(row.get(F.get("desde", ""))) if F.get("desde") else None,
dt(row.get(F.get("hasta", ""))) if F.get("hasta") else None,
dec(row.get(F.get("pneta", ""))) if F.get("pneta") else None,
dec(row.get(F.get("recarg", ""))) if F.get("recarg") else None,
dec(row.get(F.get("dpol", ""))) if F.get("dpol") else None,
dec(row.get(F.get("com", ""))) if F.get("com") else None,
dec(row.get(F.get("total", ""))) if F.get("total") else None,
freq(row.get(F.get("forma", ""))) if F.get("forma") else None,
cur(row.get(F.get("curcol", ""))) if F.get("curcol") else "MXN",
s(row.get("observaciones")),
json.dumps(cov, ensure_ascii=False) if cov else None,
@@ -255,9 +308,12 @@ def main():
pdate = dt(row.get(slot["d"]))
if amt is None and pdate is None:
continue
# Slot breakdown, where the source table has one.
insts.append((str(uuid.uuid4()), pid, slot["seq"], amt,
cur(row.get(slot["cu"])), pdate, s(row.get(slot["ck"])),
1 if truthy(row.get(slot["cash"])) else 0))
1 if truthy(row.get(slot["cash"])) else 0,
slot_dec(row, slot, "pneta"), slot_dec(row, slot, "recarg"),
slot_dec(row, slot, "dpol"), slot_dec(row, slot, "com")))
# vehicles
def add_vehicle(make, model, body, engine, plate, year=None, state=None):
@@ -328,16 +384,19 @@ def main():
1 if truthy(r["concluido"]) else 0, s(r["resolucion"])))
pol_cols = ("id,policyNumber,customerId,policyTypeId,insuranceProviderId,agentName,policyDate,"
"policyFrom,policyTo,netPremium,policyFee,commission,total,currency,observations,"
"policyFrom,policyTo,netPremium,surcharge,policyFee,commission,total,paymentFrequency,"
"currency,observations,"
"coveragesJson,liquidated,liquidationNumber,liquidationDate,legacySourceDb,"
"legacySourceTable,legacyId,updatedAt")
ph = ",".join(["%s"] * 23)
ph = ",".join(["%s"] * 25)
pol_upsert = (
f"INSERT INTO policies ({pol_cols}) VALUES ({ph}) ON DUPLICATE KEY UPDATE "
"customerId=VALUES(customerId),policyNumber=VALUES(policyNumber),policyTypeId=VALUES(policyTypeId),"
"insuranceProviderId=VALUES(insuranceProviderId),agentName=VALUES(agentName),policyDate=VALUES(policyDate),"
"policyFrom=VALUES(policyFrom),policyTo=VALUES(policyTo),netPremium=VALUES(netPremium),policyFee=VALUES(policyFee),"
"commission=VALUES(commission),total=VALUES(total),currency=VALUES(currency),observations=VALUES(observations),"
"policyFrom=VALUES(policyFrom),policyTo=VALUES(policyTo),netPremium=VALUES(netPremium),"
"surcharge=VALUES(surcharge),policyFee=VALUES(policyFee),"
"commission=VALUES(commission),total=VALUES(total),paymentFrequency=VALUES(paymentFrequency),"
"currency=VALUES(currency),observations=VALUES(observations),"
"coveragesJson=VALUES(coveragesJson),liquidated=VALUES(liquidated),liquidationNumber=VALUES(liquidationNumber),"
"liquidationDate=VALUES(liquidationDate),updatedAt=VALUES(updatedAt),archivedAt=NULL")
@@ -399,8 +458,9 @@ def main():
c.executemany("INSERT INTO policy_payment_installments "
"(id,policyId,sequence,amount,currency,paidDate,checkNumber,isCash) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s)", insts)
"(id,policyId,sequence,amount,currency,paidDate,checkNumber,isCash,"
"netPremium,surcharge,policyFee,commission) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", insts)
c.executemany("INSERT INTO vehicles (id,customerId,policyId,make,model,modelYear,bodyType,"
"engineNumber,licensePlate,stateCode,legacySourceTable,legacyId) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", vehicles)
+151 -4
View File
@@ -90,6 +90,62 @@ def load(src, name):
return df
def verify_corte(c, year: int) -> None:
"""Assert legacy's corte identity: SUM(period Y) == BALANCE FORWARD(Y+1).
This is the whole reason a year can be shown on its own. Legacy closed each
year by summing it and writing that total back as every customer's Jan-1
opening row for the next one, so if an archive is the right file, complete,
and attached to the right customers, its per-customer total lands exactly on
the next year's BALANCE FORWARD. A truncated export, a file dropped under
the wrong year, or a botched customer match all break the identity loudly
here instead of quietly six months from now.
Reported, never fatal. Legacy publishes on its own schedule, so a handful of
customers legitimately drift between the snapshot and the cut — the run that
established this reconciled 1,160 of 1,170.
"""
c.execute(
"""
SELECT sums.customerId, sums.total, bf.amount
FROM (
SELECT customerId, ROUND(SUM(amount), 2) AS total
FROM transactions
WHERE legacySourceTable = %s AND voidedAt IS NULL
GROUP BY customerId
) sums
LEFT JOIN (
SELECT t.customerId, ROUND(SUM(t.amount), 2) AS amount
FROM transactions t
JOIN type_transactions tt ON tt.id = t.typeId
WHERE tt.nameEn = 'BALANCE FORWARD' AND t.voidedAt IS NULL
AND t.transactionDate >= %s AND t.transactionDate < %s
GROUP BY t.customerId
) bf ON bf.customerId = sums.customerId
""",
(f"datos2@{year}", f"{year + 1}-01-01", f"{year + 1}-01-02"),
)
rows = c.fetchall()
matched = mismatched = 0
missing = 0
drift = Decimal(0)
for _cid, total, amount in rows:
if amount is None:
missing += 1
continue
if abs(Decimal(str(total)) - Decimal(str(amount))) < Decimal("0.02"):
matched += 1
else:
mismatched += 1
drift += abs(Decimal(str(total)) - Decimal(str(amount)))
checked = matched + mismatched
pct = (100 * matched / checked) if checked else 0
print(
f" corte {year} -> BF {year + 1}: {matched}/{checked} match ({pct:.1f}%)"
f", {mismatched} off by {drift:,.2f}, {missing} with no BF row"
)
def main():
env, sync_mode = parse_mode()
conn = connect(env)
@@ -230,9 +286,13 @@ def main():
message=s(r["conepto"]), check=s(r[check_col]) if check_col else None,
src_db="UTILITIES", src_tbl=legacy_tbl, legacy=str(int(r["_row_num"])))
def billing(name, legacy_tbl):
def billing(name, legacy_tbl, *, src="stg_utilities", skip_numids=None):
"""Load a DATOS2-shaped billing ledger.
`src` names the staging schema, so a prior-period archive
(stg_period_2025) loads through this same path: the snapshot's `datos2`
is the identical eleven-column shape, one year older.
NOPAGO is the legacy "still owed" flag. The website reads it directly —
`account.statement.php` splits the statement on `NOPAGO = 0` vs
`NOPAGO = 1` and renders the latter as the "Outstanding Bills Requiring
@@ -241,10 +301,13 @@ def main():
Only these three tables carry it (76 rows set in DATOS2 today); the
EFECTIVO/FM3 cash streams have no such column and stay 0.
"""
nonlocal skip_cust, skip_date
df = load("stg_utilities", name)
nonlocal skip_cust, skip_date, skip_recycled
df = load(src, name)
for _, r in df.iterrows():
cid = util_cust.get(norm_id(r["numid"]))
numid = norm_id(r["numid"])
if skip_numids and numid in skip_numids:
skip_recycled += 1; continue
cid = util_cust.get(numid)
if not cid:
skip_cust += 1; continue
td = dt(r["date"])
@@ -257,6 +320,53 @@ def main():
legacy=str(int(r["_row_num"])),
outstanding=1 if s(r["nopago"]) == "1" else 0)
skip_recycled = 0
recycle_report: list[tuple[int, str, str, str]] = []
def period_numid_guard(year: int) -> set[str]:
"""NUMids whose prior-period owner is not today's customer.
Prior-period rows attach by NUMid and nothing else, so a number the
office retired and reissued would file one customer's ledger under
another's name — the one error this feature must never make, because it
shows a stranger's charges to whoever holds the number now.
Reuse is real but rare: comparing each archive's DATGRAL against the
live one, 13 names moved since 2025 and 40 since 2024. Most are the same
customer re-described — a typo fixed (VIKIE -> VICKIE), a spouse added
or dropped (STEWART, ALAN R. -> STEWART, ALAN & JENNIFER). A few are
genuinely a different household (STRONKS, BOB -> SWEET, DONALD E.).
Sharing any word of three or more characters separates the two cleanly:
a rename keeps the surname, a reissue keeps nothing. Names are compared
legacy-to-legacy, archive DATGRAL against live DATGRAL, deliberately not
against `customers.name` — that column has been through the blank-name
recovery pass, and comparing to it reported 121 drifts where there are
13, every extra one a false positive that would have discarded good
history.
"""
try:
arch = load(f"stg_period_{year}", "datgral")
live = load("stg_utilities", "datgral")
except (FileNotFoundError, OSError):
return set()
def toks(v) -> set[str]:
return {w for w in "".join(ch if ch.isalnum() else " " for ch in (s(v) or "").upper()).split() if len(w) >= 3}
live_names = {norm_id(r["num_id"]): s(r["nombre"]) for _, r in live.iterrows()}
blocked: set[str] = set()
for _, r in arch.iterrows():
numid = norm_id(r["num_id"])
was, now = s(r["nombre"]), live_names.get(numid)
if not numid or not was or not now:
continue
if toks(was) & toks(now):
continue
blocked.add(numid)
recycle_report.append((year, numid, was, now))
return blocked
def iva():
nonlocal skip_cust
df = load("stg_utilities", "iva_2015")
@@ -286,6 +396,31 @@ def main():
billing("fee_anual", "FEE ANUAL")
billing("fee15", "fee15")
iva()
# --- prior periods -----------------------------------------------------
#
# Legacy kept each closed year in its own table and opened the next one with
# a Jan-1 BALANCE FORWARD carrying the closing total. The platform has one
# `transactions` table, so the period a row belongs to has to travel with
# the row: it rides in legacySourceTable as `datos2@2025`.
#
# That tag, not the date, is what a year view should filter on. The archives
# are not cleanly bounded — 2025's ledger carries ten undated rows and two
# dated into 2026 — and legacy itself never filtered by date either: its
# reader is `SELECT ... FROM \`2025\``. Keying on provenance reproduces the
# legacy period exactly and strands nothing.
#
# The tag also keeps the unique key safe. legacyId is a positional row
# ordinal, so every archive restarts it at 0 and would collide with the live
# `datos2` row-for-row if they shared a source-table name.
periods = sorted(
int(d.name.rsplit("_", 1)[1])
for d in STG.glob("stg_period_*")
if d.is_dir() and d.name.rsplit("_", 1)[1].isdigit()
)
for year in periods:
billing("datos2", f"datos2@{year}", src=f"stg_period_{year}",
skip_numids=period_numid_guard(year))
# Same record shape in the seguros DB. Labelled for consistency in the
# platform's own UI; unverifiable against the site, which only ever reads
# domain='UTILITY', so no customer-facing behaviour depends on it.
@@ -334,6 +469,7 @@ def main():
print(f" skipped (unresolved customer): {skip_cust}")
print(f" skipped (unparseable date) : {skip_date}")
print(f" skipped (EFECTIVO_BACKUP dup): {skip_dupe}")
print(f" skipped (reissued NUMid) : {skip_recycled}")
print(f" -> transactions : {count('transactions')}")
print(f" by domain : {dict(by_dom)}")
for src, n in by_src:
@@ -342,6 +478,17 @@ def main():
print(f" -> exchange_rates : {count('exchange_rates')}")
print(f" orphan transactions (bad customer FK): {orphans}")
assert orphans == 0, "transaction customer FK invariant failed"
if recycle_report:
print(f" ! reissued NUMids, prior-period rows NOT imported: {len(recycle_report)}")
for year, numid, was, now in recycle_report[:8]:
print(f" {year} NUMid {numid}: '{was}' -> '{now}'")
if len(recycle_report) > 8:
print(f" ... {len(recycle_report) - 8} more")
for year in periods:
verify_corte(c, year)
print(" validation: OK")
conn.close()
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "jorgecuadros-platform",
"version": "1.0.18",
"version": "1.0.26",
"private": true,
"workspaces": [
"apps/*",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@jorgecuadros/database",
"version": "1.0.18",
"version": "1.0.26",
"private": true,
"main": "generated/client/index.js",
"types": "generated/client/index.d.ts",
@@ -0,0 +1,12 @@
-- ANA Seguros policy OCR: the vehicle/driver tables and the printed term in
-- days that ANA's tourist book carries and GMX's property book does not.
ALTER TABLE `policy_ocr_documents`
ADD COLUMN `extractedCoveragePeriodDays` INTEGER NULL,
ADD COLUMN `extractedVehiclesJson` JSON NULL,
ADD COLUMN `extractedDriversJson` JSON NULL;
-- The parser note trail outgrew VARCHAR(191): a nine-section ANA policy runs
-- past it routinely, and the notes that got cut were the tail ones — the
-- "could not read X" warnings the reviewer most needs.
ALTER TABLE `policy_ocr_documents`
MODIFY COLUMN `matchNote` TEXT NULL;
@@ -0,0 +1,55 @@
-- The policy type the OCR parser read the product as, resolved to a
-- `policy_types` row at confirm time.
ALTER TABLE `policy_ocr_documents`
ADD COLUMN `extractedPolicyTypeName` VARCHAR(191) NULL;
-- ---------------------------------------------------------------------------
-- Repair: M_EMPR was deleted from the lookups screen and took its policies'
-- type with it.
--
-- `policies.policyTypeId` is ON DELETE SET NULL, and removePolicyType() had no
-- in-use guard, so deleting the row silently blanked the field on every policy
-- referencing it — 5 of them, all from the legacy `m_empr` table. The guard
-- against a repeat ships in the same change as this migration. What follows
-- repairs what already happened.
--
-- Idempotent on purpose: `policy_types.name` is UNIQUE so the INSERT IGNORE is
-- a no-op once the row exists, and the UPDATE is scoped to rows that are still
-- null AND came from that one legacy table, so it can never claim a policy
-- whose type was blanked for some other reason.
INSERT IGNORE INTO `policy_types` (`id`, `name`) VALUES (UUID(), 'M_EMPR');
UPDATE `policies` p
JOIN `policy_types` pt ON pt.`name` = 'M_EMPR'
SET p.`policyTypeId` = pt.`id`
WHERE p.`policyTypeId` IS NULL
AND p.`legacySourceTable` = 'm_empr';
-- INCENDIO is deliberately NOT recreated. It is the other row the migration
-- would have produced, but no policy in the book has ever carried it, so
-- adding it back would only put a dead option in the type picker.
-- ---------------------------------------------------------------------------
-- Merge the duplicate ANA carrier.
--
-- `insurance_providers` holds both "ANA" (1 policy) and "ANA SEGUROS" (738).
-- They are one carrier, and OCR is about to start assigning it automatically —
-- picking either row while both exist would keep splitting the book.
--
-- "ANA SEGUROS" is the survivor because it is where the 738 already are.
--
-- Written as joins rather than subqueries so that BOTH statements are no-ops
-- when either row is absent (a fresh database, or one where this was already
-- tidied by hand). A subquery form would resolve to NULL and blank the
-- carrier off every ANA policy.
UPDATE `policies` p
JOIN `insurance_providers` dup ON dup.`id` = p.`insuranceProviderId` AND dup.`name` = 'ANA'
JOIN `insurance_providers` keep ON keep.`name` = 'ANA SEGUROS'
SET p.`insuranceProviderId` = keep.`id`;
DELETE dup FROM `insurance_providers` dup
JOIN `insurance_providers` keep ON keep.`name` = 'ANA SEGUROS'
WHERE dup.`name` = 'ANA'
-- Belt and braces: never drop a row that still has policies hanging off
-- it, whatever the UPDATE above did or did not manage to move.
AND NOT EXISTS (SELECT 1 FROM `policies` p WHERE p.`insuranceProviderId` = dup.`id`);
@@ -0,0 +1,7 @@
-- Ranked customers whose name matches the printed insured name, for the
-- documents whose policy number found nothing and therefore need a customer
-- picked by hand. Kept in its own column rather than folded into
-- `matchCandidates`, which the review screen reads as policy-number hits —
-- a name is a suggestion and must never be able to masquerade as a match.
ALTER TABLE `policy_ocr_documents`
ADD COLUMN `customerSuggestions` JSON NULL;
@@ -0,0 +1,37 @@
-- Premium breakdown the Access capture form had and this schema did not:
-- RECARGO, IVA and PRIMA TOTAL on the policy header, the same six figures per
-- payment on the installments, and the FORMA PAGO that decides whether a
-- surcharge applies at all.
--
-- IVA and TOTAL were never columns in Access — they were unbound calculated
-- controls on the form — so there is nothing to backfill for them here and
-- every migrated row stays null until somebody edits the policy. RECARGO and
-- the per-installment figures DO exist in the legacy data; they are currently
-- stranded inside `policies.coveragesJson` (the migration swept every column
-- it did not model into that blob) and are recovered by
-- `migration/backfill_policy_premium_breakdown.py`, not by this migration.
ALTER TABLE `policy_types`
ADD COLUMN `taxRate` DECIMAL(6, 4) NULL;
ALTER TABLE `policies`
ADD COLUMN `surcharge` DECIMAL(12, 2) NULL,
ADD COLUMN `tax` DECIMAL(12, 2) NULL,
ADD COLUMN `taxRate` DECIMAL(6, 4) NULL,
ADD COLUMN `paymentFrequency` ENUM('ANNUAL', 'SEMIANNUAL', 'QUARTERLY', 'MONTHLY', 'SINGLE') NULL;
ALTER TABLE `policy_payment_installments`
ADD COLUMN `netPremium` DECIMAL(12, 2) NULL,
ADD COLUMN `surcharge` DECIMAL(12, 2) NULL,
ADD COLUMN `policyFee` DECIMAL(12, 2) NULL,
ADD COLUMN `tax` DECIMAL(12, 2) NULL,
ADD COLUMN `taxRate` DECIMAL(6, 4) NULL,
ADD COLUMN `total` DECIMAL(12, 2) NULL,
ADD COLUMN `commission` DECIMAL(12, 2) NULL;
-- Seed the rate the books actually use. The legacy IMPUESTOS and
-- IMPUESTOS_AUTOS tables each held exactly one row, both 0.0800, covering the
-- home and auto lines respectively; applying it to every existing type
-- reproduces current behaviour rather than changing it. Types created later
-- start null and fall back to the API default.
UPDATE `policy_types` SET `taxRate` = 0.0800 WHERE `taxRate` IS NULL;
@@ -0,0 +1,9 @@
-- A.N.A. prints IVA on the policy face and the parser already read it, but
-- `ParsedPolicy` had no field for it, so the figure only ever reached a review
-- note and the confirmed Policy was written with `tax` null. This gives it a
-- column, matching the premium fields beside it.
--
-- GMX stays null: its certificate carries no premium at all, so there is no
-- tax on it to read either.
ALTER TABLE `policy_ocr_documents`
ADD COLUMN `extractedTax` DECIMAL(12, 2) NULL;
+129 -33
View File
@@ -158,10 +158,31 @@ model InsuranceProvider {
@@map("insurance_providers")
}
/// How the premium is split into payments. Drives whether a surcharge
/// applies at all: the legacy books only ever charge `recargo` on a policy
/// paid in more than one exhibición, never on an annual one. Values come from
/// the Access `FORMA PAGO` column (ANNUAL / SEMESTRAL / CONTADO) plus the
/// quarterly option Jorge sells today but never recorded in Access.
enum PaymentFrequency {
ANNUAL
SEMIANNUAL
QUARTERLY
MONTHLY
/// Legacy "CONTADO" — the whole premium in one payment, no schedule.
SINGLE
}
model PolicyType {
id String @id @default(uuid())
name String @unique
shortDescription String?
/// IVA rate charged on this line of business, as a fraction (0.08 = 8%).
/// Replaces the legacy one-row IMPUESTOS / IMPUESTOS_AUTOS tables, which
/// held exactly one rate each — per line of business, editable without a
/// deploy, because the rate is a tax rule and tax rules change. Null falls
/// back to DEFAULT_TAX_RATE in the API rather than to "no tax", so a type
/// nobody has configured still computes the same 8% the books use today.
taxRate Decimal? @db.Decimal(6, 4)
policies Policy[]
@@map("policy_types")
@@ -185,10 +206,32 @@ model Policy {
policyTo DateTime?
coveragePeriodDays Int? @default(365)
netPremium Decimal? @db.Decimal(12, 2)
/// "Recargo" — the financing surcharge for paying in installments. Entered
/// by hand, never derived: it is quoted by the carrier, not computed here.
/// Only ever set when `paymentFrequency` is not ANNUAL/SINGLE, and it IS
/// part of the taxable base (verified against the Access books: policy
/// 7006785 only reconciles as (610.86 + 8.55 + 31.00) * 0.08 = 52.03).
surcharge Decimal? @db.Decimal(12, 2)
policyFee Decimal? @db.Decimal(12, 2)
brokerFee Decimal? @db.Decimal(12, 2)
commission Decimal? @db.Decimal(12, 2)
/// IVA. Access never stored this — it was an unbound calculated control on
/// the form — so every legacy row starts null and is filled going forward.
/// Stored rather than computed on read because the printed policy is the
/// record of truth and its rounding must survive a later rate change.
tax Decimal? @db.Decimal(12, 2)
/// The rate actually applied when `tax` was written, as a fraction. Kept on
/// the row so a policy issued at 8% still reads back as 8% after somebody
/// edits the PolicyType to a new rate.
taxRate Decimal? @db.Decimal(6, 4)
/// Prima total = netPremium + surcharge + policyFee + tax. Populated by the
/// capture form from now on. NOTE the legacy rows: `total` is 0 or null on
/// all but 2 of 2378 migrated policies, so list/sort code must keep using
/// netPremium as the headline (see policies.service.ts).
total Decimal? @db.Decimal(12, 2)
/// ANNUAL on all but 31 legacy rows — and the migration used to drop the
/// column entirely, so every pre-2026 policy reads null here.
paymentFrequency PaymentFrequency?
currency Currency @default(MXN)
observations String? @db.Text
notes String? @db.Text
@@ -267,6 +310,23 @@ model PolicyPaymentInstallment {
checkNumber String?
isCash Boolean @default(false)
// Per-payment premium breakdown. A policy paid in more than one exhibición
// prices EACH payment separately — its own net premium, its own surcharge,
// its own IVA — which is why the Access form printed the whole money row
// twice (P NETA / RECARGO / D POL / IVA / TOTAL / COM, once per pago) and
// why these cannot live on the policy header alone. `amount` stays the
// authoritative figure actually collected: it is what the cheque was
// written for and it drifts from `total` by a peso or two in the books
// (policy 7006785: amount 702.73 vs total 702.44), so it is deliberately
// NOT recomputed from this breakdown.
netPremium Decimal? @db.Decimal(12, 2)
surcharge Decimal? @db.Decimal(12, 2)
policyFee Decimal? @db.Decimal(12, 2)
tax Decimal? @db.Decimal(12, 2)
taxRate Decimal? @db.Decimal(6, 4)
total Decimal? @db.Decimal(12, 2)
commission Decimal? @db.Decimal(12, 2)
@@map("policy_payment_installments")
}
@@ -379,8 +439,11 @@ model PolicyDocument {
/// source PDF and optionally writes a premium Transaction.
model PolicyOcrBatch {
id String @id @default(uuid())
/// Which insurance provider portal the batch came from. "GMX" today;
/// future providers (AXA, GNP, …) extend the parser, not this table.
/// Which insurance provider portal the batch came from "GMX", "ANA", or
/// "GMX + ANA" when one upload mixed them. Set by the pipeline from what
/// the parsers actually claimed, not asked of the uploader, so it can
/// never contradict the documents. Future providers extend the parser,
/// not this table.
provider String @default("GMX")
status PolicyOcrBatchStatus @default(UPLOADED)
uploadedById String
@@ -427,36 +490,69 @@ model PolicyOcrDocument {
provider String?
// Extracted header fields, all staff-editable in review.
extractedPolicyNumber String?
extractedInsuredName String?
extractedAdditionalInsured String?
extractedAgentName String?
extractedLegalAddress String? @db.Text
extractedZip String?
extractedPolicyFrom DateTime?
extractedPolicyTo DateTime?
extractedPolicyDate DateTime?
extractedCurrency String?
extractedNetPremium Decimal? @db.Decimal(12, 2)
extractedPolicyFee Decimal? @db.Decimal(12, 2)
extractedBrokerFee Decimal? @db.Decimal(12, 2)
extractedTotal Decimal? @db.Decimal(12, 2)
extractedPolicyNumber String?
extractedInsuredName String?
extractedAdditionalInsured String?
extractedAgentName String?
extractedLegalAddress String? @db.Text
extractedZip String?
extractedPolicyFrom DateTime?
extractedPolicyTo DateTime?
extractedPolicyDate DateTime?
extractedCurrency String?
extractedNetPremium Decimal? @db.Decimal(12, 2)
extractedPolicyFee Decimal? @db.Decimal(12, 2)
extractedBrokerFee Decimal? @db.Decimal(12, 2)
/// IVA off A.N.A.'s `TAX` cell. Null on GMX, whose certificate carries no
/// premium at all. The adjacent `LOCAL TAX` is a separate levy with no
/// column of its own and is NOT summed in — it would make the figure stop
/// dividing back to a rate; the parser reports a non-zero one as a note.
extractedTax Decimal? @db.Decimal(12, 2)
extractedTotal Decimal? @db.Decimal(12, 2)
/// Per-coverage rows from the GMX "Material damages" / "Additional risk"
/// tables — preserved verbatim so a missing premium receipt still leaves
/// the coverages auditable.
extractedCoveragesJson Json?
extractedPremiumPayment String?
/// tables and ANA's numbered risk sections — preserved verbatim so a
/// missing premium receipt still leaves the coverages auditable.
extractedCoveragesJson Json?
extractedPremiumPayment String?
/// Printed term length. ANA sells 3- and 4-day tourist policies, so
/// leaving `Policy.coveragePeriodDays` at its 365 default would overstate
/// a weekend policy by a year.
extractedCoveragePeriodDays Int?
/// `ParsedVehicle[]` off ANA's ITEM/YEAR/MAKE/BODY/SERIAL/PLATES table.
/// Written to `Vehicle` rows on confirm; kept here so the review screen
/// shows what was read before anything is applied.
extractedVehiclesJson Json?
/// `ParsedDriver[]` — the insured on ANA's automobile face, the numbered
/// POLICY HOLDER list on its driver's policy. Written to `InsuredDriver`
/// rows on confirm.
extractedDriversJson Json?
/// The `PolicyType.name` the parser read the product as ("AUTO",
/// "LICENCIAS", "MULT"). A NAME, not an id — the parser never touches the
/// database, so confirm resolves it against `policy_types` and leaves
/// `Policy.policyTypeId` null if there is no such row.
extractedPolicyTypeName String?
// Match by `Policy.policyNumber` → existing Policy / Customer.
matchedPolicyId String?
matchedPolicy Policy? @relation("PolicyOcrDocumentPolicy", fields: [matchedPolicyId], references: [id])
matchedCustomerId String?
matchedCustomer Customer? @relation("PolicyOcrDocumentCustomer", fields: [matchedCustomerId], references: [id])
matchedPolicyId String?
matchedPolicy Policy? @relation("PolicyOcrDocumentPolicy", fields: [matchedPolicyId], references: [id])
matchedCustomerId String?
matchedCustomer Customer? @relation("PolicyOcrDocumentCustomer", fields: [matchedCustomerId], references: [id])
/// All policies carrying the same number, with their customer. One is
/// normal; >1 means the policy number is shared across customers and a
/// human must pick.
matchCandidates Json?
matchNote String?
matchCandidates Json?
/// `CustomerNameSuggestion[]` — customers whose name matches the printed
/// insured name, ranked. A SUGGESTION, never a match: it is deliberately
/// kept out of `matchCandidates` so the review screen cannot mistake a
/// name hint for a policy-number hit, and it never sets
/// `matchedCustomerId`. Only populated when the policy number found
/// nothing, which is exactly when staff have to pick a customer by hand.
customerSuggestions Json?
/// Text, not VARCHAR(191): this carries the parser's whole note trail, and
/// a multi-section ANA policy runs past 191 characters routinely. Silently
/// truncating it drops the tail notes, which are the ones that say what
/// could NOT be read.
matchNote String? @db.Text
reviewedById String?
reviewedBy User? @relation("PolicyOcrDocumentReviewer", fields: [reviewedById], references: [id])
@@ -990,8 +1086,8 @@ enum EmailNotificationStatus {
/// we store it always, so a customer reply quoting an old email can be traced
/// to the exact letter that was sent.
model EmailNotificationLog {
id String @id @default(uuid())
sendDate DateTime @default(now())
id String @id @default(uuid())
sendDate DateTime @default(now())
notificationType EmailNotificationType
/// Per-type discriminator, null where the type has none:
/// ACCOUNT_STATUS → 0 = yellow ("DEBAJO DEL TIPO"), 1 = red ("EN ROJO")
@@ -1009,7 +1105,7 @@ model EmailNotificationLog {
/// resolve the owner through `Property.customerId`, so this stays set on
/// job 4 too. Null only on skipped rows where the lookup itself failed.
customerId String?
customer Customer? @relation(fields: [customerId], references: [id])
customer Customer? @relation(fields: [customerId], references: [id])
customerName String
customerEmail String
/// Subject line of the email we attempted to send.
@@ -1017,16 +1113,16 @@ model EmailNotificationLog {
/// For PAYMENT_CONFIRMATION: the per-customer URL the PHP code built and
/// fetched (kept verbatim so the legacy format is reproducible). Null on
/// the other three jobs — the body is built inline.
bodyRequestUrl String? @db.Text
bodyRequestUrl String? @db.Text
/// The HTML body that was sent (or that would have been sent, for SKIPPED
/// rows). Stored verbatim so audit/customer-service can read the exact
/// letter that went out without re-running the render.
bodySnapshot String @db.Text
bodySnapshot String @db.Text
/// True when `debug` was passed — the recipient was overridden to the
/// admin address and no real customer received the mail. Kept here so a
/// "where did all these emails go" investigation finds the answer in one
/// place instead of "who ran what with what flags" archaeology.
debug Boolean @default(false)
debug Boolean @default(false)
/// SES SendEmail MessageId, when we actually got one back. Null on
/// failures, skipped rows, and dev/mock transport.
providerMessageId String?
@@ -1034,7 +1130,7 @@ model EmailNotificationLog {
/// insert so a verbose SES bounce payload can't blow the column.
providerResponse String?
status EmailNotificationStatus
error String? @db.Text
error String? @db.Text
@@index([sendDate])
@@index([notificationType, sendDate])
+405
View File
@@ -0,0 +1,405 @@
#!/usr/bin/env node
/**
* Corte (year-end cut) audit — READ ONLY. Writes nothing, voids nothing.
*
* Legacy Access ran a corte every year: it moved the year's utility movements
* into a per-year table and stamped one BALANCE FORWARD row per customer,
* dated Jan 1, carrying the closing balance. The platform inherited the ROWS
* (1,170 of them, dated 2026-01-01 — legacy's last cut before the extract) but
* not the PROCESS, and BillingService uses those rows as a per-customer floor
* (BALANCE_FLOOR_JOIN / NOT_SUPERSEDED in apps/api/src/billing/billing.service.ts).
*
* This script reports the two populations that floor does not cover:
*
* A. FLOORLESS customers — no BALANCE FORWARD row at all, so their balance
* is a raw lifetime sum. The platform only migrated the CURRENT-year
* charge ledger (datos2); the per-year charge tables live in DreamHost
* and were never staged. What survives before the cutover is therefore
* the EFECTIVO cash journal — receipts with no matching charges — so
* those sums read as the office owing money it does not owe.
*
* B. DOUBLE-BOOKED 2026 RECEIPTS — one cash receipt recorded twice, once in
* EFECTIVO with folio `N` and once in datos2 with reference `CN`. Both
* rows are after the 2026-01-01 floor, so both count. The statement hides
* them (STATEMENT_EXCLUDED_SOURCE_TABLES drops EFECTIVO); the balances
* worklist, the movement browser and the /clientes/:id card do not.
*
* The folio alone neither proves nor disproves a pair, so it is used as a
* lead and never as the verdict. Folios are reused, so `C13483` can collide
* with an unrelated receipt; folios are also mistyped, so a genuine pair can
* carry two different numbers. Detection therefore runs twice — once on the
* `CN` cross-reference, once over the C-refs that pass left orphaned, this
* time on proximity alone (same customer, within three days) — and BOTH
* passes are then judged on the money: identical amount when the two legs
* share a currency, or an implied USD->MXN rate inside the band the
* exchange_rates table actually observed that year. Anything that fails is
* reported apart and must not be counted as duplicated money.
*
* The second pass is not a refinement. Jorge Jr's own account carries
* `C13647` against EFECTIVO folio `13649` — same day, same 3,500.00 — and
* POWERS carries `C135808` against `13508`. Folio matching alone reports
* both accounts as clean.
*
* node scripts/corte-audit.mjs # summary + both sections
* node scripts/corte-audit.mjs --cutover 2026-01-01
* node scripts/corte-audit.mjs --csv-a # per-customer table, section A
* node scripts/corte-audit.mjs --csv-b # per-pair table, section B
* node scripts/corte-audit.mjs --limit 40 # rows printed per section
*
* Needs DATABASE_URL. Point it at PROD — a stale copy answers about itself.
* On a database imported before the BALANCE FORWARD type was minted those rows
* carry typeId NULL instead (see numid.service.ts:80), so the floor is matched
* in BOTH shapes here; matching only the type name reports every customer as
* floorless on such a copy.
*/
import pkg from "../packages/database/generated/client/index.js";
const { PrismaClient } = pkg;
/** Prisma hands raw DECIMAL back as Decimal|string|null; counts as BigInt. */
const d = (v) => (v == null ? 0 : Number(v));
const money = (v) => d(v).toFixed(2).padStart(13);
/**
* Raw DATE/DATETIME columns arrive as JS Date objects. String() would render
* them in the host's local zone, which turns a row stored at 2026-01-01 00:00
* UTC into "Dec 31" on a US Pacific laptop — the ledger is keyed on UTC dates
* everywhere else, so format in UTC and nowhere else.
*/
const day = (v) => (v == null ? "—" : new Date(v).toISOString().slice(0, 10));
function arg(args, name, fallback = null) {
const i = args.indexOf(name);
return i === -1 ? fallback : args[i + 1];
}
/**
* A row is a balance-forward marker in either of two shapes. Keep in step with
* EMPTY_NUMID_SQL in apps/api/src/customers/numid.service.ts.
*/
const BF_PREDICATE = `(
tt.nameEn = 'BALANCE FORWARD'
OR (t.typeId IS NULL AND MONTH(t.transactionDate) = 1 AND DAY(t.transactionDate) = 1
AND t.legacySourceTable = 'datos2')
)`;
async function main() {
const args = process.argv.slice(2);
const limit = Number(arg(args, "--limit", "25"));
const prisma = new PrismaClient();
try {
// ---- cutover -----------------------------------------------------------
// Default to the newest balance-forward date actually in the book rather
// than to the current year: the cut the data reflects is a fact, not a
// preference, and hardcoding 2026 would silently lie on any other copy.
const [bfDates] = await prisma.$queryRawUnsafe(`
SELECT MAX(t.transactionDate) AS newest, MIN(t.transactionDate) AS oldest,
COUNT(*) AS rows_, COUNT(DISTINCT t.customerId) AS custs
FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.voidedAt IS NULL AND ${BF_PREDICATE}
`);
const cutover =
arg(args, "--cutover") ??
(bfDates.newest ? new Date(bfDates.newest).toISOString().slice(0, 10) : null);
if (!cutover) {
console.log("No BALANCE FORWARD rows in this database and no --cutover given.");
return;
}
console.log(`corte audit — cutover ${cutover}`);
console.log(
` balance-forward rows: ${d(bfDates.rows_)} across ${d(bfDates.custs)} customers` +
`, dated ${day(bfDates.oldest)}..${day(bfDates.newest)}`,
);
// ---- book totals -------------------------------------------------------
const [book] = await prisma.$queryRawUnsafe(
`
WITH bfloor AS (
SELECT t.customerId, MAX(t.transactionDate) AS floorDate
FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.voidedAt IS NULL AND ${BF_PREDICATE}
GROUP BY t.customerId
)
SELECT
ROUND(SUM(CASE WHEN t.currency='MXN' THEN t.amount ELSE 0 END), 2) AS rawMxn,
ROUND(SUM(CASE WHEN t.currency='USD' THEN t.amount ELSE 0 END), 2) AS rawUsd,
ROUND(SUM(CASE WHEN t.currency='MXN' AND (b.floorDate IS NULL OR t.transactionDate >= b.floorDate)
THEN t.amount ELSE 0 END), 2) AS todayMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND (b.floorDate IS NULL OR t.transactionDate >= b.floorDate)
THEN t.amount ELSE 0 END), 2) AS todayUsd,
ROUND(SUM(CASE WHEN t.currency='MXN' AND t.transactionDate >= ?
THEN t.amount ELSE 0 END), 2) AS flooredMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND t.transactionDate >= ?
THEN t.amount ELSE 0 END), 2) AS flooredUsd
FROM transactions t
LEFT JOIN bfloor b ON b.customerId = t.customerId
WHERE t.voidedAt IS NULL AND t.outstanding = 0
`,
cutover,
cutover,
);
// ---- section A: floorless customers ------------------------------------
const floorless = await prisma.$queryRawUnsafe(
`
WITH nobf AS (
SELECT c.id, c.name
FROM customers c
WHERE EXISTS (SELECT 1 FROM transactions t WHERE t.customerId = c.id AND t.voidedAt IS NULL)
AND NOT EXISTS (
SELECT 1 FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.customerId = c.id AND t.voidedAt IS NULL AND ${BF_PREDICATE}
)
)
SELECT n.id, n.name,
COUNT(*) AS rows_,
SUM(t.transactionDate < ?) AS preRows,
SUM(t.transactionDate >= ?) AS postRows,
MIN(t.transactionDate) AS firstTx,
MAX(t.transactionDate) AS lastTx,
SUM(t.domain = 'UTILITY') AS utilRows,
SUM(t.domain = 'INSURANCE') AS insRows,
ROUND(SUM(CASE WHEN t.currency='MXN' AND t.outstanding=0 THEN t.amount ELSE 0 END), 2) AS todayMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND t.outstanding=0 THEN t.amount ELSE 0 END), 2) AS todayUsd,
ROUND(SUM(CASE WHEN t.currency='MXN' AND t.outstanding=0 AND t.transactionDate >= ?
THEN t.amount ELSE 0 END), 2) AS afterMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND t.outstanding=0 AND t.transactionDate >= ?
THEN t.amount ELSE 0 END), 2) AS afterUsd
FROM nobf n
JOIN transactions t ON t.customerId = n.id AND t.voidedAt IS NULL
GROUP BY n.id, n.name
ORDER BY ABS(SUM(CASE WHEN t.currency='MXN' AND t.transactionDate < ? THEN t.amount ELSE 0 END)) DESC
`,
cutover, cutover, cutover, cutover, cutover,
);
// Direction of the pre-cutover history, which is the whole argument for
// flooring rather than carrying it: a corte carries a NET, and a net built
// from receipts whose charges were never migrated is not one.
const [split] = await prisma.$queryRawUnsafe(
`
WITH nobf AS (
SELECT c.id FROM customers c
WHERE EXISTS (SELECT 1 FROM transactions t WHERE t.customerId = c.id AND t.voidedAt IS NULL)
AND NOT EXISTS (
SELECT 1 FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.customerId = c.id AND t.voidedAt IS NULL AND ${BF_PREDICATE}
)
)
SELECT SUM(t.amount < 0) AS charges, ROUND(SUM(CASE WHEN t.amount < 0 THEN t.amount ELSE 0 END), 2) AS chargeMxn,
SUM(t.amount > 0) AS credits, ROUND(SUM(CASE WHEN t.amount > 0 THEN t.amount ELSE 0 END), 2) AS creditMxn
FROM transactions t JOIN nobf n ON n.id = t.customerId
WHERE t.voidedAt IS NULL AND t.transactionDate < ?
`,
cutover,
);
// ---- section B: double-booked receipts ---------------------------------
const pairs = await prisma.$queryRawUnsafe(
`
SELECT d.id AS datos2Id, e.id AS efectivoId, c.name,
DATE(d.transactionDate) AS datos2Date, DATE(e.transactionDate) AS efectivoDate,
d.amount AS datos2Amount, d.currency AS datos2Currency,
e.amount AS efectivoAmount, e.currency AS efectivoCurrency,
d.reference AS datos2Ref, e.reference AS efectivoRef,
fx.lo AS rateLo, fx.hi AS rateHi
FROM transactions d
JOIN transactions e
ON e.customerId = d.customerId
AND e.legacySourceTable = 'EFECTIVO'
AND e.voidedAt IS NULL
AND e.reference = SUBSTRING(d.reference, 2)
JOIN customers c ON c.id = d.customerId
LEFT JOIN (
SELECT YEAR(effectiveDate) AS y, MIN(rate) AS lo, MAX(rate) AS hi
FROM exchange_rates GROUP BY YEAR(effectiveDate)
) fx ON fx.y = YEAR(d.transactionDate)
WHERE d.voidedAt IS NULL
AND d.legacySourceTable = 'datos2'
AND d.reference REGEXP '^C[0-9]+$'
ORDER BY d.transactionDate
`,
);
// Pass two — the leads pass one could not follow. Same shape of row, judged
// by the same money rules below, so a folio typo costs nothing.
const nearby = await prisma.$queryRawUnsafe(
`
SELECT d.id AS datos2Id, e.id AS efectivoId, c.name,
DATE(d.transactionDate) AS datos2Date, DATE(e.transactionDate) AS efectivoDate,
d.amount AS datos2Amount, d.currency AS datos2Currency,
e.amount AS efectivoAmount, e.currency AS efectivoCurrency,
d.reference AS datos2Ref, e.reference AS efectivoRef,
fx.lo AS rateLo, fx.hi AS rateHi
FROM transactions d
JOIN transactions e
ON e.customerId = d.customerId AND e.legacySourceTable = 'EFECTIVO'
AND e.voidedAt IS NULL AND e.amount > 0
AND ABS(DATEDIFF(e.transactionDate, d.transactionDate)) <= 3
JOIN customers c ON c.id = d.customerId
LEFT JOIN (
SELECT YEAR(effectiveDate) AS y, MIN(rate) AS lo, MAX(rate) AS hi
FROM exchange_rates GROUP BY YEAR(effectiveDate)
) fx ON fx.y = YEAR(d.transactionDate)
WHERE d.voidedAt IS NULL AND d.legacySourceTable = 'datos2'
AND d.reference REGEXP '^C[0-9]+$'
AND NOT EXISTS (
SELECT 1 FROM transactions x
WHERE x.customerId = d.customerId AND x.legacySourceTable = 'EFECTIVO'
AND x.voidedAt IS NULL AND x.reference = SUBSTRING(d.reference, 2)
)
ORDER BY d.transactionDate
`,
);
const [unpaired] = await prisma.$queryRawUnsafe(
`
SELECT COUNT(*) AS n
FROM transactions d
WHERE d.voidedAt IS NULL AND d.legacySourceTable = 'datos2'
AND d.reference REGEXP '^C[0-9]+$'
AND NOT EXISTS (
SELECT 1 FROM transactions e
WHERE e.customerId = d.customerId AND e.legacySourceTable = 'EFECTIVO'
AND e.voidedAt IS NULL AND e.reference = SUBSTRING(d.reference, 2)
)
`,
);
// ---- CSV escapes -------------------------------------------------------
if (args.includes("--csv-a")) return dumpCsv(floorless);
if (args.includes("--csv-b")) return dumpCsv([...pairs, ...nearby]);
// ---- report ------------------------------------------------------------
console.log("\nBOOK (voided and outstanding rows excluded)");
console.log(` raw lifetime sum, no floor ${money(book.rawMxn)} MXN ${money(book.rawUsd)} USD`);
console.log(` today (per-customer BF floor) ${money(book.todayMxn)} MXN ${money(book.todayUsd)} USD`);
console.log(` flat floor at ${cutover} ${money(book.flooredMxn)} MXN ${money(book.flooredUsd)} USD`);
const preRowsTotal = floorless.reduce((s, r) => s + d(r.preRows), 0);
const wouldZero = floorless.filter((r) => d(r.postRows) === 0);
const deltaMxn = floorless.reduce((s, r) => s + (d(r.todayMxn) - d(r.afterMxn)), 0);
const deltaUsd = floorless.reduce((s, r) => s + (d(r.todayUsd) - d(r.afterUsd)), 0);
console.log(`\nA. FLOORLESS CUSTOMERS — ${floorless.length}`);
console.log(` pre-cutover rows they still count: ${preRowsTotal}`);
console.log(` of those rows: ${d(split.charges)} charges (${d(split.chargeMxn).toFixed(2)})` +
` vs ${d(split.credits)} credits (${d(split.creditMxn).toFixed(2)})`);
console.log(` balance moved by flooring: ${money(-deltaMxn)} MXN ${money(-deltaUsd)} USD`);
console.log(` customers left with NO rows at all after the cut: ${wouldZero.length}` +
` (their balance becomes 0 — an assertion, not a migrated figure)`);
console.log(
`\n ${"customer".padEnd(30)} ${"pre".padStart(4)} ${"post".padStart(4)}` +
` ${"today MXN".padStart(13)} ${"after MXN".padStart(13)} ${"first tx".padStart(10)}`,
);
for (const r of floorless.slice(0, limit)) {
console.log(
` ${(r.name || "(sin nombre)").slice(0, 30).padEnd(30)}` +
` ${String(d(r.preRows)).padStart(4)} ${String(d(r.postRows)).padStart(4)}` +
` ${money(r.todayMxn)} ${money(r.afterMxn)} ${day(r.firstTx).padStart(10)}`,
);
}
if (floorless.length > limit) console.log(` ... ${floorless.length - limit} more (--csv-a)`);
// A folio match is a hypothesis; the money is the evidence. The band is
// widened by 10% either side of what exchange_rates observed that year,
// because the office keys receipts at its own counter rate, not at a
// published one, and a pair should not be called false over a few centavos.
const classify = (p) => {
const dAmt = d(p.datos2Amount);
const eAmt = d(p.efectivoAmount);
if (p.datos2Currency === p.efectivoCurrency) {
return Math.abs(dAmt - eAmt) < 0.005 ? "confirmed" : "suspect";
}
if (p.efectivoCurrency !== "USD" || p.datos2Currency !== "MXN") return "suspect";
if (!eAmt || !p.rateLo) return "suspect";
const implied = dAmt / eAmt;
return implied >= d(p.rateLo) * 0.9 && implied <= d(p.rateHi) * 1.1
? "confirmed"
: "suspect";
};
for (const p of pairs) {
p.pass = "folio";
p.verdict = classify(p);
}
for (const p of nearby) {
p.pass = "proximity";
p.verdict = classify(p);
}
pairs.push(...nearby);
const confirmed = pairs.filter((p) => p.verdict === "confirmed");
const suspect = pairs.filter((p) => p.verdict === "suspect");
const byCust = new Set(confirmed.map((p) => p.name));
const sameCur = confirmed.filter((p) => p.datos2Currency === p.efectivoCurrency);
const converted = confirmed.filter((p) => p.efectivoCurrency === "USD" && p.datos2Currency === "MXN");
const efecMxn = confirmed.reduce((s, p) => s + (p.efectivoCurrency === "MXN" ? d(p.efectivoAmount) : 0), 0);
const efecUsd = confirmed.reduce((s, p) => s + (p.efectivoCurrency === "USD" ? d(p.efectivoAmount) : 0), 0);
console.log(`\nB. DOUBLE-BOOKED RECEIPTS — ${confirmed.length} confirmed pairs across ${byCust.size} customers`);
const byFolio = confirmed.filter((p) => p.pass === "folio").length;
console.log(` candidates examined: ${pairs.length} (confirmed ${confirmed.length}, rejected on money ${suspect.length})`);
console.log(` found by folio cross-reference: ${byFolio}, by proximity after a folio miss: ${confirmed.length - byFolio}`);
console.log(` confirmed same-currency, amount equal to the cent: ${sameCur.length}`);
console.log(` confirmed USD receipt posted to datos2 in MXN: ${converted.length}`);
console.log(` datos2 C-refs with no EFECTIVO partner at all: ${d(unpaired.n)}`);
console.log(` EFECTIVO side of the confirmed pairs: ${money(efecMxn)} MXN ${money(efecUsd)} USD`);
console.log(
`\n ${"customer".padEnd(28)} ${"datos2".padStart(10)} ${"efectivo".padStart(10)}` +
` ${"datos2 amt".padStart(13)} ${"efectivo amt".padStart(13)} ref`,
);
for (const p of confirmed.slice(0, limit)) {
console.log(
` ${(p.name || "(sin nombre)").slice(0, 28).padEnd(28)}` +
` ${day(p.datos2Date).padStart(10)} ${day(p.efectivoDate).padStart(10)}` +
` ${money(p.datos2Amount)} ${p.datos2Currency}` +
` ${money(p.efectivoAmount)} ${p.efectivoCurrency} ${p.datos2Ref}/${p.efectivoRef}`,
);
}
if (confirmed.length > limit) console.log(` ... ${confirmed.length - limit} more (--csv-b)`);
if (suspect.length) {
console.log(`\n REJECTED — folio matched, money did not. Not duplicates on this evidence:`);
for (const p of suspect.slice(0, limit)) {
const implied =
d(p.efectivoAmount) && p.datos2Currency !== p.efectivoCurrency
? ` implied ${(d(p.datos2Amount) / d(p.efectivoAmount)).toFixed(2)}`
: "";
console.log(
` ${(p.name || "(sin nombre)").slice(0, 28).padEnd(28)}` +
` ${day(p.datos2Date).padStart(10)} ${day(p.efectivoDate).padStart(10)}` +
` ${money(p.datos2Amount)} ${p.datos2Currency}` +
` ${money(p.efectivoAmount)} ${p.efectivoCurrency} ${p.datos2Ref}${implied}`,
);
}
if (suspect.length > limit) console.log(` ... ${suspect.length - limit} more (--csv-b)`);
}
console.log(
"\nNOTE: nothing above has been changed. Section A is a proposal to move the\n" +
"floor, not a carried-forward balance: the pre-cutover charge ledger was\n" +
"never migrated, so no true opening balance can be computed from this\n" +
"database. It exists in the DreamHost per-year tables. Section B is an\n" +
"independent defect and does not need a corte to fix.",
);
} finally {
await prisma.$disconnect();
}
}
function dumpCsv(rows) {
if (!rows.length) return;
const cols = Object.keys(rows[0]);
console.log(cols.join(","));
for (const r of rows) {
console.log(cols.map((c) => JSON.stringify(r[c] == null ? "" : String(r[c]))).join(","));
}
}
main().catch((err) => {
console.error(err);
process.exit(1);
});