Compare commits

...
18 Commits
Author SHA1 Message Date
rmancinasandClaude Opus 5 36158ae761 feat(notificaciones): sweep one aseguradora at a time, and stop the robot quoting a premium
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m59s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m27s
The office works GMX and ANA as separate batches, so the manual barrido now
takes a "Compañía" selection. It filters the pending list as well as the
sweep, so what is on screen is exactly what "Ejecutar barrido" will mail, and
the confirmation names the carrier — running GMX when ANA was meant is the
mistake the filter exists to prevent, and it is not reversible once the mail
is out.

The carrier is chosen by InsuranceProvider id, from the same /lookups the
policy form reads, so a renamed or newly added aseguradora needs no code
change here.

A carrier-scoped run deliberately does NOT advance `lastSuccessfulAt`. The
sweep's catch-up window is computed from it, so advancing after a run that
looked at every day but mailed only one carrier would push every OTHER
carrier's letters out of tomorrow's window and they would never be sent.
Same reasoning that already keeps a debug run from advancing it.

Separately, the letter's "Prima" row is now dropped from the unattended
scheduled sweep only. A premium can still be re-rated at renewal, and an
amount a robot mailed out is one the office has to walk back; every send a
person triggers — the manual barrido and the per-row "Enviar aviso" — still
quotes it. `recordLog` renders with the same flag, so `bodySnapshot` cannot
show the office a letter the customer never received.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:24:28 -07:00
rmancinasandClaude Opus 5 fa38ff581e fix(deploy): reclaim superseded images so the host stops filling up
Every build pushes a new api + web image and every deploy pulls both onto
galactus, but nothing ever removed the pair they replaced. That reached 63
images / 83.85GB, of which 79.26GB was unused, and filled the 98GB root
filesystem to 100%.

The symptom was not a disk alert. It was "re-import is broken": the
Operaciones REIMPORT job leads with a mysqldump safety backup, that write
had nowhere to go, and PIPEFAIL took the job down before it touched the
database. Nothing in the ops_jobs log pointed at the disk.

Prune runs last, after the verify step, because Docker refuses to prune an
image that a container references — the running stack is what protects the
release just shipped. `until` adds a grace window on top so a rollback
dispatch stays a stack swap instead of a re-pull, but note it filters on
image creation time rather than pull time, so it does NOT cover rolling
back to an old tag; the running-container rule is what does.

continue-on-error: housekeeping that fails leaves a fat host, not a broken
release.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 20:32:52 -07:00
rmancinasandClaude Opus 5 c4213aa697 fix(audit): the corte audit computed a book the app no longer shows
The script carried its own copy of the balance rules, and since 1.0.26 that
copy was stale: it reported 631,078.98 MXN where the application reports
-416,403.42. An audit that disagrees with production is worse than no audit,
because its numbers look authoritative and diff cleanly against yesterday's
run. It now mirrors NOT_CASH_JOURNAL and archiveIsHistorySql, keeps the
floor-only figure as a labelled line so older runs still diff, and says
plainly which line is the app's.

Section B stops being an open defect list. EFECTIVO is a receipt book whose
receipts are posted to the datos2 ledger by design, so those rows exist and
always will; what matters is whether any of them still reaches a balance.
That is now a counted assertion which must stay at 0, and it fails loudly if
someone writes a balance query that forgets the exclusion.

Section A grew the line that changes its recommendation. "Floor them, never
carry" was written when the floorless group looked like utilities receipts
whose charges were never migrated. Measured on production today: of the 99,
ninety-five carry insurance-line cash and NONE carry utilities rows. The
seguros EFECTIVO is that line's only ledger, so flooring them deletes
receipts rather than removing a double count.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 20:05:47 -07:00
gitea-actions f7507f2370 chore(release): v1.0.26
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m50s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m26s
Deploy on tag / Deploy to galactus (push) Successful in 38s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.26.
2026-08-20 02:44:12 +00:00
rmancinasandClaude Opus 5 2f9a9afc0d fix(billing): the cash receipt book is not a second ledger
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m49s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m28s
EFECTIVO is a journal, not a ledger. The office writes a numbered paper
receipt for money handed over the counter and then posts that same receipt
to the utilities ledger as reference `C<folio>`. Legacy summed the ledger
alone — ledger_repository.php reads `datosfreak`, materialized from DATOS2
only — but the migration flattened both tables into one `transactions`
table, so every balance counted each counter payment twice.

Confirmed against the live legacy database rather than inferred: of the 297
receipts written in 2026, 296 carry a matching DATOS2 posting. Six of them
post converted to pesos under a mistyped folio, which is why matching pairs
on folio and amount found fewer duplicates than exist — and why this
excludes the whole journal instead of a list of confirmed pairs. Only folio
13536 (CL 717, $400 USD) has no posting anywhere; that one wants a human.

The database qualifier is load-bearing. `SEGUROS 16_be` keeps its own table
also called EFECTIVO, and that one is the insurance line's only ledger —
nothing posts it anywhere else. Excluding by table name alone would erase
55,444.95 USD and 63,957.78 MXN across 102 customers, 99 of whom have no
other rows at all. Extending the qualified rule to the statement and the
customer file also gives those 99 back a statement that is not empty.

The same queries were missing the archive window the statement already had,
so the worklist and the book also counted a closed year twice for customers
floored inside an archive.

Measured on production, utilities MXN: NUMid 6 and 173 unchanged to the
cent, 501 unchanged at -10,874.33 (still the portal's number), 10 drops
2,362.20 -> -1,137.80 (exactly the 3,500.00 duplicate), 295 drops
14,377.46 -> 4,377.46 — which is what his statement already said. The
worklist and the statement now agree, which is the point.

Left alone deliberately: the movement browser, which is inventory rather
than balance and should still show what was captured; and stats()'s
outstanding rows, which turn on a client decision that is still open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 18:36:49 -07:00
gitea-actions 7e71a993d0 chore(release): v1.0.25
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m0s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m40s
Deploy on tag / Deploy to galactus (push) Successful in 1m9s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.25.
2026-08-20 00:31:07 +00:00
rmancinasandClaude Opus 5 aa5867c8ea fix(statements): an archive is history below the year start, not nothing
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m49s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m20s
c6feae9 excluded imported periods from the current period outright. That is
right for the rows an archive spills into the following January — those already
sit inside the next year's BALANCE FORWARD, which is the sum of the whole
archive, so counting them again would double-book them and file a closed year's
row as current.

It is wrong for everything below the year start. Those rows are what `opening`
exists for, and for a customer whose newest BALANCE FORWARD lives *inside* an
archive they are the only carry there is: the corte skipped NUMid 295 in 2026,
so his floor is the archive's own January 1 and excluding it dropped his entire
2025 closing balance. His statement read 3,592.00 against a true 4,377.46.

So the rule is a window, not an exclusion: an archive row counts below the year
start and never at or above it. Applied identically to statement(), the
edo-cuenta-datos report and the customer-file card, which have to agree.

Spelled as a positive OR rather than NOT(tag AND date). `NOT (col LIKE '...'
AND ...)` is NULL for a row with no legacySourceTable, so the negated form would
have silently dropped every app-captured movement — the same NULL trap the
source-table exclusion was already fixed for.

Verified against prod, which now carries datos2@2025: exactly one customer is
affected and the book moves by his 785.46. NUMid 501 is unchanged at -10,874.33
and still matches the portal; 6 and 173 are unchanged to the cent; the two
customers whose archives spill into January 2026 still keep those rows out of
the current period.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 17:29:08 -07:00
gitea-actions 5549a1e0cf chore(release): v1.0.24
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m52s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m4s
Deploy on tag / Deploy to galactus (push) Successful in 22s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.24.
2026-08-19 23:58:27 +00:00
rmancinasandClaude Opus 5 c6feae9522 fix(customers): the customer file's balances are the statement's balances
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m41s
Build and Push Images / Build jorgecuadros-api (push) Successful in 3m7s
The /clientes/:id ledger card is titled "Estado de cuenta" and links straight to
the statement, but its per-line tiles came from a groupBy with `voidedAt: null`
and nothing else — no balance floor, no source exclusion, no outstanding rule.
It was a raw lifetime sum, double-counting the pre-cutover history that each
BALANCE FORWARD row already absorbs, and the card said so in its own footnote
rather than being fixed.

Importing prior periods turned that from wrong into badly wrong. Every closed
year is now also held as its own tagged copy, so an unfloored sum adds each one
a second time on top of the opening balance that contains it. NUMid 501 read
-7,119.29 before the archives landed and -15,270.59 after, against a true
-10,715.29 — the gap being exactly the 2024 and 2025 closing balances.

The tiles now take the same three rules the statement takes, and agree with it
for all 600 customers sampled.

Two places needed the archives excluded explicitly, because the balance floor
does not do it:

  - A customer whose newest BALANCE FORWARD lives inside an archive floors at
    that archive's own January 1, so every row of it clears the floor. One
    customer, 28 rows.
  - The archives are not cleanly bounded. datos2@2024 carries rows dated 2022,
    2023, 2025 and one in 2026; datos2@2025 two more. Those clear any floor and
    land in the current year next to the live ledger's own copy of them.

That second point is a defect in 93f8171, not only in this card: statement()
and the edo-cuenta-datos report were both filtering the current period by
source and date without excluding the period tags, so four customers on the
dev book would have read a closed year's rows as current. Both are fixed here.
The portal already had it right.

The year's movement list on the card keeps showing every non-archive row, as
before — it is a list of what happened, not a balance.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 16:19:06 -07:00
rmancinasandClaude Opus 5 93f817158e feat(statements): a year selector, reading each closed year from its archive
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m49s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m4s
The statement has been pinned to the calendar year in progress since bc74905.
Now that prior years are imported, the year becomes a choice: the current one
still reads the live ledger, and any earlier one reads that year's archive.

A period is selected by its `datos2@YYYY` tag, not by a date range. That is how
legacy addressed it — one table per closed year, `SELECT ... FROM `2025`` — and
the distinction is load-bearing: the archives carry rows dated a day or two into
the following January, so a date window would file them under the wrong year in
one direction and drop them in the other.

Two things the archive branch must not inherit:

  - The balance floor. It exists to stop a later opening balance double-counting
    the history it summarizes; for a period view that history is precisely what
    is being asked for, so applying it would return nothing at all.
  - The cash-source exclusion. It reproduces legacy's DATOS2-only datosfreak,
    and an archive is DATOS2 rows already.

No fold into an opening balance either — the archive holds its own Jan-1 BALANCE
FORWARD row, which is the carry, listed exactly as legacy listed it.

The current period stays deliberately open-ended at the top. A period is a table
in legacy, not a date range, so whatever the office filed in it belongs to it,
including the future-dated rows the live ledger carries out to 2028. Bounding it
would hide them from every view.

`availableYears` reports the periods a customer actually has, so the picker never
offers a year that would render empty — "you had no activity in 2019" is a
different claim from "2019 was never imported", and only one of them is true.
The selector hides itself entirely for a customer with a single period, and a
year outside the list is a 404 rather than a silent fall back to the current one.

The same period rule lands on the printable twin (edo-cuenta-datos gains a
"Periodo (año)" parameter) and on the portal, where fetchLedgerRowsPlatform was
also filtering by date with no source exclusion at all — so period=2025 would
have returned the archive rows on top of that year's EFECTIVO receipts, counting
every prior-year payment twice. The portal's allowlist is now built per data
source and validated at the point of use: DreamHost holds the current year plus
one archive table, the platform holds however many were imported, and
fetchLedgerRowsLegacy interpolates the period as a table name, so a
platform-only year must not reach it — including on the fallback path when the
platform is unreachable.

Left alone: the /clientes/:id ledger card still shows the current year. It reads
transactionYear off the customers endpoint rather than the statement, it is a
summary that links to the full statement, and giving it its own year state would
duplicate the page it links to.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 15:48:25 -07:00
rmancinasandClaude Opus 5 29ae9fa5bc feat(migration): import prior periods so a closed year can be shown on its own
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m43s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m19s
Legacy ran a year-end corte: it summed the closing year, wrote that total back
as each customer's Jan-1 BALANCE FORWARD, and started the next year clean. The
platform inherited those opening rows but never the years behind them — only
the current year's charges were ever staged, so every prior year held receipts
and no bills. Rendering one would have shown a customer credits with nothing
owed against them, which is worse than showing nothing.

The archives are whole-database Access snapshots named for the period they
hold, so `2025.accdb` is discovered by filename, staged, and loaded through the
existing DATOS2 branch — a snapshot's `datos2` is the identical shape, one year
older. Only the ledger and DATGRAL come out of a snapshot; everything else in
it is a year-stale copy of a live table. The cash side is deliberately left
behind: EFECTIVO is a lifetime journal, so the snapshot's copy is a subset of
the live one and importing it would double-book every prior-year receipt.

Period travels with the row, in legacySourceTable as `datos2@2025`. That tag,
not the date, is what a year view should filter on: the archives are not
cleanly bounded (2025 carries ten undated rows and two dated into 2026) and
legacy never filtered by date either — its reader is `SELECT ... FROM \`2025\``.
The tag also keeps legacyId safe, since it is a positional ordinal that
restarts at 0 in every archive and would otherwise collide row-for-row.

Two guards, because attaching a prior year by NUMid is the one thing here that
can go quietly wrong:

  - Reissued numbers are skipped, not imported. Comparing each archive's
    DATGRAL against the live one, 13 names moved since 2025 and 40 since 2024;
    most are the same customer re-described, but a few are a different
    household holding a recycled number, and filing their ledger under the new
    owner would show a stranger's charges. Sharing any word of three or more
    characters separates a rename from a reissue. Names are compared
    legacy-to-legacy: `customers.name` has been through blank-name recovery,
    and comparing to it reported 121 drifts where there are 13.
  - Every period is checked against the corte identity it must satisfy —
    SUM(year N) == BALANCE FORWARD(N+1) — and the result is reported per year.
    A truncated export, a file dropped under the wrong year, or a botched
    customer match all fail loudly here. 2025 reconciles 1,159/1,167 (99.3%)
    and 2024 1,144/1,156 (99.0%); the recycle guard raised 2024 from 98.2%.

Balances are untouched: BALANCE_FLOOR_JOIN floors on the newest BALANCE FORWARD
per customer, so rows behind it are already excluded from every balance read.

Uploads go through the existing ingest endpoint, allowlisted by an anchored
`AAAA.accdb` pattern that also keeps a caller-supplied name inside the ingest
directory. The Operaciones page grows an entry point for an archive that has no
row yet, reading the period off the chosen file's own name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 15:26:58 -07:00
rmancinasandClaude Opus 5 9973488330 fix(billing): folio typos hid two double-booked receipts from the audit
Matching the datos2 `CN` reference against EFECTIVO folio `N` finds pairs
only where both were keyed correctly. Jorge Cuadros Jr's account carries
`C13647` against EFECTIVO folio `13649` — same day, same 3,500.00 MXN, one
receipt — and POWERS carries `C135808` against `13508`. Folio matching alone
calls both accounts clean, which is exactly backwards: an account used as a
validator reporting a false negative is worse than no audit.

A second pass now sweeps the C-refs the first pass left orphaned, on
proximity alone (same customer, same three-day window), and both passes are
judged by the same money rules. The folio is demoted to a lead: it can be
wrong in either direction, so it never decides anything on its own.

278 confirmed pairs, up from 276 — 989,740.00 MXN and 88,392.00 USD on the
EFECTIVO side. Of the 97 orphaned C-refs only 3 had any EFECTIVO row nearby,
so the remaining 94 are datos2-only postings rather than misses.

Rejections rise to 3. RAMIREZ, SUSANA pairs 3,320.00 MXN against 18,000.00
the next day; that is a partial application, not a duplicate, and it needs a
human rather than a rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 23:35:53 -07:00
rmancinasandClaude Opus 5 75dcbc11b8 feat(billing): audit the two populations the balance floor misses
Read-only. Reports what a corte would touch without touching it.

The platform inherited legacy's BALANCE FORWARD rows (1,170, all dated
2026-01-01) but not the yearly process that produces them, and
BillingService floors balances per customer on those rows. Two populations
fall outside that floor, and they are unrelated defects that happen to
surface as the same symptom — a customer whose balance reads as a credit
the office does not owe.

A. 102 customers have no BALANCE FORWARD row, so their balance is a raw
   lifetime sum. Only the current-year charge ledger (datos2) was migrated;
   the per-year charge tables stayed in DreamHost. What survives before the
   cutover is the EFECTIVO cash journal, and it shows: 199 of their 201
   pre-cutover rows are credits. Flooring them at 2026-01-01 moves the book
   by -82,297.78 MXN and -52,020.20 USD. 91 of the 102 are left with no rows
   at all, so their balance becomes zero — an assertion, not a figure
   recovered from anywhere, which is why this script proposes and does not
   apply.

B. 276 cash receipts booked twice in 2026 — once in EFECTIVO under folio N,
   once in datos2 as reference CN — across 120 customers, 986,240.00 MXN and
   88,252.00 USD on the EFECTIVO side. Both rows sit after the floor so both
   count. The statement already hides them via
   STATEMENT_EXCLUDED_SOURCE_TABLES; the balances worklist, the movement
   browser and the /clientes/:id card do not.

The folio alone does not establish a pair — folios are reused. Every pair is
corroborated on money too: equal amounts when both legs share a currency, or
an implied USD->MXN rate inside the band exchange_rates observed that year.
Two of 278 folio matches fail that test and are reported apart rather than
counted, both same-day partial applications that need a human.

Matches the balance-forward row in both shapes, as numid.service.ts does.
Databases imported before the type was minted carry those rows with typeId
NULL, and name-only matching reports every customer as floorless on such a
copy — including the dev database, which the API itself currently reads as
+20,653,109.15 MXN against a floored -9,194.61.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 23:18:48 -07:00
rmancinasandClaude Opus 5 9929a9a3ac feat(customers): the customer file's estado de cuenta follows the same year rule
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m48s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m5s
`/clientes/:id` carries its own "Estado de cuenta" card, fed by
CustomersService.detail rather than by BillingService.statement, so the
previous commit left it reading the old way: the last 100 movements of all
time, newest first. Two pages, one label, two orders.

It now covers the current calendar year oldest-first, like the statement and
like the sheet the office prints. The `take: 100` is gone with it — capping a
descending list hid the oldest rows, but capping an ascending one would hide
the newest, and a single year is small (365 rows for the heaviest customer in
the book).

The per-domain totals above the list are untouched and still historical: they
are an unfloored groupBy, so they double-count every customer's opening
balance. That is a separate bug from this one; the card now says out loud that
those figures are lifetime, not this year's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 20:21:51 -07:00
gitea-actions 8b8de0fdca chore(release): v1.0.23
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m0s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m19s
Deploy on tag / Deploy to galactus (push) Successful in 29s
Cut by rmancinas via the "Cut release" workflow. Pushing the tag triggers build.yml; deploy separately with tag=1.0.23.
2026-08-19 03:10:16 +00:00
rmancinasandClaude Opus 5 bc749055e7 feat(statements): scope the estado de cuenta to the current year, oldest-first
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m11s
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m50s
The office's EDO CUENTA sheet has always been a *year* statement: a balance
forward line dated January 1st, then that year's movements in the order they
happened. Both of ours read the other way — every year the customer ever had,
newest first — so staff comparing the screen against the printed sheet were
reading two different documents.

Movements are now bounded to the calendar year and returned ascending, on the
screen (/estado-cuenta/[id]) and in the printable `edo-cuenta-datos` report
alike.

Earlier rows are dropped from the *list*, not from the arithmetic. The balance
floor normally lands on January 1st already, so for most customers nothing
extra is dropped at all; when it doesn't — a customer the last legacy publish
skipped, or one that never had an opening balance — the earlier rows are
folded into a carried balance and shown as a single "saldo anterior" line.
Discarding them instead would restart every balance at zero on January 1st and
nothing would throw; the numbers would just be wrong, which is how the
double-counting bug survived for years. `opening` is exposed per currency and
per business line so the totals still reconcile against the last running
balance printed.

Two things the report was missing on its own are fixed while it is being
touched, since it must agree with the screen to the peso:

  - it never applied the balance floor, so every pre-cutover row was counted
    twice — once inside the opening balance and once as itself;
  - its source-table exclusion used a bare `notIn`, and `NULL NOT IN (...)` is
    NULL rather than true, so every app-captured row (which has no
    legacySourceTable) silently vanished from the printout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 08:45:15 -07:00
rmancinasandClaude Opus 5 75e9f582b4 feat(policy-ocr): store the IVA A.N.A. already prints
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m16s
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m50s
The parser has been reading A.N.A.'s `TAX` cell since the ANA layout landed,
but `ParsedPolicy` had nowhere to put it, so the figure only ever reached a
review note and every OCR-confirmed policy was written with `tax` null — even
though the paper states it.

`TAX` now flows parser -> `extractedTax` -> `Policy.tax`, alongside the premium
fields beside it. GMX stays null: its certificate carries no premium at all, so
there is no tax on it either.

The other two cells stay out, for reasons worth keeping:

  - `LOCAL TAX` is a separate levy with no destination column, and summing it
    into `tax` would produce an IVA that no longer divides back to a rate —
    which is the whole reason to store the figure. It reads 0.00 on every
    policy seen so far; a non-zero one now raises a note saying the total will
    not reconcile, instead of quietly inflating the IVA.
  - `DISCOUNT` has no column and prints as a bare "-" when unused, which is
    what makes the row positional rather than "find six amounts".

`taxRate` is left null by confirm. A.N.A. prints the amount, not the rate, and
back-dividing it would mint a rate the document never stated; the capture form
resolves one from the line of business instead.

The review screen gains derecho de póliza next to the new IVA field. It was
already parsed and already written on confirm, but never shown — and an IVA
with no fee beside it leaves the reviewer unable to see why premium + fee + tax
equals the printed total.

The spec's assertion moved off the note and onto the field, plus a check that
the row reconciles: 298.61 + 30.00 at 8% is 26.29, totalling 354.90. That
agreement is what proves the positional mapping landed on the right cells
rather than merely on six numbers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 07:38:23 -07:00
rmancinasandClaude Opus 5 48e01ddd21 feat(policies): capture the full premium breakdown
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m19s
Build and Push Images / Build jorgecuadros-web (push) Successful in 2m1s
The capture form only ever had prima neta, derecho de póliza and comisión.
The Access form it replaces has seven figures, and the four that were missing
are the ones that make a policy paid in installments add up.

Adds recargo, IVA, prima total and forma de pago to the policy header, the
same breakdown per installment, and a per-line-of-business IVA rate.

IVA and prima total are the only derived figures:

    base  = prima neta + recargo + derecho de póliza
    IVA   = round(base * tasa)
    total = base + IVA

The recargo is inside the taxable base. That is not a guess — policy 7006785
prints IVA 52.03 on 610.86 + 8.55 + 31.00, and leaving the recargo out gives
51.35, which matches nothing on the page. Both of its money rows are asserted
in premium.spec.ts. The recargo itself is never derived: the carrier quotes it,
so staff key it in, and the field is disabled on ANNUAL/SINGLE. Both derived
figures are stored rather than recomputed on read, and stay editable, because
the printed policy is the record of truth and a later rate change must not
silently restate what was issued.

The rate lives on PolicyType (seeded to 0.08, editable in Catálogos), which is
the legacy one-row IMPUESTOS / IMPUESTOS_AUTOS tables made configurable. The
rate applied is stamped on the policy so an old one reads back at its original
rate.

Per-installment, not two fixed slots on the header: a policy split into several
exhibiciones prices each payment separately — that is why the Access form drew
the money row twice — and a trimestral policy needs four, which the Access
layout could not hold.

Also fixes two losses in the ETL, which is how these went missing:

  - `forma_pago` was marked consumed by the coverage sweep and then never
    written to any column, so FORMA PAGO existed nowhere in the platform.
  - `recargo` and the whole second money row fell into `coveragesJson` as
    loose strings, mislabeled as coverage amounts.

transform_policies.py now writes all of it directly;
backfill_policy_premium_breakdown.py recovers it on a database that must not be
re-imported, and strips the migrated keys back out of coveragesJson. Both are
COALESCE-only, so a figure a human has corrected in the app wins.

IVA and TOTAL are NOT backfilled: they were unbound calculated controls on the
Access form, never columns, so there is nothing to recover and every migrated
policy reads null until it is edited.

The backfill warns on 5 annual policies that carry a non-zero recargo — a
contradiction that predates this change and is left for a human, not silently
corrected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 00:24:22 -07:00
58 changed files with 3575 additions and 236 deletions
+25
View File
@@ -25,6 +25,10 @@
# through this workflow at all.
# 4. app (api + web) the new images.
# 5. verify ask the running API what it actually is.
# 6. prune images reclaim the superseded api/web images. LAST, and
# after verify: Docker will not prune an image a
# container references, so the running stack is what
# protects the release we just shipped.
#
# Rollback = re-dispatch with an older `tag`. That rolls back CODE only; the
# schema stays forward. This is exactly why every schema change must be
@@ -386,3 +390,24 @@ jobs:
echo "dispatched '$WANT'; tiers report '$API_VER' (not directly comparable)"
;;
esac
# --- housekeeping ------------------------------------------------------
# Runs LAST, and only after the verify step proved the new containers are
# up. See deploy/scripts/prune-images.mjs: Docker refuses to prune an
# image a container references, so "the stack is running" is what makes
# the current images safe. Pruning earlier would have nothing holding
# them.
#
# continue-on-error: reclaiming disk is not what the deploy is for. A
# prune that fails leaves a fat host, not a broken release.
- name: Prune unused images
continue-on-error: true
env:
PORTAINER_URL: ${{ secrets.PORTAINER_URL_GALACTUS }}
PORTAINER_API_KEY: ${{ secrets.PORTAINER_API_KEY_GALACTUS }}
PORTAINER_ENDPOINT_ID: ${{ secrets.PORTAINER_ENDPOINT_ID_GALACTUS }}
# Grace window. Keeps the previous few releases on disk so a rollback
# dispatch is a stack swap instead of a re-pull.
KEEP_HOURS: "168"
NODE_TLS_REJECT_UNAUTHORIZED: "0"
run: node deploy/scripts/prune-images.mjs
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@jorgecuadros/api",
"version": "1.0.22",
"version": "1.0.26",
"private": true,
"scripts": {
"build": "nest build",
+44 -10
View File
@@ -115,13 +115,26 @@ describe("balance floor", () => {
);
});
/**
* statement() issues two findMany calls: first the period discovery (which
* years this customer has an archive for), then the statement rows. Select
* the rows query by its shape so adding another lookup later moves nothing
* here — the previous version indexed call 0 and broke the moment period
* support landed.
*/
function rowsQuery(findMany: jest.Mock) {
const call = findMany.mock.calls.find((c) => c[0]?.orderBy);
if (!call) throw new Error("statement() issued no ordered rows query");
return call[0];
}
it("bounds the statement at the floor, inclusive", async () => {
const floor = new Date("2026-01-01T00:00:00Z");
const { service, findMany } = serviceWith(floor);
await service.statement("c1");
expect(findMany.mock.calls[0][0].where).toMatchObject({
expect(rowsQuery(findMany).where).toMatchObject({
customerId: "c1",
transactionDate: { gte: floor },
});
@@ -132,23 +145,44 @@ describe("balance floor", () => {
await service.statement("c1");
expect(findMany.mock.calls[0][0].where).not.toHaveProperty(
"transactionDate",
);
expect(rowsQuery(findMany).where).not.toHaveProperty("transactionDate");
});
it("keeps the source-table exclusion alongside the floor", async () => {
// The two guards answer different questions — one reproduces legacy's
// DATOS2-only materialization, the other drops superseded history — and
// dropping either one changes the customer's balance.
// The three guards answer different questions — one windows imported
// periods, one drops the cash receipt book the ledger already posts, the
// floor drops superseded history — and dropping any one of them changes
// the customer's balance.
const { service, findMany } = serviceWith(new Date("2026-01-01T00:00:00Z"));
await service.statement("c1");
const where = findMany.mock.calls[0][0].where;
expect(where.OR).toEqual([
const where = rowsQuery(findMany).where;
// Imported periods are windowed rather than excluded: history below the
// year start (the only carry a floored-by-archive customer has), never
// at or above it (those rows sit inside the next BALANCE FORWARD).
expect(where.AND).toEqual([
{
OR: [
{ legacySourceTable: null },
{ legacySourceTable: { notIn: expect.arrayContaining(["EFECTIVO"]) } },
{ legacySourceTable: { not: { startsWith: "datos2@" } } },
{ transactionDate: { lt: expect.any(Date) } },
],
},
// The cash journal, qualified by the database it came from — the
// insurance line has its own EFECTIVO and that one is a real ledger.
{
OR: [
{ legacySourceDb: null },
{ legacySourceDb: { not: "UTILITIES" } },
{ legacySourceTable: null },
{
legacySourceTable: {
notIn: expect.arrayContaining(["EFECTIVO"]),
},
},
],
},
]);
});
});
+16 -3
View File
@@ -119,10 +119,23 @@ export class BillingController {
return this.billing.byCheck(n);
}
/** One customer's full statement across both business lines. */
/**
* One customer's statement across both business lines, for one period.
*
* `year` omitted means the current one. Any earlier year is served from its
* imported archive; the response carries `availableYears` so the caller can
* offer only the periods this customer actually has.
*/
@Get("customers/:id")
statement(@Param("id") id: string) {
return this.billing.statement(id);
statement(@Param("id") id: string, @Query("year") year?: string) {
let parsed: number | undefined;
if (year !== undefined && year !== "") {
parsed = Number(year);
if (!Number.isInteger(parsed)) {
throw new BadRequestException("year debe ser un año de cuatro dígitos");
}
}
return this.billing.statement(id, parsed);
}
/** Cross-customer movement browser. */
+285 -40
View File
@@ -210,18 +210,47 @@ export const BALANCE_FLOOR_JOIN = Prisma.sql`
export const NOT_SUPERSEDED = Prisma.sql`(bfloor.floorDate IS NULL OR t.transactionDate >= bfloor.floorDate)`;
/**
* Source tables excluded from the customer-facing statement.
* `legacySourceTable` of an imported prior period.
*
* The legacy portal's `datosfreak` table was materialized from DATOS2 only
* (`objects.json:1358`), so the customer's "current balance" never saw
* EFECTIVO / EFECTIVO FM3 / CHEQUE FM3 / EFECTIVO_BACKUP cash receipts, nor
* the IVA 2015 snapshot. The unified `transactions` table has all of them, so
* the statement must drop them to match the legacy number the customer has
* been quoted for years. The staff-facing balances worklist and movement
* browser keep them — they're real money, just tracked separately
* (FM3 = visa fee stream, EFECTIVO = cash receipt stream).
* A closed year arrives as its own Access snapshot and is tagged rather than
* dated (see migration/transform_transactions.py). The tag is what a period
* view filters on: the archives are not cleanly date-bounded — 2025 carries
* rows dated into 2026 — and legacy did not filter by date either, it selected
* `FROM \`2025\``. Filtering on provenance reproduces the legacy period exactly.
*/
const STATEMENT_EXCLUDED_SOURCE_TABLES: readonly string[] = [
export const periodSourceTable = (year: number) => `datos2@${year}`;
/** Matches any imported period tag, for discovering which years a customer has. */
export const PERIOD_TABLE_PREFIX = "datos2@";
/**
* The legacy cash receipt book — a journal, not a ledger.
*
* `EFECTIVO` is the office's numbered receipt pad: money is handed over the
* counter, a folio is written, and the same receipt is then *posted* to the
* utilities ledger (`DATOS2`) as reference `C<folio>`. Legacy summed the ledger
* alone — `ws/v2/lib/ledger_repository.php` reads `datosfreak`, which is
* materialized from DATOS2 only (`objects.json:1358`). The migration flattened
* both tables into one `transactions` table, so anything summing a customer's
* rows counts every cash receipt twice.
*
* Verified against the live legacy database on 2026-08-20: of the 297 receipts
* written in 2026, 296 carry a matching DATOS2 posting. Only folio 13536 (CL
* 717, $400 USD) has no posting anywhere, and it wants a human's eyes rather
* than a code change. Six receipts post converted to pesos under a mistyped
* folio, which is why matching on folio and amount found fewer duplicate pairs
* than actually exist — a reason to exclude the whole journal rather than to
* exclude a list of confirmed pairs.
*
* THE DATABASE QUALIFIER IS LOAD-BEARING. `SEGUROS 16_be` keeps its own table
* also called `EFECTIVO`, and that one is the insurance line's *only* ledger —
* nothing posts it anywhere else. Excluding by table name alone erases the
* whole insurance balance: 55,444.95 USD and 63,957.78 MXN across 102
* customers, 99 of whom have no other rows at all.
*/
export const CASH_JOURNAL_SOURCE_DB = "UTILITIES";
export const CASH_JOURNAL_SOURCE_TABLES: readonly string[] = [
"EFECTIVO",
"EFECTIVO_BACKUP",
"EFECTIVO FM3",
@@ -229,6 +258,53 @@ const STATEMENT_EXCLUDED_SOURCE_TABLES: readonly string[] = [
"IVA 2015",
];
/**
* Prisma form of the cash-journal exclusion.
*
* Spelled as a positive OR on purpose. `notIn` alone compiles to SQL `NOT IN`,
* and `NULL NOT IN (...)` is NULL rather than true, so every app-captured row
* (no `legacySourceTable`) would silently vanish. Same for the database test.
*/
export const notCashJournal = (): Prisma.TransactionWhereInput => ({
OR: [
{ legacySourceDb: null },
{ legacySourceDb: { not: CASH_JOURNAL_SOURCE_DB } },
{ legacySourceTable: null },
{ legacySourceTable: { notIn: [...CASH_JOURNAL_SOURCE_TABLES] } },
],
});
/** Raw-SQL form, for the aggregate queries that cannot use Prisma's builder. */
export const NOT_CASH_JOURNAL = Prisma.sql`(
t.legacySourceDb IS NULL
OR t.legacySourceDb <> ${CASH_JOURNAL_SOURCE_DB}
OR t.legacySourceTable IS NULL
OR t.legacySourceTable NOT IN (${Prisma.join([
...CASH_JOURNAL_SOURCE_TABLES,
])}))`;
/**
* Keeps an imported prior period out of the *current* period, NULL-safely.
*
* The balance floor does not settle the archives on its own, in both
* directions. A customer whose newest BALANCE FORWARD lives *inside* an archive
* floors at that archive's own January 1st, so every row of it clears the floor
* — and that is correct, because below the year start the archive is the only
* carry there is. At or above the year start it must go: the archives spill a
* couple of rows into the following January and those already sit inside the
* next year's BALANCE FORWARD, which is the sum of the whole archive.
*
* Spelled as a positive OR for the same NULL reason as above.
*/
export const archiveIsHistorySql = (yearStart: Date) => Prisma.sql`(
t.legacySourceTable IS NULL
OR t.legacySourceTable NOT LIKE ${`${PERIOD_TABLE_PREFIX}%`}
OR t.transactionDate < ${yearStart})`;
/** January 1st of the running year, UTC — the current period's lower bound. */
export const currentYearStart = () =>
new Date(Date.UTC(new Date().getUTCFullYear(), 0, 1));
@Injectable()
export class BillingService {
constructor(private readonly prisma: PrismaService) {}
@@ -398,6 +474,16 @@ export class BillingService {
async balances(params: BalanceParams) {
const { query, page, pageSize, currency, balance, domain, sort } = params;
// A balance is what the customer owes, so it takes the same rules the
// statement takes: the floor, the cash journal, and the archive window.
// Without the last two the worklist quoted a different number than the
// customer's own statement — NUMid 295 read 14,377.46 against a statement
// of 4,377.46, and NUMid 10 read 2,362.20 against -1,137.80, the gap in
// each case being a cash receipt already posted to the ledger.
const scope = Prisma.sql`AND ${NOT_CASH_JOURNAL} AND ${archiveIsHistorySql(
currentYearStart(),
)}`;
const filters: Prisma.Sql[] = [];
if (domain) filters.push(Prisma.sql`t.domain = ${domain}`);
const txFilter = filters.length
@@ -460,7 +546,7 @@ export class BillingService {
FROM customers c
JOIN transactions t ON t.customerId = c.id
${BALANCE_FLOOR_JOIN}
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND ${NOT_SUPERSEDED} ${nameFilter} ${txFilter}
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND ${NOT_SUPERSEDED} ${scope} ${nameFilter} ${txFilter}
GROUP BY c.id, c.name, c.nameSource, c.nameMissing, c.city, c.state
${having}
${orderBy}
@@ -476,7 +562,7 @@ export class BillingService {
-- Must match the page query's filters exactly, or the total disagrees
-- with the rows. (The void exclusion was missing here before the
-- outstanding work; a voided-only customer inflated the count.)
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND ${NOT_SUPERSEDED} ${nameFilter} ${txFilter}
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND ${NOT_SUPERSEDED} ${scope} ${nameFilter} ${txFilter}
GROUP BY c.id
${having}
) x
@@ -523,11 +609,22 @@ export class BillingService {
* Two different questions live here and they use different row sets.
* `movements`, `ledgerCustomers`, `crossLineCustomers` and the date range are
* INVENTORY — what is stored — and count everything not voided. Everything
* under `byCurrency` / `byDomain` is a BALANCE, so it applies NOT_SUPERSEDED
* and drops rows an opening balance already accounts for. The four aggregates
* moved from Prisma groupBy to raw SQL to express that join; groupBy cannot.
* under `byCurrency` / `byDomain` is a BALANCE, so it takes the same scope
* `balances()` takes — the opening-balance floor, the cash journal and the
* archive window — and the book has to agree with the worklist that sits
* under it. The four aggregates moved from Prisma groupBy to raw SQL to
* express that join; groupBy cannot.
*
* KNOWN DIVERGENCE, left deliberately: these three do not drop outstanding
* rows, while `balances()` does. Reconciling them moves the book by about
* 1.95M MXN and turns on whether an unfunded charge is owed by the customer,
* which is the client's call and not settled yet.
*/
async stats() {
const scope = Prisma.sql`AND ${NOT_CASH_JOURNAL} AND ${archiveIsHistorySql(
currentYearStart(),
)}`;
const [movements, ledgerCustomers] = await Promise.all([
this.prisma.transaction.count({ where: NOT_VOIDED }),
this.prisma.transaction
@@ -559,7 +656,7 @@ export class BillingService {
SUM(t.amount > 0) AS creditCount
FROM transactions t
${BALANCE_FLOOR_JOIN}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED} ${scope}
GROUP BY t.currency
`;
@@ -575,7 +672,7 @@ export class BillingService {
SUM(t.amount) AS net, COUNT(*) AS count
FROM transactions t
${BALANCE_FLOOR_JOIN}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED} ${scope}
GROUP BY t.domain, t.currency
`;
@@ -596,7 +693,7 @@ export class BillingService {
SELECT t.customerId, t.currency, SUM(t.amount) AS bal
FROM transactions t
${BALANCE_FLOOR_JOIN}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED}
WHERE t.voidedAt IS NULL AND ${NOT_SUPERSEDED} ${scope}
GROUP BY t.customerId, t.currency
) x
GROUP BY currency
@@ -701,13 +798,22 @@ export class BillingService {
/**
* One customer's statement across both business lines.
*
* Returns the *whole* ledger rather than a page of it: the heaviest customer
* Scoped to one calendar year and listed oldest-first, matching the legacy
* EDO CUENTA report the office has printed for years: an opening balance at
* the top, then the year's movements in the order they happened.
*
* `year` selects the period. The current year is read from the live tables;
* any earlier year is read from its imported archive, which legacy kept as a
* separate table and this reads by its `datos2@YYYY` tag. `availableYears`
* reports which periods this customer actually has, so a caller never offers
* a year that would render empty.
*
* Returns the *whole* year rather than a page of it: the heaviest customer
* carries 365 movements (mean 26), and a running balance is meaningless if
* the client only holds a slice. The running balance is accumulated per
* currency in chronological order, then the list is handed back newest-first
* with each row's balance-after already attached.
* currency in chronological order, with each row's balance-after attached.
*/
async statement(customerId: string) {
async statement(customerId: string, year?: number) {
const customer = await this.prisma.customer.findUnique({
where: { id: customerId },
select: {
@@ -731,6 +837,49 @@ export class BillingService {
throw new NotFoundException(`Customer ${customerId} not found`);
}
// Which periods this customer has. The current year is always offered —
// it is the live ledger even when empty — and each imported archive adds
// the year it holds.
const archives = await this.prisma.transaction.findMany({
where: {
customerId,
voidedAt: null,
legacySourceTable: { startsWith: PERIOD_TABLE_PREFIX },
},
distinct: ["legacySourceTable"],
select: { legacySourceTable: true },
});
const thisYear = new Date().getUTCFullYear();
const archiveYears = archives
.map((a) => Number(a.legacySourceTable?.slice(PERIOD_TABLE_PREFIX.length)))
.filter((y) => Number.isInteger(y) && y < thisYear);
const availableYears = [...new Set([thisYear, ...archiveYears])].sort(
(a, b) => b - a,
);
// An unknown year would silently render as the current one, which reads as
// "this customer had no activity in 2019" rather than "there is no 2019".
const requested = year ?? thisYear;
if (!availableYears.includes(requested)) {
throw new NotFoundException(
`El cliente no tiene movimientos del periodo ${requested}.`,
);
}
const isArchive = requested !== thisYear;
//
// Deliberately open-ended at the top. A period is a table in legacy, not a
// date range, so whatever the office filed in it belongs to it — including
// the future-dated rows the current ledger carries (it runs to 2028). An
// upper bound would hide them from every view, which is not what legacy did
// and not what the office has been reading.
//
// An archive needs no fold at all: it *is* the period, and its own Jan-1
// BALANCE FORWARD row is the carry, listed exactly as legacy listed it.
const yearStart = isArchive
? new Date(0)
: new Date(Date.UTC(requested, 0, 1));
// One customer, so the balance floor is a single date rather than the
// derived table the aggregate queries join. See NOT_SUPERSEDED: rows before
// the opening balance are already inside it, and showing them would both
@@ -756,22 +905,49 @@ export class BillingService {
const rows = await this.prisma.transaction.findMany({
where: {
customerId,
...(isArchive
? // An archive is already exactly one period's ledger, so the tag is
// the whole filter. The balance floor is deliberately NOT applied:
// it exists to stop a later opening balance double-counting the
// history it summarizes, and here that history is the thing being
// asked for. The exclusion list is moot too — an archive holds only
// DATOS2 rows, which is what legacy's year table held.
{ legacySourceTable: periodSourceTable(requested) }
: {
...(floor ? { transactionDate: { gte: floor.transactionDate } } : {}),
// NULL-safe exclusion. `notIn` alone compiles to SQL `NOT IN`, and
// `NULL NOT IN (...)` is NULL, not true — so every app-captured row
// (which has no legacySourceTable) silently vanished from the
// statement while still showing in the movement browser. Rows the app
// books must appear on the customer's statement, so the null case is
// spelled out.
// An archive row belongs to this period only as history. Below
// the year start it is exactly what `opening` is for, and for the
// one customer whose newest BALANCE FORWARD lives *inside* an
// archive it is the only carry there is — dropping it outright
// understated NUMid 295 by his whole 2025 closing balance, 785.46.
//
// At or above the year start it must go. The archives spill a
// couple of rows into the following January, and those are
// already inside the next year's BALANCE FORWARD (which is the
// sum of the whole archive), so listing them here would both
// double-count and file a closed year's row as current.
//
// Spelled as a positive OR because `NOT (col LIKE ... AND ...)`
// is NULL for an app-captured row, which would drop every one.
AND: [
{
OR: [
{ legacySourceTable: null },
{
legacySourceTable: {
notIn: STATEMENT_EXCLUDED_SOURCE_TABLES as string[],
not: { startsWith: PERIOD_TABLE_PREFIX },
},
},
{ transactionDate: { lt: yearStart } },
],
},
// The cash receipt book is the ledger's own postings written a
// second time, so listing it here would show every counter
// payment twice and double the credit side.
notCashJournal(),
],
}),
},
orderBy: [{ transactionDate: "asc" }, { id: "asc" }],
select: {
id: true,
@@ -790,15 +966,50 @@ export class BillingService {
},
});
// The statement covers one calendar year. The floor above normally lands on
// January 1st of it already — the legacy publish writes one BALANCE FORWARD
// per customer per year — in which case nothing extra is dropped here. When
// it doesn't (a customer the last publish skipped, or one that never had an
// opening balance), the earlier rows still have to be *counted* or every
// balance below is wrong, so they are folded into `opening` rather than
// listed. That is the same thing a BALANCE FORWARD row does, just computed.
const running = new Map<string, Prisma.Decimal>();
const movements = rows.map((r) => {
/** Balance carried into `yearStart`, per currency. */
const opening = new Map<string, Prisma.Decimal>();
/** The same carried balance split by business line, keyed `domain|currency`. */
const openingByDomain = new Map<
string,
{ domain: TransactionDomain; currency: string; amount: Prisma.Decimal }
>();
/** The rows the statement lists — this year's. Totals are built from these. */
const visible: typeof rows = [];
const movements = rows.flatMap((r) => {
const voided = r.voidedAt != null;
const prev = running.get(r.currency) ?? new Prisma.Decimal(0);
// Neither a voided row nor an outstanding (unpaid) one moves the running
// balance — both show tagged, with the balance unchanged from the previous
// live movement. Outstanding rows start counting once resolved.
const next = voided || r.outstanding ? prev : prev.plus(r.amount);
const counted = !voided && !r.outstanding;
const next = counted ? prev.plus(r.amount) : prev;
running.set(r.currency, next);
if (r.transactionDate < yearStart) {
if (counted) {
opening.set(r.currency, next);
const dk = `${r.domain}|${r.currency}`;
const od = openingByDomain.get(dk) ?? {
domain: r.domain,
currency: r.currency,
amount: new Prisma.Decimal(0),
};
od.amount = od.amount.plus(r.amount);
openingByDomain.set(dk, od);
}
return [];
}
visible.push(r);
return {
id: r.id,
transactionDate: r.transactionDate,
@@ -818,7 +1029,6 @@ export class BillingService {
balanceAfter: next.toFixed(2),
};
});
movements.reverse();
// Per-currency summary, and the same split by business line so the two
// ledgers are visibly one statement without being illegally added up.
@@ -846,7 +1056,29 @@ export class BillingService {
}
>();
for (const r of rows) {
for (const [currency] of opening) {
perCurrency.set(currency, {
currency,
charges: new Prisma.Decimal(0),
credits: new Prisma.Decimal(0),
chargeCount: 0,
creditCount: 0,
count: 0,
first: null,
last: null,
});
}
for (const [key, o] of openingByDomain) {
perDomain.set(key, {
domain: o.domain,
currency: o.currency,
charges: new Prisma.Decimal(0),
credits: new Prisma.Decimal(0),
count: 0,
});
}
for (const r of visible) {
// Voided rows never enter a total; outstanding rows don't either until
// they're resolved (legacy SALDOS ULTIMO 0's `HAVING NOPAGO = 0`).
if (r.voidedAt != null || r.outstanding) continue;
@@ -896,7 +1128,7 @@ export class BillingService {
string,
{ name: string; currency: string; total: Prisma.Decimal; count: number }
>();
for (const r of rows) {
for (const r of visible) {
if (r.voidedAt != null || r.outstanding) continue;
if (!r.amount.lessThan(0)) continue;
const name = r.type?.nameEs || r.type?.nameEn || "Sin clasificar";
@@ -915,25 +1147,38 @@ export class BillingService {
propertyCount: customer._count.properties,
policyCount: customer._count.policies,
},
summary: [...perCurrency.values()].map((c) => ({
year: requested,
availableYears,
summary: [...perCurrency.values()].map((c) => {
const open = opening.get(c.currency) ?? new Prisma.Decimal(0);
return {
currency: c.currency,
/** Balance carried in from before this year — legacy's BALANCE FORWARD. */
opening: open.toFixed(2),
charges: c.charges.toFixed(2),
credits: c.credits.toFixed(2),
balance: c.charges.plus(c.credits).toFixed(2),
balance: open.plus(c.charges).plus(c.credits).toFixed(2),
chargeCount: c.chargeCount,
creditCount: c.creditCount,
count: c.count,
firstMovement: c.first,
lastMovement: c.last,
})),
byDomain: [...perDomain.values()].map((d) => ({
};
}),
byDomain: [...perDomain.values()].map((d) => {
const open =
openingByDomain.get(`${d.domain}|${d.currency}`)?.amount ??
new Prisma.Decimal(0);
return {
domain: d.domain,
currency: d.currency,
opening: open.toFixed(2),
charges: d.charges.toFixed(2),
credits: d.credits.toFixed(2),
balance: d.charges.plus(d.credits).toFixed(2),
balance: open.plus(d.charges).plus(d.credits).toFixed(2),
count: d.count,
})),
};
}),
byType: [...byType.values()]
.map((t) => ({
name: t.name,
+77
View File
@@ -0,0 +1,77 @@
import { Prisma } from "@jorgecuadros/database";
import {
CASH_JOURNAL_SOURCE_DB,
CASH_JOURNAL_SOURCE_TABLES,
NOT_CASH_JOURNAL,
notCashJournal,
} from "./billing.service";
/**
* `EFECTIVO` is the office's paper receipt book, and every receipt in it is
* also posted to the utilities ledger as `C<folio>`. Both copies were imported
* into one `transactions` table, so a balance that reads the journal counts
* each counter payment twice — 1,094,347.78 MXN of phantom credit book-wide,
* and 3,500.00 of it on NUMid 10 alone.
*
* These fail silently in the worst way: the numbers stay plausible, they are
* just too generous to the customer. Two shapes of mistake are easy to make
* here and both are covered below — dropping the database qualifier (which
* erases the insurance line's only ledger) and writing the exclusion as a bare
* `NOT IN` (which erases every app-captured row).
*/
describe("cash journal exclusion", () => {
describe("raw SQL form", () => {
it("binds the source database rather than interpolating it", () => {
expect(NOT_CASH_JOURNAL.values).toContain(CASH_JOURNAL_SOURCE_DB);
});
it("qualifies the table names with the database they came from", () => {
// `SEGUROS 16_be` has its own EFECTIVO and it is the insurance line's
// ONLY ledger — nothing posts it anywhere else. Matching on the table
// name alone erases 55,444.95 USD and 63,957.78 MXN across 102 customers.
expect(NOT_CASH_JOURNAL.sql).toContain("t.legacySourceDb <>");
expect(NOT_CASH_JOURNAL.values).toContain(CASH_JOURNAL_SOURCE_DB);
});
it("spells both null cases out instead of relying on NOT IN", () => {
// `NULL NOT IN (...)` is NULL, not true. Without these branches every
// app-captured row — the ones staff key in by hand — drops out of the
// balance while still showing in the movement browser.
expect(NOT_CASH_JOURNAL.sql).toContain("t.legacySourceDb IS NULL");
expect(NOT_CASH_JOURNAL.sql).toContain("t.legacySourceTable IS NULL");
});
it("covers the whole cash family, not just EFECTIVO", () => {
for (const table of CASH_JOURNAL_SOURCE_TABLES) {
expect(NOT_CASH_JOURNAL.values).toContain(table);
}
});
it("is a single parenthesised term, safe to AND into a WHERE clause", () => {
// It is composed as `... AND ${NOT_CASH_JOURNAL} AND ...`. An unbracketed
// OR chain would swallow every condition after it and silently widen the
// whole query to the entire table.
const sql = NOT_CASH_JOURNAL.sql.trim();
expect(sql.startsWith("(")).toBe(true);
expect(sql.endsWith(")")).toBe(true);
});
});
describe("Prisma form", () => {
it("matches the raw form's terms so the two cannot drift apart", () => {
const branches = notCashJournal().OR as Prisma.TransactionWhereInput[];
expect(branches).toEqual([
{ legacySourceDb: null },
{ legacySourceDb: { not: CASH_JOURNAL_SOURCE_DB } },
{ legacySourceTable: null },
{ legacySourceTable: { notIn: [...CASH_JOURNAL_SOURCE_TABLES] } },
]);
});
it("returns a fresh object each call", () => {
// It is spread into `AND: [...]` arrays that Prisma may mutate; a shared
// singleton would leak one query's filters into the next.
expect(notCashJournal()).not.toBe(notCashJournal());
});
});
});
@@ -0,0 +1,36 @@
import { periodSourceTable } from "./billing.service";
/**
* A closed year is imported as its own tagged set of rows rather than being
* identified by date. The tag is written by migration/transform_transactions.py
* and read by BillingService.statement, the edo-cuenta-datos report, and the
* PHP portal — three places that must agree on the exact string.
*/
describe("periodSourceTable", () => {
it("names the archive the migration writes", () => {
expect(periodSourceTable(2025)).toBe("datos2@2025");
expect(periodSourceTable(2024)).toBe("datos2@2024");
});
it("stays distinct from the live ledger's own table", () => {
// The live table is plain `datos2`. legacyId is a positional ordinal that
// restarts at 0 in every archive, so a shared name would collide with the
// current year row-for-row on the unique key.
expect(periodSourceTable(2025)).not.toBe("datos2");
expect(periodSourceTable(2025).startsWith("datos2@")).toBe(true);
});
it("is not matched by the statement's cash-source exclusion list", () => {
// STATEMENT_EXCLUDED_SOURCE_TABLES drops the EFECTIVO family to reproduce
// legacy's DATOS2-only datosfreak. An archive holds DATOS2 rows, so it must
// survive that filter or a prior year renders empty.
const excluded = [
"EFECTIVO",
"EFECTIVO_BACKUP",
"EFECTIVO FM3",
"CHEQUE FM3",
"IVA 2015",
];
expect(excluded).not.toContain(periodSourceTable(2025));
});
});
+145
View File
@@ -0,0 +1,145 @@
import { Prisma } from "@jorgecuadros/database";
import { BillingService } from "./billing.service";
/**
* The statement is a *year* statement, like the EDO CUENTA report the office
* prints: this year's movements, oldest-first, opening on the balance carried
* in from before it.
*
* The carrying is the part worth testing. Dropping earlier rows from the list
* is easy; dropping them from the arithmetic too would restart every balance at
* zero on January 1st, and nothing would throw — the numbers would just be
* wrong, which is exactly how the double-counting bug lived for years.
*/
describe("statement year scoping", () => {
const YEAR = new Date().getUTCFullYear();
function d(iso: string) {
return new Date(`${iso}T00:00:00.000Z`);
}
type RowSpec = {
id: string;
date: Date;
amount: string;
currency?: string;
domain?: string;
voidedAt?: Date | null;
outstanding?: boolean;
};
function row(r: RowSpec) {
return {
id: r.id,
transactionDate: r.date,
domain: r.domain ?? "UTILITY",
amount: new Prisma.Decimal(r.amount),
currency: r.currency ?? "MXN",
reference: null,
period: null,
checkNumber: null,
message: null,
legacySourceTable: null,
voidedAt: r.voidedAt ?? null,
outstanding: r.outstanding ?? false,
type: { nameEn: "WATER", nameEs: "AGUA" },
};
}
/** No BALANCE FORWARD row, so the floor is null and every row is fetched. */
function serviceWith(rows: RowSpec[]) {
const prisma = {
customer: {
findUnique: jest.fn().mockResolvedValue({
id: "c1",
name: "CUADROS, JORGE H.",
preferredCurrency: "MXN",
_count: { properties: 0, policies: 0 },
}),
},
transaction: {
findFirst: jest.fn().mockResolvedValue(null),
findMany: jest.fn().mockResolvedValue(rows.map(row)),
},
};
return new BillingService(prisma as never);
}
it("lists the year's movements oldest-first", async () => {
const s = await serviceWith([
{ id: "a", date: d(`${YEAR}-01-02`), amount: "-100" },
{ id: "b", date: d(`${YEAR}-03-04`), amount: "250" },
{ id: "c", date: d(`${YEAR}-07-16`), amount: "-40" },
]).statement("c1");
expect(s.movements.map((m) => m.id)).toEqual(["a", "b", "c"]);
});
it("leaves earlier years off the list", async () => {
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 1}-11-30`), amount: "-500" },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-100" },
]).statement("c1");
expect(s.movements.map((m) => m.id)).toEqual(["new"]);
});
it("carries the earlier years' balance instead of discarding it", async () => {
// 1,000 credit left over from last year, 300 charged this year: the
// customer is 700 in credit, not 300 in debt.
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 1}-12-15`), amount: "1000" },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-300" },
]).statement("c1");
const mxn = s.summary.find((x) => x.currency === "MXN");
expect(mxn?.opening).toBe("1000.00");
expect(mxn?.charges).toBe("-300.00");
expect(mxn?.balance).toBe("700.00");
// The running balance on the listed row picks up where last year left off.
expect(s.movements[0].balanceAfter).toBe("700.00");
});
it("carries it per business line as well", async () => {
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 1}-12-15`), amount: "1000", domain: "INSURANCE" },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-300", domain: "INSURANCE" },
]).statement("c1");
const line = s.byDomain.find((x) => x.domain === "INSURANCE");
expect(line?.opening).toBe("1000.00");
expect(line?.balance).toBe("700.00");
});
it("still reports a currency that only moved in earlier years", async () => {
// Otherwise a customer sitting on a dollar credit they haven't touched all
// year would appear to have no dollar balance at all.
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 2}-05-01`), amount: "180.83", currency: "USD" },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-300" },
]).statement("c1");
const usd = s.summary.find((x) => x.currency === "USD");
expect(usd?.balance).toBe("180.83");
expect(usd?.count).toBe(0);
});
it("does not carry a voided earlier row", async () => {
const s = await serviceWith([
{ id: "old", date: d(`${YEAR - 1}-12-15`), amount: "1000", voidedAt: d(`${YEAR - 1}-12-16`) },
{ id: "new", date: d(`${YEAR}-02-11`), amount: "-300" },
]).statement("c1");
const mxn = s.summary.find((x) => x.currency === "MXN");
expect(mxn?.opening).toBe("0.00");
expect(mxn?.balance).toBe("-300.00");
});
it("reports the year it covers", async () => {
const s = await serviceWith([
{ id: "a", date: d(`${YEAR}-01-02`), amount: "-100" },
]).statement("c1");
expect(s.year).toBe(YEAR);
});
});
@@ -0,0 +1,122 @@
import { CustomersService } from "./customers.service";
import { BALANCE_FORWARD_TYPE } from "../billing/billing.service";
/**
* The /clientes/:id ledger card is titled "Estado de cuenta" and links straight
* to the statement, so its per-line totals must be the statement's numbers.
*
* They were a raw lifetime sum — no floor, no source exclusion — which
* double-counted the pre-cutover history each BALANCE FORWARD row absorbs.
* Importing prior periods made it visibly worse: every closed year is now held
* a second time as its own tagged copy, so an unfloored sum adds each one on
* top of the opening balance that already contains it.
*/
describe("customer file ledger card", () => {
function serviceWith(floor: Date | null) {
const groupBy = jest.fn().mockResolvedValue([]);
const prisma = {
customer: {
findUnique: jest.fn().mockResolvedValue({ id: "c1", transactions: [] }),
},
transaction: {
findFirst: jest
.fn()
.mockResolvedValue(floor ? { transactionDate: floor } : null),
groupBy,
},
};
return {
service: new CustomersService(prisma as never),
prisma,
groupBy,
};
}
it("takes the same balance floor the statement takes", async () => {
const floor = new Date("2026-01-01T00:00:00Z");
const { service, prisma, groupBy } = serviceWith(floor);
await service.detail("c1");
expect(prisma.transaction.findFirst).toHaveBeenCalledWith(
expect.objectContaining({
where: expect.objectContaining({
type: { nameEn: BALANCE_FORWARD_TYPE },
}),
}),
);
expect(groupBy.mock.calls[0][0].where).toMatchObject({
transactionDate: { gte: floor },
});
});
it("applies no floor when the customer never had an opening balance", async () => {
// 102 customers have no BALANCE FORWARD row at all. Inventing a floor for
// them would hide their whole ledger.
const { service, groupBy } = serviceWith(null);
await service.detail("c1");
expect(groupBy.mock.calls[0][0].where).not.toHaveProperty("transactionDate");
});
it("counts an archive as history but never as current", async () => {
// The floor alone is not enough: a customer floored by an archive clears
// it with every row of that archive, and the rows archives spill into the
// following January clear any floor. But excluding archives outright is
// wrong too — below the year start they are the only carry a
// floored-by-archive customer has (NUMid 295, 785.46).
const { service, groupBy } = serviceWith(new Date("2026-01-01T00:00:00Z"));
await service.detail("c1");
const and = groupBy.mock.calls[0][0].where.AND;
const rule = and.find((c: { OR?: unknown[] }) =>
JSON.stringify(c).includes("datos2@"),
);
expect(rule.OR).toEqual([
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: "datos2@" } } },
{ transactionDate: { lt: expect.any(Date) } },
]);
});
it("keeps the cash-source exclusion so it reads like the statement", async () => {
const { service, groupBy } = serviceWith(new Date("2026-01-01T00:00:00Z"));
await service.detail("c1");
const and = groupBy.mock.calls[0][0].where.AND;
const sourceRule = and.find((c: { OR?: unknown[] }) =>
JSON.stringify(c).includes("EFECTIVO"),
);
expect(sourceRule).toBeDefined();
});
it("drops outstanding rows, as every balance does", async () => {
const { service, groupBy } = serviceWith(null);
await service.detail("c1");
expect(groupBy.mock.calls[0][0].where).toMatchObject({
outstanding: false,
});
});
it("keeps archives out of the year's movement list too", async () => {
// datos2@2024 carries rows dated into 2026; a date test alone would show
// them as current-year movements next to the live ledger's own copy.
const { service, prisma } = serviceWith(null);
await service.detail("c1");
const include = prisma.customer.findUnique.mock.calls[0][0].include;
expect(include.transactions.where.OR).toEqual([
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: "datos2@" } } },
// Nothing below yearStart reaches this list, so the third branch never
// admits an archive row here — it is carried for one shared rule.
{ transactionDate: { lt: expect.any(Date) } },
]);
});
});
+88 -4
View File
@@ -3,6 +3,35 @@ import { Prisma } from "@jorgecuadros/database";
import { PrismaService } from "../prisma/prisma.service";
import { CreateCustomerDto } from "./create-customer.dto";
import { UpdateCustomerDto } from "./update-customer.dto";
import {
BALANCE_FORWARD_TYPE,
notCashJournal,
PERIOD_TABLE_PREFIX,
} from "../billing/billing.service";
/**
* Keeps an imported prior period out of the *current* period, NULL-safely.
*
* A closed year is imported as its own tagged copy (`datos2@2025`). Below the
* year start it is history and counts — for the one customer whose newest
* BALANCE FORWARD lives inside an archive it is the only carry there is, and
* dropping it understated NUMid 295 by his entire 2025 closing balance. At or
* above the year start it must go: the archives spill a couple of rows into the
* following January, and those already sit inside the next year's BALANCE
* FORWARD, which is the sum of the whole archive.
*
* Spelled as a positive OR because `NOT (col LIKE ... AND ...)` evaluates to
* NULL for an app-captured row (no legacySourceTable), dropping every one.
*/
const archiveIsHistory = (
yearStart: Date,
): Prisma.TransactionWhereInput => ({
OR: [
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: PERIOD_TABLE_PREFIX } } },
{ transactionDate: { lt: yearStart } },
],
});
export interface ListParams {
query?: string;
@@ -96,6 +125,13 @@ export class CustomersService {
/** Full unified customer view: identity + both business lines + ledger. */
async detail(id: string) {
// The movement list on the customer file is the same statement the office
// prints, so it follows the same rule as BillingService.statement: this
// calendar year, oldest-first. No `take` any more — the cap used to hide
// the end of a busy customer's year once the order flipped, and a single
// year is small (365 rows for the heaviest customer in the book).
const yearStart = new Date(Date.UTC(new Date().getUTCFullYear(), 0, 1));
const customer = await this.prisma.customer.findUnique({
where: { id },
include: {
@@ -117,8 +153,21 @@ export class CustomersService {
},
},
transactions: {
orderBy: { transactionDate: "desc" },
take: 100,
// Archives are excluded by tag, not by date. They are not cleanly
// bounded — datos2@2024 carries rows dated 2022, 2023, 2025 and one
// in 2026, datos2@2025 two more — so a date test alone would surface
// a closed year's rows in the current year's list, duplicating the
// live ledger's own copy of them for three customers.
// Archives are kept out by tag, not by date. They are not cleanly
// bounded — datos2@2025 carries rows dated into 2026 — so a date test
// alone would surface a closed year's rows in the current year's
// list. Nothing below yearStart reaches this list anyway, so the
// window rule reduces to a plain exclusion here.
where: {
transactionDate: { gte: yearStart },
...archiveIsHistory(yearStart),
},
orderBy: [{ transactionDate: "asc" }, { id: "asc" }],
include: { type: true },
},
},
@@ -130,16 +179,51 @@ export class CustomersService {
// Ledger totals per domain + currency (the "one statement across both
// business lines" payoff), computed in the DB rather than in JS.
//
// These have to answer the same question BillingService.statement answers,
// because this card is titled "Estado de cuenta" and links straight to it —
// two screens quoting one customer two different balances is worse than
// either number alone. So it takes the same three rules the statement uses:
// the balance floor, the cash-source exclusion, and dropping outstanding
// rows the office has not paid yet.
//
// Without the floor these were a raw lifetime sum, double-counting the
// pre-cutover history each BALANCE FORWARD row already absorbs. Importing
// prior periods made that visibly worse: for NUMid 501 the tiles read
// -7,119.29 before the archives landed and -15,270.59 after, against a true
// -10,715.29 — the difference being exactly the 2024 and 2025 closing
// balances, added a second time on top of the opening row that contains
// them.
const floor = await this.prisma.transaction.findFirst({
where: {
customerId: id,
voidedAt: null,
type: { nameEn: BALANCE_FORWARD_TYPE },
},
orderBy: { transactionDate: "desc" },
select: { transactionDate: true },
});
const summary = await this.prisma.transaction.groupBy({
by: ["domain", "currency"],
// Exclude voided rows so the per-domain balance matches the statement.
where: { customerId: id, voidedAt: null },
where: {
customerId: id,
voidedAt: null,
outstanding: false,
...(floor ? { transactionDate: { gte: floor.transactionDate } } : {}),
// The floor alone does not settle the archives: a customer floored by
// an archive clears it with every row of that archive, and the rows
// archives spill into the following January clear any floor.
AND: [archiveIsHistory(yearStart), notCashJournal()],
},
_sum: { amount: true },
_count: { _all: true },
});
return {
...customer,
/** Calendar year the movement list covers. */
transactionYear: yearStart.getUTCFullYear(),
transactionSummary: summary.map((s) => ({
domain: s.domain,
currency: s.currency,
+76 -8
View File
@@ -31,6 +31,29 @@ export const INGEST_FILES = [
] as const;
export type IngestName = (typeof INGEST_FILES)[number];
/**
* A prior-period archive: one Access snapshot per closed year, named for the
* period it holds. `2025.accdb` is UTILITIES as it stood when 2025 was cut.
*
* The filename is the entire declaration of the period — nothing inside the
* file names its year, because a snapshot's `datos2` is indistinguishable from
* the live one — so this pattern is both the allowlist and the contract. It is
* anchored and allows no separator, which is what keeps an upload from
* escaping the ingest directory.
*/
const PERIOD_FILE_RE = /^(\d{4})\.accdb$/i;
/** Earliest period we will accept, so a typo'd year cannot mint a bogus one. */
const PERIOD_MIN_YEAR = 1990;
export function periodYearOf(name: string): number | null {
const m = PERIOD_FILE_RE.exec(name);
if (!m) return null;
const year = Number(m[1]);
if (year < PERIOD_MIN_YEAR || year > new Date().getUTCFullYear()) return null;
return year;
}
/**
* Prefix for every command containing a pipe. Without it the exit status of
* `mysqldump | gzip` is gzip's, so a dump that failed immediately still looks
@@ -120,19 +143,30 @@ export class OpsService implements OnModuleInit {
/* -------------------------------------------------------------- ingest */
private assertIngestName(name: string): IngestName {
if (!INGEST_FILES.includes(name as IngestName)) {
private assertIngestName(name: string): string {
if (INGEST_FILES.includes(name as IngestName)) return name;
if (periodYearOf(name) !== null) return name;
throw new BadRequestException(
`Archivo no permitido. Debe ser uno de: ${INGEST_FILES.join(", ")}`,
`Archivo no permitido. Debe ser uno de: ${INGEST_FILES.join(", ")}` +
`, o un archivo de periodo anterior con nombre AAAA.accdb (por ejemplo 2025.accdb).`,
);
}
return name as IngestName;
}
async listIngest(): Promise<
{ name: string; present: boolean; size: number | null; modifiedAt: string | null }[]
{
name: string;
present: boolean;
size: number | null;
modifiedAt: string | null;
/** Set only on a prior-period archive; null on the four fixed sources. */
periodYear: number | null;
}[]
> {
return Promise.all(
// The four fixed sources are listed whether present or not — they are
// required, so "missing" is the useful state to show. Period archives are
// optional and unbounded, so they are listed only once uploaded, newest
// year first.
const fixed = await Promise.all(
INGEST_FILES.map(async (name) => {
try {
const st = await fs.stat(path.join(this.ingestDir, name));
@@ -141,12 +175,46 @@ export class OpsService implements OnModuleInit {
present: true,
size: st.size,
modifiedAt: st.mtime.toISOString(),
periodYear: null as number | null,
};
} catch {
return { name, present: false, size: null, modifiedAt: null };
return {
name,
present: false,
size: null,
modifiedAt: null,
periodYear: null as number | null,
};
}
}),
);
let entries: string[] = [];
try {
entries = await fs.readdir(this.ingestDir);
} catch {
entries = [];
}
const periods = (
await Promise.all(
entries
.map((name) => ({ name, year: periodYearOf(name) }))
.filter((e): e is { name: string; year: number } => e.year !== null)
.sort((a, b) => b.year - a.year)
.map(async ({ name, year }) => {
const st = await fs.stat(path.join(this.ingestDir, name));
return {
name,
present: true,
size: st.size,
modifiedAt: st.mtime.toISOString(),
periodYear: year,
};
}),
)
).filter(Boolean);
return [...fixed, ...periods];
}
async saveIngest(name: string, data: Buffer): Promise<void> {
+48
View File
@@ -0,0 +1,48 @@
import { periodYearOf } from "./ops.service";
/**
* `periodYearOf` is the upload allowlist for prior-period archives, so it is
* doing two jobs at once: deciding what counts as a period file, and keeping a
* caller-supplied name from escaping the ingest directory. Both are pinned here.
*/
describe("periodYearOf", () => {
it("accepts a four-digit year archive", () => {
expect(periodYearOf("2025.accdb")).toBe(2025);
expect(periodYearOf("1999.accdb")).toBe(1999);
});
it("is case-insensitive on the extension", () => {
expect(periodYearOf("2025.ACCDB")).toBe(2025);
});
it("rejects a path that would escape the ingest directory", () => {
// The name is joined onto the ingest path, so anything with a separator or
// a parent reference has to fail before it reaches the filesystem.
expect(periodYearOf("../2025.accdb")).toBeNull();
expect(periodYearOf("../../etc/passwd")).toBeNull();
expect(periodYearOf("sub/2025.accdb")).toBeNull();
expect(periodYearOf("2025.accdb/../../x")).toBeNull();
});
it("rejects names that only look like a period", () => {
expect(periodYearOf("202.accdb")).toBeNull();
expect(periodYearOf("20255.accdb")).toBeNull();
expect(periodYearOf("2025.mdb")).toBeNull();
expect(periodYearOf("copia 2025.accdb")).toBeNull();
expect(periodYearOf("2025.accdb.bak")).toBeNull();
expect(periodYearOf("UTILITIES.accdb")).toBeNull();
});
it("rejects years outside the plausible range", () => {
// A typo'd year would otherwise mint a period nobody can ever reconcile:
// there is no BALANCE FORWARD for the year after it to check against.
expect(periodYearOf("1889.accdb")).toBeNull();
expect(periodYearOf(`${new Date().getUTCFullYear() + 1}.accdb`)).toBeNull();
});
it("accepts the current year, which is the earliest a period can be cut", () => {
expect(periodYearOf(`${new Date().getUTCFullYear()}.accdb`)).toBe(
new Date().getUTCFullYear(),
);
});
});
+18
View File
@@ -12,6 +12,10 @@ import { Currency } from "@jorgecuadros/database";
// Each child DTO covers create; updates reuse the same shape with all fields
// optional via the corresponding Update class. Route supplies the policyId.
// A policy split into several exhibiciones prices each payment on its own —
// the Access form printed the whole money row once per pago — so the premium
// breakdown repeats here. `amount` remains what was actually collected and is
// never recomputed from the breakdown; the two differ by rounding in the books.
export class InstallmentDto {
@IsInt() sequence!: number;
@IsOptional() @IsNumber() amount?: number;
@@ -20,6 +24,13 @@ export class InstallmentDto {
@IsOptional() @IsString() paidDate?: string;
@IsOptional() @IsString() checkNumber?: string;
@IsOptional() @IsBoolean() isCash?: boolean;
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() surcharge?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() taxRate?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsNumber() commission?: number;
}
export class UpdateInstallmentDto {
@IsOptional() @IsInt() sequence?: number;
@@ -29,6 +40,13 @@ export class UpdateInstallmentDto {
@IsOptional() @IsString() paidDate?: string;
@IsOptional() @IsString() checkNumber?: string;
@IsOptional() @IsBoolean() isCash?: boolean;
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() surcharge?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() taxRate?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsNumber() commission?: number;
}
export class VehicleDto {
+7 -1
View File
@@ -1,4 +1,4 @@
import { IsOptional, IsString, MinLength } from "class-validator";
import { IsNumber, IsOptional, IsString, Max, Min, MinLength } from "class-validator";
export class ProviderDto {
@IsString() @MinLength(1) name!: string;
@@ -7,13 +7,19 @@ export class UpdateProviderDto {
@IsOptional() @IsString() @MinLength(1) name?: string;
}
// `taxRate` is the IVA fraction for this line of business (0.08 = 8%), the
// legacy one-row IMPUESTOS / IMPUESTOS_AUTOS tables made editable. Bounded at
// 1 because a rate is a fraction, not a percentage: 8 entered here would tax a
// $600 premium $4,800, and the mistake is easy to make.
export class PolicyTypeDto {
@IsString() @MinLength(1) name!: string;
@IsOptional() @IsString() shortDescription?: string;
@IsOptional() @IsNumber() @Min(0) @Max(1) taxRate?: number;
}
export class UpdatePolicyTypeDto {
@IsOptional() @IsString() @MinLength(1) name?: string;
@IsOptional() @IsString() shortDescription?: string;
@IsOptional() @IsNumber() @Min(0) @Max(1) taxRate?: number;
}
export class AdjusterDto {
+31 -2
View File
@@ -250,7 +250,16 @@ export class PoliciesService {
const [types, providers] = await this.prisma.$transaction([
this.prisma.policyType.findMany({
orderBy: { name: "asc" },
select: { id: true, name: true, _count: { select: { policies: true } } },
select: {
id: true,
name: true,
shortDescription: true,
// The capture form computes IVA client-side as the operator types,
// so the rate has to travel with the type list it already loads —
// an extra round-trip per keystroke is not an option.
taxRate: true,
_count: { select: { policies: true } },
},
}),
this.prisma.insuranceProvider.findMany({
orderBy: { name: "asc" },
@@ -259,7 +268,13 @@ export class PoliciesService {
]);
return {
types: types.map((t) => ({ id: t.id, name: t.name, count: t._count.policies })),
types: types.map((t) => ({
id: t.id,
name: t.name,
shortDescription: t.shortDescription,
taxRate: t.taxRate,
count: t._count.policies,
})),
providers: providers.map((p) => ({
id: p.id,
name: p.name,
@@ -397,6 +412,13 @@ export class PoliciesService {
paidDate: toDate(dto.paidDate) ?? undefined,
checkNumber: dto.checkNumber,
isCash: dto.isCash,
netPremium: dto.netPremium,
surcharge: dto.surcharge,
policyFee: dto.policyFee,
tax: dto.tax,
taxRate: dto.taxRate,
total: dto.total,
commission: dto.commission,
},
});
}
@@ -412,6 +434,13 @@ export class PoliciesService {
...(dto.paidDate !== undefined && { paidDate: toDate(dto.paidDate) }),
checkNumber: dto.checkNumber,
isCash: dto.isCash,
netPremium: dto.netPremium,
surcharge: dto.surcharge,
policyFee: dto.policyFee,
tax: dto.tax,
taxRate: dto.taxRate,
total: dto.total,
commission: dto.commission,
},
});
}
+14 -2
View File
@@ -6,12 +6,16 @@ import {
IsString,
MinLength,
} from "class-validator";
import { Currency } from "@jorgecuadros/database";
import { Currency, PaymentFrequency } from "@jorgecuadros/database";
import { IsEnum } from "class-validator";
/** Editable policy-header fields. coveragesJson (freeform legacy blob) is not
* exposed for editing. Dates arrive as ISO strings and are coerced by the
* service. `total` is legacy-dead data — the UI uses netPremium. */
* service. `total` is legacy-dead data on migrated rows — list and sort code
* still uses netPremium — but the capture form writes it going forward, along
* with `tax`, from the arithmetic in premium.ts. Both arrive as plain numbers
* rather than being recomputed server-side: the printed policy is the record
* of truth and staff must be able to key its rounding verbatim. */
export class CreatePolicyDto {
@IsString() @MinLength(1) policyNumber!: string;
@IsString() @MinLength(1) customerId!: string;
@@ -24,10 +28,14 @@ export class CreatePolicyDto {
@IsOptional() @IsString() policyTo?: string;
@IsOptional() @IsInt() coveragePeriodDays?: number;
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() surcharge?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() brokerFee?: number;
@IsOptional() @IsNumber() commission?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() taxRate?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsEnum(PaymentFrequency) paymentFrequency?: PaymentFrequency;
@IsOptional() @IsEnum(Currency) currency?: Currency;
@IsOptional() @IsString() observations?: string;
@IsOptional() @IsString() notes?: string;
@@ -48,10 +56,14 @@ export class UpdatePolicyDto {
@IsOptional() @IsString() policyTo?: string;
@IsOptional() @IsInt() coveragePeriodDays?: number;
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() surcharge?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() brokerFee?: number;
@IsOptional() @IsNumber() commission?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() taxRate?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsEnum(PaymentFrequency) paymentFrequency?: PaymentFrequency;
@IsOptional() @IsEnum(Currency) currency?: Currency;
@IsOptional() @IsString() observations?: string;
@IsOptional() @IsString() notes?: string;
+90
View File
@@ -0,0 +1,90 @@
import {
DEFAULT_TAX_RATE,
computeTax,
computeTotal,
resolveTaxRate,
surchargeApplies,
taxableBase,
} from "./premium";
/**
* The reference case is policy 7006785 (MULT, semestral, GMX, two payments) as
* it stands in the Access books — the screen Jorge sent. Both of its money
* rows are asserted, because the second one is the case that proves the
* surcharge belongs in the taxable base and that a zero policy fee is a real
* value rather than a missing one.
*/
describe("premium arithmetic", () => {
it("matches the first payment of policy 7006785", () => {
const parts = { netPremium: 610.86, surcharge: 8.55, policyFee: 31.0 };
expect(taxableBase(parts)).toBe(650.41);
expect(computeTax(parts, 0.08)).toBe(52.03);
expect(computeTotal(parts, 0.08)).toBe(702.44);
});
it("matches the second payment of policy 7006785", () => {
const parts = { netPremium: 589.71, surcharge: 8.26, policyFee: 0 };
expect(computeTax(parts, 0.08)).toBe(47.84);
expect(computeTotal(parts, 0.08)).toBe(645.81);
});
it("excluding the surcharge does NOT reconcile", () => {
// Guards the one decision in this module that is easy to get wrong: the
// spoken-language version of the rule ("prima neta + derecho * 8%") gives
// 51.35, and the printed policy says 52.03.
const withoutSurcharge = { netPremium: 610.86, surcharge: 0, policyFee: 31.0 };
expect(computeTax(withoutSurcharge, 0.08)).not.toBe(52.03);
});
it("treats blank and null money as zero, not NaN", () => {
expect(taxableBase({ netPremium: "610.86", surcharge: null, policyFee: "" })).toBe(
610.86,
);
expect(computeTax({ netPremium: undefined, surcharge: null, policyFee: null }, 0.08))
.toBe(0);
});
it("rounds half-up to cents", () => {
// 100.06 * 0.08 = 8.0048 -> 8.00; 100.13 * 0.08 = 8.0104 -> 8.01.
expect(computeTax({ netPremium: 100.06, surcharge: 0, policyFee: 0 }, 0.08)).toBe(8);
expect(computeTax({ netPremium: 100.13, surcharge: 0, policyFee: 0 }, 0.08)).toBe(8.01);
});
describe("surchargeApplies", () => {
it("is false for the two single-payment frequencies", () => {
expect(surchargeApplies("ANNUAL")).toBe(false);
expect(surchargeApplies("SINGLE")).toBe(false);
});
it("is true for every split frequency", () => {
expect(surchargeApplies("SEMIANNUAL")).toBe(true);
expect(surchargeApplies("QUARTERLY")).toBe(true);
expect(surchargeApplies("MONTHLY")).toBe(true);
});
it("allows it when the frequency is unknown", () => {
// Every migrated policy is null here — the original ETL dropped FORMA
// PAGO — and those rows DO carry recargo figures in the legacy data.
expect(surchargeApplies(null)).toBe(true);
expect(surchargeApplies(undefined)).toBe(true);
});
});
describe("resolveTaxRate", () => {
it("prefers the rate the policy was issued at", () => {
expect(resolveTaxRate(0.16, 0.08)).toBe(0.16);
});
it("falls back to the line of business", () => {
expect(resolveTaxRate(null, 0.08)).toBe(0.08);
});
it("falls back to the default when nothing is configured", () => {
expect(resolveTaxRate(null, null)).toBe(DEFAULT_TAX_RATE);
expect(resolveTaxRate(undefined, "")).toBe(DEFAULT_TAX_RATE);
});
it("accepts a zero rate as a real choice, not as absent", () => {
// An exempt line of business must read 0, not silently fall through to 8%.
expect(resolveTaxRate(null, 0)).toBe(0);
});
it("accepts Prisma's decimal strings", () => {
expect(resolveTaxRate(null, "0.0800")).toBe(0.08);
});
});
});
+86
View File
@@ -0,0 +1,86 @@
/**
* The premium arithmetic the Access capture form did in unbound calculated
* controls, moved somewhere it can be tested.
*
* Two figures are derived, everything else is keyed by hand:
*
* base = netPremium + surcharge + policyFee
* tax = round(base * rate)
* total = base + tax
*
* The surcharge IS part of the taxable base. That is not an assumption — it is
* the only reading that reconciles the books. Policy 7006785 (MULT, semestral,
* two payments) prints IVA 52.03 and 47.84 against net premiums 610.86 / 589.71,
* surcharges 8.55 / 8.26 and policy fees 31.00 / 0.00; excluding the surcharge
* gives 51.35, which matches nothing on the page.
*
* The surcharge itself is NEVER derived. It is the carrier's financing charge
* for paying in installments, quoted per policy, so staff key it in. It only
* ever appears on a policy that is not paid annually or in a single exhibición
* — `surchargeApplies` is what the UI uses to grey the field out.
*/
/** Used when neither the policy nor its type carries a rate. Matches the
* single row both legacy IMPUESTOS tables held (0.08 = 8%). */
export const DEFAULT_TAX_RATE = 0.08;
export type PaymentFrequencyValue =
| "ANNUAL"
| "SEMIANNUAL"
| "QUARTERLY"
| "MONTHLY"
| "SINGLE";
/** Paying in more than one exhibición is what earns a surcharge. A null
* frequency (every migrated row — Access's FORMA PAGO was dropped by the
* original ETL) is treated as "unknown, allow it" rather than "annual":
* refusing to show a figure that is sitting in the legacy data would hide it. */
export function surchargeApplies(
frequency: PaymentFrequencyValue | null | undefined,
): boolean {
return frequency !== "ANNUAL" && frequency !== "SINGLE";
}
function num(v: unknown): number {
if (v === null || v === undefined || v === "") return 0;
const n = typeof v === "number" ? v : Number(v);
return Number.isFinite(n) ? n : 0;
}
/** Half-up to cents, the way the printed policy rounds. */
export function round2(n: number): number {
return Math.round((n + Number.EPSILON) * 100) / 100;
}
export interface PremiumParts {
netPremium?: unknown;
surcharge?: unknown;
policyFee?: unknown;
}
export function taxableBase(p: PremiumParts): number {
return round2(num(p.netPremium) + num(p.surcharge) + num(p.policyFee));
}
export function computeTax(p: PremiumParts, rate: number): number {
return round2(taxableBase(p) * rate);
}
export function computeTotal(p: PremiumParts, rate: number): number {
return round2(taxableBase(p) + computeTax(p, rate));
}
/** Rate ladder: the figure stored on the policy (so an old policy keeps the
* rate it was issued at even after the catalog changes), else the rate on its
* line of business, else the shipped default. */
export function resolveTaxRate(
policyRate: unknown,
policyTypeRate: unknown,
): number {
for (const candidate of [policyRate, policyTypeRate]) {
if (candidate === null || candidate === undefined || candidate === "") continue;
const n = Number(candidate);
if (Number.isFinite(n) && n >= 0) return n;
}
return DEFAULT_TAX_RATE;
}
@@ -689,8 +689,23 @@ describe("parsePolicy / ANA automobile", () => {
// reading would shift every value one column left.
expect(p.netPremium).toBe(298.61);
expect(p.policyFee).toBe(30);
expect(p.tax).toBe(26.29);
expect(p.total).toBe(354.9);
expect(p.notes.join(" | ")).toMatch(/impuesto: 26\.29/);
});
it("reads a TAX that reconciles against the rest of the row", () => {
// 298.61 + 30.00 = 328.61, taxed at 8% -> 26.29, totalling 354.90. The
// whole row agreeing is what proves the positional mapping landed on the
// right cells rather than merely on six numbers.
const base = p.netPremium! + p.policyFee!;
expect(Math.round(base * 0.08 * 100) / 100).toBe(p.tax);
expect(Math.round((base + p.tax!) * 100) / 100).toBe(p.total);
});
it("does not fold LOCAL TAX into the IVA", () => {
// It prints 0.00 here, so nothing to fold — but the guard is that a
// non-zero one would surface as a note instead of inflating `tax`.
expect(p.notes.join(" | ")).not.toMatch(/impuesto local/);
});
it("reads the vehicle by token role, not by column", () => {
@@ -36,6 +36,17 @@ export interface ParsedPolicy {
netPremium: number | null;
policyFee: number | null;
brokerFee: number | null;
/**
* IVA, off A.N.A.'s `TAX` cell. Null on GMX — its certificate carries no
* premium at all, so there is no tax on it to read either.
*
* The adjacent `LOCAL TAX` cell is deliberately NOT folded in here. It is a
* separate levy with no column of its own, and summing the two would report
* an IVA figure that no longer divides back to a rate — the whole point of
* storing it. It prints 0.00 on every policy seen so far and is surfaced as
* a note when it is not.
*/
tax: number | null;
total: number | null;
/** "CONTADO" / "MENSUAL" / … — premium-payment cadence text. */
premiumPayment: string | null;
@@ -319,6 +330,7 @@ function emptyParsedPolicy(provider: string): ParsedPolicy {
netPremium: null,
policyFee: null,
brokerFee: null,
tax: null,
total: null,
premiumPayment: null,
coverages: [],
@@ -1178,6 +1190,7 @@ interface AnaHeader {
coveragePeriodDays: number | null;
netPremium: number | null;
policyFee: number | null;
tax: number | null;
total: number | null;
}
@@ -1253,8 +1266,13 @@ function parseAnaHeader(lines: string[], notes: string[]): AnaHeader {
// ----- money row ---------------------------------------------------------
const row = anaMoneyRow(lines);
if (!row) notes.push("no se pudo leer el renglón de primas");
if (row?.tax) notes.push(`impuesto: ${row.tax.toFixed(2)}`);
if (row?.discount) notes.push(`descuento: ${row.discount.toFixed(2)}`);
// LOCAL TAX has no destination column and prints 0.00 on every A.N.A. policy
// seen so far. A non-zero one means the total will not reconcile against the
// stored IVA, so say so rather than folding it in and hiding the difference.
if (row?.localTax) {
notes.push(`impuesto local ${row.localTax.toFixed(2)} no capturado`);
}
return {
policyNumber,
@@ -1266,6 +1284,7 @@ function parseAnaHeader(lines: string[], notes: string[]): AnaHeader {
coveragePeriodDays,
netPremium: row?.netPremium ?? null,
policyFee: row?.policyFee ?? null,
tax: row?.tax ?? null,
total: row?.total ?? null,
};
}
@@ -1455,6 +1474,7 @@ function parseAnaAutomobile(lines: string[]): ParsedPolicy {
currency: anaCurrency(text, notes),
netPremium: header.netPremium,
policyFee: header.policyFee,
tax: header.tax,
total: header.total,
premiumPayment: paymentDeadline,
coverages,
@@ -1854,6 +1874,7 @@ function parseAnaDriverPolicy(lines: string[]): ParsedPolicy {
currency: anaCurrency(text, notes),
netPremium: header.netPremium,
policyFee: header.policyFee,
tax: header.tax,
total: header.total,
coverages,
coveragePeriodDays: header.coveragePeriodDays,
@@ -43,6 +43,7 @@ export class ConfirmPolicyDocumentDto {
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() brokerFee?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsString() premiumPayment?: string;
/** Printed term in days. Omitted leaves the parsed value (or the schema's
@@ -79,6 +80,7 @@ export class ReviewPolicyDocumentDto {
@IsOptional() @IsNumber() netPremium?: number;
@IsOptional() @IsNumber() policyFee?: number;
@IsOptional() @IsNumber() brokerFee?: number;
@IsOptional() @IsNumber() tax?: number;
@IsOptional() @IsNumber() total?: number;
@IsOptional() @IsString() premiumPayment?: string;
@IsOptional() @IsInt() @Min(1) @Max(3660) coveragePeriodDays?: number;
@@ -192,6 +192,8 @@ export class PolicyOcrService {
parsed.policyFee != null ? new Prisma.Decimal(parsed.policyFee) : null,
extractedBrokerFee:
parsed.brokerFee != null ? new Prisma.Decimal(parsed.brokerFee) : null,
extractedTax:
parsed.tax != null ? new Prisma.Decimal(parsed.tax) : null,
extractedTotal:
parsed.total != null ? new Prisma.Decimal(parsed.total) : null,
extractedCoveragesJson: parsed.coverages.length
@@ -365,6 +367,7 @@ export class PolicyOcrService {
dto.policyFee != null ? new Prisma.Decimal(dto.policyFee) : undefined,
extractedBrokerFee:
dto.brokerFee != null ? new Prisma.Decimal(dto.brokerFee) : undefined,
extractedTax: dto.tax != null ? new Prisma.Decimal(dto.tax) : undefined,
extractedTotal:
dto.total != null ? new Prisma.Decimal(dto.total) : undefined,
extractedCoveragesJson: dto.coveragesJson
@@ -799,6 +802,7 @@ function buildPolicyUpdateFromDoc(
extractedNetPremium: Prisma.Decimal | null;
extractedPolicyFee: Prisma.Decimal | null;
extractedBrokerFee: Prisma.Decimal | null;
extractedTax: Prisma.Decimal | null;
extractedTotal: Prisma.Decimal | null;
extractedCoveragesJson: Prisma.JsonValue | null;
extractedPremiumPayment: string | null;
@@ -845,6 +849,10 @@ function buildPolicyUpdateFromDoc(
netPremium: numOrUndef(item.netPremium, doc.extractedNetPremium),
policyFee: numOrUndef(item.policyFee, doc.extractedPolicyFee),
brokerFee: numOrUndef(item.brokerFee, doc.extractedBrokerFee),
// `taxRate` is deliberately left alone. A.N.A. prints the IVA amount, not
// the rate, and back-dividing it would mint a rate the document never
// stated — the policy form resolves one from the line of business instead.
tax: numOrUndef(item.tax, doc.extractedTax),
total: numOrUndef(item.total, doc.extractedTotal),
// coveragesJson / observations: freeform, keep the GMX data when present.
coveragesJson:
@@ -886,6 +894,7 @@ function buildPolicyCreateFromDoc(
extractedNetPremium: Prisma.Decimal | null;
extractedPolicyFee: Prisma.Decimal | null;
extractedBrokerFee: Prisma.Decimal | null;
extractedTax: Prisma.Decimal | null;
extractedTotal: Prisma.Decimal | null;
extractedCoveragesJson: Prisma.JsonValue | null;
extractedPremiumPayment: string | null;
@@ -936,6 +945,10 @@ function buildPolicyCreateFromDoc(
netPremium: numOrUndef(item.netPremium, doc.extractedNetPremium),
policyFee: numOrUndef(item.policyFee, doc.extractedPolicyFee),
brokerFee: numOrUndef(item.brokerFee, doc.extractedBrokerFee),
// `taxRate` is deliberately left alone. A.N.A. prints the IVA amount, not
// the rate, and back-dividing it would mint a rate the document never
// stated — the policy form resolves one from the line of business instead.
tax: numOrUndef(item.tax, doc.extractedTax),
total: numOrUndef(item.total, doc.extractedTotal),
coveragesJson:
item.coveragesJson !== undefined
@@ -46,6 +46,20 @@ describe("renderRenewalEmail", () => {
expect(result.html).toContain("Calle Uno 123");
});
it("omits the premium when the sender did not ask for it", () => {
// The unattended sweep quotes no amount: the premium can still be
// re-rated at renewal, and a number a robot mailed out is one the office
// has to walk back.
const result = renderRenewalEmail(letter(), { includePremium: false });
expect(result.html).not.toContain("Prima");
expect(result.html).not.toContain("1,392.00");
// Everything else the customer needs is still there.
expect(result.html).toContain("POL-123");
expect(result.html).toContain("01/09/2026");
expect(result.html).toContain("Ana Pérez");
});
it("uses overdue wording for generation three", () => {
const result = renderRenewalEmail(letter({ generation: 3 }));
+15 -2
View File
@@ -34,10 +34,23 @@ function row(label: string, value: string): string {
return `<tr><th style="padding:8px 12px;text-align:left;background:#f4f4f4;border:1px solid #ddd">${escapeHtml(label)}</th><td style="padding:8px 12px;border:1px solid #ddd">${escapeHtml(value)}</td></tr>`;
}
export function renderRenewalEmail(letter: RenewalLetterRow): {
/**
* Render one renewal letter.
*
* `includePremium` decides whether the "Prima" row appears. The unattended
* sweep sends without it — an amount quoted by a robot, on a premium that may
* still be re-rated at renewal, is a number the office has to walk back — and
* every staff-triggered send (the manual barrido and the per-row "Enviar
* aviso") keeps it, because a person chose to quote it.
*/
export function renderRenewalEmail(
letter: RenewalLetterRow,
options: { includePremium?: boolean } = {},
): {
subject: string;
html: string;
} {
const includePremium = options.includePremium !== false;
const expired = letter.generation === 3;
const subject = expired
? `Póliza vencida: ${letter.policyNumber}`
@@ -51,7 +64,7 @@ export function renderRenewalEmail(letter: RenewalLetterRow): {
row("Tipo de póliza", letter.policyType),
row("Aseguradora", letter.provider),
row("Fecha de vencimiento", displayDate(letter.policyTo)),
row("Prima", money(premium, letter.currency)),
...(includePremium ? [row("Prima", money(premium, letter.currency))] : []),
row("Cliente", letter.customerName),
row("Correo", letter.customerEmail ?? "No disponible"),
row("Teléfono", phone),
@@ -183,6 +183,50 @@ describe("renewal notices write the shared notification log", () => {
expect(prisma.renewalNotice.upsert).not.toHaveBeenCalled();
});
it("quotes the premium on a staff-triggered sweep but not the scheduled one", async () => {
const manual = build({});
await manual.service.sweep("user-1");
expect(manual.record.mock.calls[0][0].bodySnapshot).toContain("Prima");
const automatic = build({});
await automatic.service.scheduledSweep();
const body = automatic.record.mock.calls[0][0].bodySnapshot;
// The snapshot has to match the mail that actually went out, or the
// office reads a letter the customer never received.
expect(body).not.toContain("Prima");
expect(body).toContain("700442181");
});
it("scopes a sweep to one aseguradora without advancing the catch-up window", async () => {
const { service, prisma, send } = build({});
const result = await service.sweep("user-1", { providerId: "gmx-id" });
expect(result.sent).toBe(1);
expect(result.providerId).toBe("gmx-id");
expect(prisma.policy.findMany.mock.calls[0][0].where).toMatchObject({
insuranceProviderId: "gmx-id",
});
expect(send).toHaveBeenCalledTimes(1);
// Only one carrier was mailed, so the days this run covered are still owed
// to every other carrier: advancing `lastSuccessfulAt` would move them out
// of tomorrow's window and they would never be sent.
const release = prisma.scheduledJobState.update.mock.calls.at(-1)?.[0];
expect(release.data.lastSuccessfulAt).toBeUndefined();
});
it("advances the catch-up window on a clean unfiltered sweep", async () => {
const { service, prisma } = build({});
await service.sweep("user-1");
expect(prisma.policy.findMany.mock.calls[0][0].where).not.toHaveProperty(
"insuranceProviderId",
);
const release = prisma.scheduledJobState.update.mock.calls.at(-1)?.[0];
expect(release.data.lastSuccessfulAt).toBeInstanceOf(Date);
});
it("does not fail a delivered notice when the log write throws", async () => {
const { service, record } = build({});
record.mockRejectedValue(new Error("log table gone"));
+14 -2
View File
@@ -25,6 +25,13 @@ class RenewalFlagsDto {
debug?: boolean;
}
class SweepRenewalsDto extends RenewalFlagsDto {
/** Sweep one aseguradora only (GMX, ANA, …). Omitted = todas. */
@IsOptional()
@IsString()
providerId?: string;
}
class SendRenewalDto extends RenewalFlagsDto {
@IsString()
policyId!: string;
@@ -43,17 +50,22 @@ export class RenewalsController {
constructor(private readonly renewals: RenewalsService) {}
@Get("pending")
pending(@Query("days") days?: string) {
pending(
@Query("days") days?: string,
@Query("providerId") providerId?: string,
) {
return this.renewals.pending(
Math.min(365, Math.max(1, Number(days) || 30)),
providerId?.trim() || undefined,
);
}
@Post("sweep")
@RequireAbility("renewal:send")
sweep(@Body() dto: RenewalFlagsDto, @Req() req: Request) {
sweep(@Body() dto: SweepRenewalsDto, @Req() req: Request) {
return this.renewals.sweep((req.user as { id: string }).id, {
debug: dto?.debug,
providerId: dto?.providerId,
});
}
+55 -10
View File
@@ -84,10 +84,14 @@ export class RenewalsService implements OnModuleInit {
/** The unattended run always sends for real: `debug` is a per-click switch
* in the UI, never persisted, so the schedule cannot inherit a forgotten
* test toggle and silently stop mailing customers. */
* test toggle and silently stop mailing customers.
*
* `automatic` is what drops the premium from the letter — see
* `renderRenewalEmail`. It is set here and nowhere else, so every sweep a
* person clicks still quotes the amount. */
async scheduledSweep(): Promise<void> {
try {
await this.sweep();
await this.sweep(undefined, { automatic: true });
} catch (error) {
this.logger.error(
`Falló el barrido de renovaciones: ${(error as Error).message}`,
@@ -95,7 +99,10 @@ export class RenewalsService implements OnModuleInit {
}
}
async pending(days = 30) {
/** @param providerId Restrict to one aseguradora. The list has to agree
* with what a sweep would send, or the carrier-scoped barrido shows rows it
* will not mail. */
async pending(days = 30, providerId?: string) {
const today = dateInTimeZone(new Date());
const state = await this.prisma.scheduledJobState.findUnique({
where: { name: JOB_NAME },
@@ -111,6 +118,7 @@ export class RenewalsService implements OnModuleInit {
item,
today,
state?.lastSuccessfulAt ?? null,
providerId,
),
})),
);
@@ -122,8 +130,20 @@ export class RenewalsService implements OnModuleInit {
);
}
async sweep(userId?: string, flags: { debug?: boolean } = {}) {
/**
* @param flags.providerId Sweep only one aseguradora. GMX and ANA are worked
* as separate batches by the office, so mixing them in one run is what this
* exists to prevent.
* @param flags.automatic Set only by the scheduler. Drops the premium from
* the letter.
*/
async sweep(
userId?: string,
flags: { debug?: boolean; providerId?: string; automatic?: boolean } = {},
) {
const debug = !!flags.debug;
const providerId = flags.providerId?.trim() || undefined;
const includePremium = !flags.automatic;
const now = new Date();
const state = await this.acquireLock(now);
@@ -145,6 +165,7 @@ export class RenewalsService implements OnModuleInit {
cadence,
today,
state.lastSuccessfulAt,
providerId,
);
eligible += policies.length;
@@ -157,13 +178,17 @@ export class RenewalsService implements OnModuleInit {
await this.recordLog(policy, cadence.generation, "", {
status: "SKIPPED_NO_EMAIL",
debug,
includePremium,
});
skipped++;
continue;
}
try {
await this.deliver(policy, cadence.generation, to, userId, debug);
await this.deliver(policy, cadence.generation, to, userId, {
debug,
includePremium,
});
sent++;
} catch (error) {
failures.push({
@@ -182,11 +207,18 @@ export class RenewalsService implements OnModuleInit {
failed: failures.length,
failures,
debug,
providerId: providerId ?? null,
};
// A debug run must not advance `lastSuccessfulAt`: it wrote no
// RenewalNotice rows, so the days it "covered" are still owed, and
// narrowing tomorrow's window back to a single day would drop them.
await this.releaseLock(!debug && failures.length === 0 ? now : null);
//
// A carrier-scoped run must not advance it either, for the same reason
// one step out: it looked at the whole window but only mailed one
// aseguradora, so every other carrier's letters in those days would fall
// outside tomorrow's window and never be sent at all.
const complete = !debug && !providerId && failures.length === 0;
await this.releaseLock(complete ? now : null);
void this.audit.log(userId, "renewalNotice.sweep", result);
return result;
} catch (error) {
@@ -233,12 +265,14 @@ export class RenewalsService implements OnModuleInit {
throw new BadRequestException("El cliente no tiene correo registrado.");
}
// A person clicked this, so the premium stays in the letter — only the
// scheduler's unattended run omits it.
const { sentAt, providerMessageId, addressedTo } = await this.deliver(
policy,
generation,
to,
userId,
debug,
{ debug, includePremium: true },
);
return {
policyId,
@@ -270,10 +304,12 @@ export class RenewalsService implements OnModuleInit {
generation: number,
to: string,
userId?: string,
debug = false,
options: { debug?: boolean; includePremium?: boolean } = {},
) {
const debug = !!options.debug;
const includePremium = options.includePremium !== false;
const letter = toRenewalLetterRow(policy, generation);
const message = renderRenewalEmail(letter);
const message = renderRenewalEmail(letter, { includePremium });
const addressedTo = debug ? DEBUG_RECIPIENT : to;
let result: Awaited<ReturnType<MailService["send"]>>;
@@ -291,6 +327,7 @@ export class RenewalsService implements OnModuleInit {
status: "FAILED",
error: detail,
debug,
includePremium,
});
throw error;
}
@@ -324,6 +361,7 @@ export class RenewalsService implements OnModuleInit {
providerResponse: result.response || undefined,
sendDate: sentAt,
debug,
includePremium,
});
void this.audit.log(userId, "renewalNotice.send", {
policyId: policy.id,
@@ -355,10 +393,15 @@ export class RenewalsService implements OnModuleInit {
error?: string;
sendDate?: Date;
debug?: boolean;
/** Must match what `deliver` rendered, or `bodySnapshot` shows the
* office a letter the customer never received. */
includePremium?: boolean;
},
): Promise<void> {
const letter = toRenewalLetterRow(policy, generation);
const message = renderRenewalEmail(letter);
const message = renderRenewalEmail(letter, {
includePremium: outcome.includePremium,
});
try {
await this.notificationLog.record({
notificationType: "RENEWAL_NOTICE",
@@ -391,11 +434,13 @@ export class RenewalsService implements OnModuleInit {
cadence: (typeof RENEWAL_CADENCE)[number],
today: Date,
lastSuccessfulAt: Date | null,
providerId?: string,
) {
const window = renewalWindow(today, cadence.offsetDays, lastSuccessfulAt);
return this.prisma.policy.findMany({
where: {
archivedAt: null,
...(providerId && { insuranceProviderId: providerId }),
policyTo: { gte: window.from, lte: window.to },
customer: {
archivedAt: null,
+118 -18
View File
@@ -15,6 +15,11 @@
*/
import { Prisma } from "@jorgecuadros/database";
import {
BALANCE_FORWARD_TYPE,
notCashJournal,
periodSourceTable,
} from "../billing/billing.service";
import {
intParam,
NOT_VOIDED,
@@ -751,13 +756,22 @@ const edoCuentaDatos: ReportDef = {
title: "Estado de cuenta",
description:
"Estado de cuenta de un cliente: saldos por moneda, desglose por " +
"ramo y concepto, y el historial completo de movimientos con saldo " +
"corrido. El reporte del cliente final.",
"ramo y concepto, y los movimientos del año en curso con saldo " +
"corrido, abriendo con el saldo anterior. El reporte del cliente final.",
domain: "estado-cuenta",
legacyName: "EDO CUENTA DATOS",
format: "statement",
params: [
{ key: "customerId", label: "Cliente", kind: "customer-picker" },
// Which period to print. Blank means the year in progress; an earlier year
// prints from its imported archive, the same source the on-screen
// statement reads.
{
key: "year",
label: "Periodo (año)",
kind: "number",
placeholder: "año en curso",
},
],
columns: [
// Statement rows carry synthetic `__kind` discriminators instead of
@@ -789,22 +803,63 @@ const edoCuentaDatos: ReportDef = {
});
if (!customer) return { rows: [], subtitle: "Cliente no encontrado" };
// Reuse the same NOT_VOIDED + STATEMENT_EXCLUDED_SOURCE_TABLES filter
// as BillingService.statement so the numbers match what the customer
// already sees in /estado-cuenta/[id].
// The source-table exclusion, the balance floor and the year scope below
// are BillingService.statement's, because this report and
// /estado-cuenta/[id] are the same statement — one printable, one on
// screen — and a customer holding both must not read two balances.
const floor = await prisma.transaction.findFirst({
where: {
customerId,
voidedAt: null,
type: { nameEn: BALANCE_FORWARD_TYPE },
},
orderBy: { transactionDate: "desc" },
select: { transactionDate: true },
});
// Which period to print. An earlier year comes from its imported archive,
// tagged rather than dated, exactly as the on-screen statement reads it.
const thisYear = new Date().getUTCFullYear();
const askedYear = Number(p.year);
const requestedYear =
Number.isInteger(askedYear) && askedYear > 0 ? askedYear : thisYear;
const isArchive = requestedYear !== thisYear;
const rows = await prisma.transaction.findMany({
where: {
customerId,
voidedAt: null,
legacySourceTable: {
notIn: [
"EFECTIVO",
"EFECTIVO_BACKUP",
"EFECTIVO FM3",
"CHEQUE FM3",
"IVA 2015",
...(isArchive
? // The archive is one period's ledger already, so the tag is the
// whole filter and the balance floor must not apply — the floor
// hides exactly the history this period is asking for.
{ legacySourceTable: periodSourceTable(requestedYear) }
: {
...(floor ? { transactionDate: { gte: floor.transactionDate } } : {}),
// Archive rows count as history below the year start (that is
// what `opening` is for, and for a customer floored by an archive
// it is the only carry there is) and are dropped at or above it.
// Same rule as the on-screen twin — see BillingService.statement.
AND: [
{
OR: [
{ legacySourceTable: null },
{ legacySourceTable: { not: { startsWith: "datos2@" } } },
{
transactionDate: {
lt: new Date(Date.UTC(requestedYear, 0, 1)),
},
},
],
},
// The cash receipt book, which the ledger already carries as
// its own `C<folio>` postings. Taken from the shared helper
// rather than restated, so the printed statement and the screen
// cannot drift apart — and so this keeps the database
// qualifier that spares the insurance line's own EFECTIVO.
notCashJournal(),
],
}),
},
orderBy: [{ transactionDate: "asc" }, { id: "asc" }],
select: {
@@ -822,12 +877,32 @@ const edoCuentaDatos: ReportDef = {
},
});
// Compute running balance per currency, then return newest-first.
// Scoped to the calendar year and listed oldest-first, the way the legacy
// EDO CUENTA sheet reads. Rows from earlier years still move the running
// balance — they are folded into `opening` and printed as a single "saldo
// anterior" line, which is what a BALANCE FORWARD row is.
// An archive needs no fold: it *is* the period, and its own Jan-1 BALANCE
// FORWARD row is the carry, printed like legacy printed it.
const yearStart = isArchive
? new Date(0)
: new Date(Date.UTC(requestedYear, 0, 1));
const year = requestedYear;
const running = new Map<string, Prisma.Decimal>();
const movements = rows.map((r) => {
const opening = new Map<string, Prisma.Decimal>();
const visible: typeof rows = [];
const movements = rows.flatMap((r) => {
const prev = running.get(r.currency) ?? new Prisma.Decimal(0);
const next = prev.plus(r.amount);
running.set(r.currency, next);
if (r.transactionDate < yearStart) {
opening.set(r.currency, next);
return [];
}
visible.push(r);
return {
date: r.transactionDate.toISOString().slice(0, 10),
domain: r.domain,
@@ -840,14 +915,38 @@ const edoCuentaDatos: ReportDef = {
balanceAfter: next.toFixed(2),
};
});
movements.reverse();
// Per-currency summary + per-domain breakdown.
// The carried balance, printed as the statement's first line — same shape
// as a movement row so it needs nothing special from the renderer.
const carried = [...opening.entries()]
.filter(([, amount]) => !amount.isZero())
.map(([currency, amount]) => ({
date: yearStart.toISOString().slice(0, 10),
domain: "UTILITY",
currency,
reference: "",
period: `Al cierre de ${year - 1}`,
checkNumber: "",
concept: "SALDO ANTERIOR",
amount: amount.toFixed(2),
balanceAfter: amount.toFixed(2),
}));
// Per-currency summary, seeded with the carried balance so it reconciles
// against the last running balance printed below.
const perCurrency = new Map<
string,
{ currency: string; charges: Prisma.Decimal; credits: Prisma.Decimal; count: number }
>();
for (const r of rows) {
for (const [currency, amount] of opening) {
perCurrency.set(currency, {
currency,
charges: amount.lessThan(0) ? amount : new Prisma.Decimal(0),
credits: amount.lessThan(0) ? new Prisma.Decimal(0) : amount,
count: 0,
});
}
for (const r of visible) {
const c =
perCurrency.get(r.currency) ??
{
@@ -881,9 +980,10 @@ const edoCuentaDatos: ReportDef = {
count: c.count,
})),
{ __kind: "movements-header" },
...carried,
...movements,
],
subtitle: `${nameOf(customer)} · ${rows.length} movimientos`,
subtitle: `${nameOf(customer)} · ${year} · ${visible.length} movimientos`,
};
},
};
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@jorgecuadros/web",
"version": "1.0.22",
"version": "1.0.26",
"private": true,
"scripts": {
"dev": "next dev -p 4500",
+4
View File
@@ -18,6 +18,10 @@ const TYPE: ChildConfig = {
fields: [
{ key: "name", label: "Nombre" },
{ key: "shortDescription", label: "Descripción" },
// The rate is stored as a fraction, not a percentage, and the label has to
// say so: 8 typed here would tax a $600 premium $4,800. The API rejects
// anything above 1 rather than trusting the label alone.
{ key: "taxRate", label: "IVA (0.08 = 8%)", type: "number", step: "0.0001" },
],
};
const ADJUSTER: ChildConfig = {
+8 -5
View File
@@ -128,6 +128,7 @@ function Detail({ id }: { id: string }) {
customerId={data.id}
summary={data.transactionSummary}
transactions={data.transactions}
year={data.transactionYear}
/>
<DocumentosSection data={data} />
</div>
@@ -746,16 +747,18 @@ function EstadoCuentaSection({
customerId,
summary,
transactions,
year,
}: {
customerId: string;
summary: TransactionSummaryRow[];
transactions: Transaction[];
year: number;
}) {
return (
<section className="section">
<SectionHead
rule="cuenta"
title="Estado de cuenta"
title={`Estado de cuenta ${year}`}
count={transactions.length}
countSuffix="movimientos"
/>
@@ -782,7 +785,7 @@ function EstadoCuentaSection({
<div className="card">
{transactions.length === 0 ? (
<div className="empty-inline">Sin movimientos registrados.</div>
<div className="empty-inline">Sin movimientos en {year}.</div>
) : (
<div className="tx-scroll">
<table className="tx-table">
@@ -804,11 +807,11 @@ function EstadoCuentaSection({
</table>
</div>
)}
{transactions.length >= 100 && (
<div className="section-note" style={{ padding: "0 16px 14px" }}>
Mostrando los 100 movimientos más recientes.
Movimientos de {year}, del más antiguo al más reciente. Los saldos de
arriba son el saldo actual por línea de negocio los mismos del
estado de cuenta, no la suma del año.
</div>
)}
</div>
{transactions.length > 0 && (
<p className="section-note">
+100 -10
View File
@@ -36,10 +36,14 @@ import type {
* charge and an insurance payment finally sit on the same page, under the same
* person, with a running balance.
*
* The running balance is per currency (the API accumulates it chronologically
* before handing the list back newest-first), so the movement table is scoped
* to one currency at a time — a column that alternated between pesos and
* dollars would be a meaningless number.
* The running balance is per currency, so the movement table is scoped to one
* currency at a time — a column that alternated between pesos and dollars would
* be a meaningless number.
*
* Like the legacy EDO CUENTA report, the table covers one calendar year and runs
* oldest-first, opening on the balance carried in from before it. The period
* selector switches years; earlier ones are served from the imported archive of
* that year, which is how legacy kept them — one table per closed year.
*/
export default function EstadoCuentaDetailPage({
params,
@@ -65,19 +69,27 @@ function StatementView({ id }: { id: string }) {
const [currency, setCurrency] = useState<LedgerCurrency | null>(null);
const [domain, setDomain] = useState<TransactionDomain | "">("");
/** null = the current period; the API decides what that is. */
const [year, setYear] = useState<number | null>(null);
function reload() {
let alive = true;
setLoading(true);
setError(null);
getStatement(id)
getStatement(id, year ?? undefined)
.then((d) => {
if (!alive) return;
setData(d);
// Default to the currency the customer actually moves the most in;
// preserve a previously-chosen currency across reloads.
// preserve a previously-chosen currency across reloads — but only if
// the loaded period still has it. Switching to a year the customer
// never moved dollars in would otherwise leave the picker on USD with
// no matching option, showing an empty table for a year that has rows.
const busiest = [...d.summary].sort((a, b) => b.count - a.count)[0];
setCurrency((prev) => prev ?? busiest?.currency ?? "MXN");
const fallback = busiest?.currency ?? "MXN";
setCurrency((prev) =>
prev && d.summary.some((s) => s.currency === prev) ? prev : fallback,
);
setLoading(false);
})
.catch((e) => {
@@ -99,7 +111,7 @@ function StatementView({ id }: { id: string }) {
getBillingFacets().then(setFacets).catch(() => setFacets(null));
return cleanup;
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [id]);
}, [id, year]);
const movements = useMemo(() => {
if (!data || !currency) return [];
@@ -194,7 +206,7 @@ function StatementView({ id }: { id: string }) {
<section className="section">
<SectionHead
rule="cuenta"
title="Movimientos"
title={`Movimientos ${data.year}`}
count={movements.length}
countSuffix={movements.length === 1 ? "movimiento" : "movimientos"}
right={
@@ -227,6 +239,26 @@ function StatementView({ id }: { id: string }) {
)}
<div className="filter-row">
{/* Only the periods this customer has. A year with no archive would
render an empty table that reads as "no hubo movimientos" when the
truth is that the year was never imported. */}
{data.availableYears.length > 1 && (
<label className="filter-field">
<span className="filter-label">Periodo</span>
<select
className="input select"
value={data.year}
onChange={(e) => setYear(Number(e.target.value))}
>
{data.availableYears.map((y) => (
<option key={y} value={y}>
{y}
{y === data.availableYears[0] ? " (en curso)" : ""}
</option>
))}
</select>
</label>
)}
<label className="filter-field">
<span className="filter-label">Moneda</span>
<select
@@ -260,7 +292,7 @@ function StatementView({ id }: { id: string }) {
<div className="card">
{movements.length === 0 ? (
<div className="empty-inline">
Sin movimientos en {currency}
Sin movimientos de {data.year} en {currency}
{domain ? ` para ${domainLabel(domain)}` : ""}.
</div>
) : (
@@ -282,6 +314,26 @@ function StatementView({ id }: { id: string }) {
</tr>
</thead>
<tbody>
{/*
The carried balance, shown the way the legacy report shows
it: a BALANCE FORWARD line above the year's movements. It
only appears when there is something to carry — when the
customer's opening-balance row is itself dated inside this
year (the usual case) it is listed as an ordinary movement
and this row is zero, so it is left out.
Suppressed under a business-line filter: the carried balance
is the customer's, across both lines, and printing it above
one line's rows would read as that line's opening balance.
*/}
{!domain && Number(active?.opening ?? 0) !== 0 && (
<OpeningRow
opening={active!.opening}
currency={currency}
year={data.year}
canVoid={canVoid}
/>
)}
{movements.map((m) => (
<StatementRow
key={m.id}
@@ -481,6 +533,44 @@ function ConceptosSection({
);
}
/** The balance carried into the statement year — legacy's BALANCE FORWARD. */
function OpeningRow({
opening,
currency,
year,
canVoid,
}: {
opening: string;
currency: LedgerCurrency;
year: number;
canVoid: boolean;
}) {
return (
<tr>
<td className="mono" style={{ whiteSpace: "nowrap" }}>
{formatDate(`${year}-01-01T00:00:00.000Z`)}
</td>
<td className="tx-domain-cell">Ambas líneas</td>
<td>
Saldo anterior
<div className="tx-concept">Al cierre de {year - 1}</div>
</td>
<td className="tx-ref"></td>
<td className="num">
<span className={`tx-amount ${Number(opening) < 0 ? "neg" : "pos"}`}>
{formatMoney(opening, currency)}
</span>
</td>
<td className="num">
<span className={`bal-running ${balanceTone(opening)}`}>
{formatMoney(opening, currency)}
</span>
</td>
{canVoid && <td />}
</tr>
);
}
function StatementRow({
m,
canVoid,
+15
View File
@@ -926,6 +926,15 @@ button {
color: var(--ink-soft);
margin-bottom: 0.4375rem;
}
/* Sub-label under an input: the computed figure behind an override field, or
why a field is disabled. Quiet enough not to compete with .field-label. */
.field-hint {
display: block;
font-size: 0.75rem;
line-height: 1.35;
color: var(--muted-2);
margin-top: 0.3125rem;
}
.input {
width: 100%;
font-family: inherit;
@@ -940,6 +949,12 @@ button {
.input::placeholder {
color: var(--muted-2);
}
.input:disabled,
.select:disabled {
background: var(--surface-2, var(--surface));
color: var(--muted-2);
cursor: not-allowed;
}
.input:focus {
outline: none;
border-color: var(--brand-600);
+58 -4
View File
@@ -68,6 +68,7 @@ function Operaciones() {
const [starting, setStarting] = useState(false);
const fileInputs = useRef<Record<string, HTMLInputElement | null>>({});
const periodInput = useRef<HTMLInputElement | null>(null);
const refreshLists = useCallback(() => {
listIngest().then(setIngest).catch(() => setIngest([]));
@@ -143,6 +144,20 @@ function Operaciones() {
}
}
/** Upload a prior-period archive under its own filename. */
async function handleUploadPeriod(file: File | undefined) {
if (!file) return;
if (!/^\d{4}\.accdb$/i.test(file.name)) {
setError(
`"${file.name}" no es un archivo de periodo. Debe llamarse AAAA.accdb, por ejemplo 2025.accdb.`,
);
if (periodInput.current) periodInput.current.value = "";
return;
}
await handleUpload(file.name, file);
if (periodInput.current) periodInput.current.value = "";
}
async function handleDeleteIngest(name: string) {
setError(null);
try {
@@ -197,7 +212,11 @@ function Operaciones() {
else await start("RESTORE", c.file);
}
const ingestReady = (ingest ?? []).every((f) => f.present);
// Only the four fixed Access sources gate a run. Period archives are
// optional extras — having none simply means no prior years are available.
const ingestReady = (ingest ?? [])
.filter((f) => f.periodYear === null)
.every((f) => f.present);
return (
<>
@@ -244,8 +263,9 @@ function Operaciones() {
<div className="card" style={{ padding: 20, marginBottom: 20 }}>
<h2 className="section-title">Carpeta de ingesta</h2>
<p className="inline-form-note">
Los cuatro archivos originales de Access. La reimportación y la
sincronización leen de aquí. Tamaño máximo por archivo: {formatBytes(INGEST_MAX_BYTES)}.
Los cuatro archivos originales de Access, más los archivos de periodos
anteriores. La reimportación y la sincronización leen de aquí. Tamaño
máximo por archivo: {formatBytes(INGEST_MAX_BYTES)}.
</p>
<div className="tx-scroll">
<table className="tx-table">
@@ -262,7 +282,14 @@ function Operaciones() {
{(ingest ?? []).map((f) => (
<Fragment key={f.name}>
<tr>
<td className="mono">{f.name}</td>
<td className="mono">
{f.name}
{f.periodYear !== null && (
<span className="badge" style={{ marginLeft: 8 }}>
periodo {f.periodYear}
</span>
)}
</td>
<td>
<span className={`badge ${f.present ? "badge-positive" : "badge-negative"}`}>
{f.present ? "Presente" : "Falta"}
@@ -313,6 +340,33 @@ function Operaciones() {
</tbody>
</table>
</div>
{/* A period archive that has never been uploaded has no row to click,
so it needs its own entry point. The file names itself: the archive
IS `2025.accdb`, and that name is what declares the period, so the
control reads it off the chosen file rather than asking twice. */}
<div className="row-actions" style={{ marginTop: 16 }}>
<input
ref={periodInput}
type="file"
accept=".accdb"
style={{ display: "none" }}
onChange={(e) => handleUploadPeriod(e.target.files?.[0])}
/>
<button
className="btn btn-outline"
type="button"
disabled={uploading !== null}
onClick={() => periodInput.current?.click()}
>
Agregar periodo anterior
</button>
<span className="inline-form-note">
Un archivo de Access por año cerrado, nombrado con su periodo:{" "}
<span className="mono">2025.accdb</span>. Aporta el año anterior al
estado de cuenta; no altera el saldo actual.
</span>
</div>
</div>
{/* Operations */}
+45 -4
View File
@@ -26,7 +26,13 @@ import {
premiumHeadline,
SIN_NOMBRE,
} from "@/lib/labels";
import type { AdjusterRow, Installment, PolicyDetail } from "@/lib/types";
import {
PAYMENT_FREQUENCY_LABELS,
type AdjusterRow,
type Installment,
type PolicyDetail,
} from "@/lib/types";
import { formatRate } from "@/lib/premium";
export default function PolizaDetailPage({
params,
@@ -186,6 +192,10 @@ function ChildrenEditor({
const INSTALLMENTS: ChildConfig = {
apiKind: "installments",
title: "Pagos",
// A policy paid in several exhibiciones prices each payment on its own, so
// the whole premium breakdown repeats per row — that is the two-row money
// block on the Access form. `amount` stays what was actually collected and
// is deliberately separate from `total`; they differ by rounding.
fields: [
{ key: "sequence", label: "Sec.", type: "number" },
{ key: "amount", label: "Monto", type: "number" },
@@ -195,6 +205,12 @@ function ChildrenEditor({
{ key: "paidDate", label: "Pagado", type: "date" },
{ key: "checkNumber", label: "Cheque" },
{ key: "isCash", label: "Efectivo", type: "checkbox" },
{ key: "netPremium", label: "Prima neta", type: "number" },
{ key: "surcharge", label: "Recargo", type: "number" },
{ key: "policyFee", label: "Derecho", type: "number" },
{ key: "tax", label: "IVA", type: "number" },
{ key: "total", label: "Prima total", type: "number" },
{ key: "commission", label: "Comisión", type: "number" },
],
};
const VEHICLES: ChildConfig = {
@@ -412,15 +428,40 @@ function CondicionesSection({ data }: { data: PolicyDetail }) {
data.coveragePeriodDays ? `${data.coveragePeriodDays} días` : null
}
/>
<KV
label="Forma de pago"
value={
data.paymentFrequency
? PAYMENT_FREQUENCY_LABELS[data.paymentFrequency]
: null
}
/>
<KV label="Prima neta" value={formatMoney(data.netPremium, cur)} />
{/* Only ever set on a policy paid in installments, so showing an
empty row on the other 98% would be noise. */}
{data.surcharge != null && Number(data.surcharge) !== 0 && (
<KV label="Recargo" value={formatMoney(data.surcharge, cur)} />
)}
<KV label="Derecho de póliza" value={formatMoney(data.policyFee, cur)} />
<KV label="Comisión" value={formatMoney(data.commission, cur)} />
<KV label="Honorarios" value={formatMoney(data.brokerFee, cur)} />
{/* Access never stored IVA — it was a calculated control on the form
— so every migrated policy reads null here until it is edited. */}
{data.tax != null && (
<KV
label={
data.taxRate != null
? `IVA (${formatRate(Number(data.taxRate))})`
: "IVA"
}
value={formatMoney(data.tax, cur)}
/>
)}
{/* The legacy `total` is 0 or null on all but 2 of 2378 policies —
only show it when it actually carries a figure. */}
{data.total != null && Number(data.total) > 0 && (
<KV label="Total" value={formatMoney(data.total, cur)} />
<KV label="Prima total" value={formatMoney(data.total, cur)} />
)}
<KV label="Comisión" value={formatMoney(data.commission, cur)} />
<KV label="Honorarios" value={formatMoney(data.brokerFee, cur)} />
<KV
label="Liquidación"
value={
+4 -1
View File
@@ -9,6 +9,9 @@ export type FieldDef = {
type?: "text" | "number" | "date" | "checkbox" | "select";
options?: { value: string; label: string }[];
width?: number;
/** Numeric granularity. Defaults to money (0.01); a tax rate stored as a
* fraction needs finer, or the browser rejects 0.0825 as off-step. */
step?: string;
};
export type ChildConfig = {
@@ -158,7 +161,7 @@ export function ChildCollection({
<input
className="input"
type={f.type === "number" ? "number" : f.type === "date" ? "date" : "text"}
step={f.type === "number" ? "0.01" : undefined}
step={f.type === "number" ? f.step ?? "0.01" : undefined}
value={String(values[f.key] ?? "")}
onChange={(e) => setValues({ ...values, [f.key]: e.target.value })}
/>
@@ -4,7 +4,13 @@ import { useCallback, useEffect, useState } from "react";
import { useCan } from "@/lib/abilities";
import { formatDate, formatMoney } from "@/lib/labels";
import { NotificationLogPanel } from "@/components/NotificationLogPanel";
import { apiFetch, POLIZAS_LOG_SCOPE, type NotificationFlags } from "@/lib/api";
import {
apiFetch,
getLookups,
POLIZAS_LOG_SCOPE,
type NotificationFlags,
} from "@/lib/api";
import type { ProviderRow } from "@/lib/types";
/**
* Renewal notices — the "Pólizas" half of /notificaciones. Shows which
@@ -22,6 +28,12 @@ import { apiFetch, POLIZAS_LOG_SCOPE, type NotificationFlags } from "@/lib/api";
* thing here as it does for servicios: the mail is diverted to the override
* inbox. It additionally does NOT mark the notice as sent, so a test send
* leaves the row exactly where it was — pending.
*
* The barrido manual is scoped by aseguradora because the office works GMX and
* ANA as separate batches. The selection filters the pending list too, so what
* is on screen is exactly what "Ejecutar barrido" will mail. A carrier-scoped
* run deliberately does not advance the sweep's catch-up window — it only
* covered one carrier — so the other carriers' letters stay pending.
*/
export interface RenewalLetter {
@@ -46,6 +58,8 @@ export interface RenewalSweepResult {
failed: number;
failures: { policyId: string; generation: number; error: string }[];
debug: boolean;
/** Echoed back so the confirmation says which carrier actually ran. */
providerId: string | null;
}
export interface RenewalSendResult {
@@ -68,6 +82,10 @@ export function NotificacionesPolizas({ flags }: { flags: NotificationFlags }) {
const allowed = useCan("renewal:send");
const debug = !!flags.debug;
const [days, setDays] = useState(30);
/** "" = ambas/todas. Holds an InsuranceProvider id, never a name — carriers
* are renamed in the lookups screen and the filter must survive that. */
const [providerId, setProviderId] = useState("");
const [providers, setProviders] = useState<ProviderRow[]>([]);
const [pending, setPending] = useState<RenewalLetter[] | null>(null);
const [pendingError, setPendingError] = useState<string | null>(null);
const [actionError, setActionError] = useState<string | null>(null);
@@ -81,8 +99,10 @@ export function NotificacionesPolizas({ flags }: { flags: NotificationFlags }) {
const refresh = useCallback(async () => {
setPendingError(null);
try {
const params = new URLSearchParams({ days: String(days) });
if (providerId) params.set("providerId", providerId);
const data = await apiFetch<RenewalLetter[]>(
`/renewals/pending?days=${days}`,
`/renewals/pending?${params.toString()}`,
);
setPending(data);
} catch (e) {
@@ -91,18 +111,44 @@ export function NotificacionesPolizas({ flags }: { flags: NotificationFlags }) {
);
setPending([]);
}
}, [days]);
}, [days, providerId]);
useEffect(() => {
if (allowed) refresh();
}, [allowed, refresh]);
// Carriers come from the same lookups the policy form uses, so a new
// aseguradora shows up here without a code change.
useEffect(() => {
if (!allowed) return;
let cancelled = false;
getLookups()
.then((data) => {
if (!cancelled) setProviders(data.providers);
})
.catch(() => {
// A failed lookup only costs the filter; the unfiltered sweep still
// works, so this must not blank the screen.
if (!cancelled) setProviders([]);
});
return () => {
cancelled = true;
};
}, [allowed]);
const providerLabel =
providers.find((item) => item.id === providerId)?.name ?? "todas las compañías";
async function handleSweep() {
// Only worth confirming when debug is off — that is the case where real
// customers receive mail. Mirrors "Ejecutar todos" on the servicios tab.
// The carrier is named in the prompt: running GMX when ANA was meant is
// exactly the mistake this filter exists to prevent, and it is not
// reversible once the mail is out.
if (!debug) {
const ok = window.confirm(
"debug está desactivado: los avisos irán a los correos reales de los clientes. ¿Ejecutar el barrido?",
`debug está desactivado: los avisos irán a los correos reales de los clientes. ` +
`¿Ejecutar el barrido de ${providerLabel}?`,
);
if (!ok) return;
}
@@ -112,10 +158,11 @@ export function NotificacionesPolizas({ flags }: { flags: NotificationFlags }) {
try {
const result = await apiFetch<RenewalSweepResult>("/renewals/sweep", {
method: "POST",
body: JSON.stringify({ debug }),
body: JSON.stringify({ debug, providerId: providerId || undefined }),
});
setNotice(
`Enviados ${result.sent} avisos (${result.failed} con error).` +
`Enviados ${result.sent} avisos de ${providerLabel} ` +
`(${result.failed} con error).` +
(result.debug
? " Modo debug: fueron al buzón de pruebas y siguen pendientes."
: ""),
@@ -199,7 +246,9 @@ export function NotificacionesPolizas({ flags }: { flags: NotificationFlags }) {
<h2 className="section-title">Barrido manual</h2>
<p className="muted small" style={{ marginTop: 4 }}>
Usa la fecha actual del servidor como referencia para seleccionar
avisos vencidos a 30 y 15 días, y vencidos hace 7 días.
avisos vencidos a 30 y 15 días, y vencidos hace 7 días. La
compañía elegida filtra también la lista de abajo: se envía
exactamente lo que está en pantalla.
</p>
</div>
<button
@@ -211,7 +260,11 @@ export function NotificacionesPolizas({ flags }: { flags: NotificationFlags }) {
{sweeping ? "Enviando…" : "Ejecutar barrido"}
</button>
</div>
<div className="field" style={{ maxWidth: 180, marginTop: 12, marginBottom: 0 }}>
<div
className="row-actions"
style={{ marginTop: 12, alignItems: "flex-end", gap: 16 }}
>
<div className="field" style={{ maxWidth: 180, marginBottom: 0 }}>
<span className="field-label">Ventana (días)</span>
<input
className="input"
@@ -224,6 +277,22 @@ export function NotificacionesPolizas({ flags }: { flags: NotificationFlags }) {
}
/>
</div>
<div className="field" style={{ maxWidth: 260, marginBottom: 0 }}>
<span className="field-label">Compañía</span>
<select
className="input"
value={providerId}
onChange={(e) => setProviderId(e.target.value)}
>
<option value="">Todas las compañías</option>
{providers.map((item) => (
<option key={item.id} value={item.id}>
{item.name}
</option>
))}
</select>
</div>
</div>
</section>
{pendingError && <div className="state-box state-error">{pendingError}</div>}
+119 -6
View File
@@ -4,12 +4,22 @@ import { useEffect, useState } from "react";
import { useRouter } from "next/navigation";
import { CustomerPicker } from "@/components/CustomerPicker";
import { createPolicy, getLookups, updatePolicy } from "@/lib/api";
import type {
Currency,
LookupsResponse,
PolicyDetail,
PolicyInput,
import {
PAYMENT_FREQUENCY_LABELS,
type Currency,
type LookupsResponse,
type PaymentFrequency,
type PolicyDetail,
type PolicyInput,
} from "@/lib/types";
import {
computeTax,
computeTotal,
formatRate,
resolveTaxRate,
surchargeApplies,
taxableBase,
} from "@/lib/premium";
function toDateInput(v: string | null | undefined): string {
if (!v) return "";
@@ -36,9 +46,16 @@ type V = {
policyFrom: string;
policyTo: string;
netPremium: string;
surcharge: string;
policyFee: string;
brokerFee: string;
commission: string;
/** Blank means "use the computed figure". Only ever holds a value once the
* operator overrides it, so a later change to prima neta keeps flowing
* through instead of being frozen by a value the form itself wrote. */
tax: string;
total: string;
paymentFrequency: PaymentFrequency | "";
currency: Currency;
liquidated: boolean;
liquidationNumber: string;
@@ -58,9 +75,13 @@ function initial(p?: PolicyDetail): V {
policyFrom: toDateInput(p?.policyFrom),
policyTo: toDateInput(p?.policyTo),
netPremium: p?.netPremium != null ? String(p.netPremium) : "",
surcharge: p?.surcharge != null ? String(p.surcharge) : "",
policyFee: p?.policyFee != null ? String(p.policyFee) : "",
brokerFee: p?.brokerFee != null ? String(p.brokerFee) : "",
commission: p?.commission != null ? String(p.commission) : "",
tax: p?.tax != null ? String(p.tax) : "",
total: p?.total != null ? String(p.total) : "",
paymentFrequency: p?.paymentFrequency ?? "",
currency: (p?.currency as Currency) ?? "MXN",
liquidated: p?.liquidated ?? false,
liquidationNumber: p?.liquidationNumber ?? "",
@@ -101,6 +122,27 @@ export function PolicyForm({
setV((p) => ({ ...p, [k]: val }));
}
// IVA and Total are the only two figures the form derives. Everything else,
// the recargo included, is keyed by hand — the carrier quotes the financing
// charge, we do not compute it.
const selectedType = lookups?.types.find((t) => t.id === v.policyTypeId);
const taxRate = resolveTaxRate(policy?.taxRate, selectedType?.taxRate);
const parts = {
netPremium: v.netPremium,
// A recargo on an annual policy is a data-entry mistake, so it is dropped
// from the arithmetic as well as disabled in the UI. Otherwise switching
// ANNUAL after typing one would leave it silently inflating the IVA.
surcharge: surchargeApplies(v.paymentFrequency || null) ? v.surcharge : "",
policyFee: v.policyFee,
};
const computedTax = computeTax(parts, taxRate);
const computedTotal = computeTotal(parts, taxRate);
// Blank field = take the computed figure. A typed one wins, so staff can key
// the carrier's rounding verbatim when it disagrees with ours by a centavo.
const effectiveTax = v.tax.trim() === "" ? computedTax : Number(v.tax);
const effectiveTotal = v.total.trim() === "" ? computedTotal : Number(v.total);
const showSurcharge = surchargeApplies(v.paymentFrequency || null);
async function submit(e: React.FormEvent) {
e.preventDefault();
if (!customerId) {
@@ -118,9 +160,17 @@ export function PolicyForm({
policyFrom: s(v.policyFrom),
policyTo: s(v.policyTo),
netPremium: numOrUndef(v.netPremium),
surcharge: showSurcharge ? numOrUndef(v.surcharge) : undefined,
policyFee: numOrUndef(v.policyFee),
brokerFee: numOrUndef(v.brokerFee),
commission: numOrUndef(v.commission),
// The derived figures are persisted, not recomputed on read: the printed
// policy is the record of truth and a later rate change must not silently
// restate what was issued. `taxRate` rides along for the same reason.
tax: Number.isFinite(effectiveTax) ? effectiveTax : undefined,
taxRate,
total: Number.isFinite(effectiveTotal) ? effectiveTotal : undefined,
paymentFrequency: v.paymentFrequency || undefined,
currency: v.currency,
liquidated: v.liquidated,
liquidationNumber: s(v.liquidationNumber),
@@ -208,7 +258,7 @@ export function PolicyForm({
</div>
<div className="card" style={{ padding: 20, marginBottom: 16 }}>
<h2 className="section-title" style={{ marginBottom: 14 }}>Vigencia y prima</h2>
<h2 className="section-title" style={{ marginBottom: 14 }}>Vigencia</h2>
<div className="form-grid">
<label className="field">
<span className="field-label">Emisión</span>
@@ -225,21 +275,84 @@ export function PolicyForm({
<input className="input" type="date" value={v.policyTo}
onChange={(e) => set("policyTo", e.target.value)} />
</label>
<label className="field">
<span className="field-label">Forma de pago</span>
<select className="select" value={v.paymentFrequency}
onChange={(e) =>
set("paymentFrequency", e.target.value as PaymentFrequency | "")
}>
<option value=""></option>
{(
Object.keys(PAYMENT_FREQUENCY_LABELS) as PaymentFrequency[]
).map((f) => (
<option key={f} value={f}>{PAYMENT_FREQUENCY_LABELS[f]}</option>
))}
</select>
</label>
</div>
</div>
<div className="card" style={{ padding: 20, marginBottom: 16 }}>
<h2 className="section-title" style={{ marginBottom: 4 }}>Primas</h2>
<p className="muted" style={{ fontSize: 12, marginBottom: 14 }}>
IVA y prima total se calculan solos sobre (prima neta + recargo +
derecho de póliza). Puede sobrescribirlos si la póliza impresa
redondea distinto.
</p>
<div className="form-grid">
<label className="field">
<span className="field-label">Prima neta</span>
<input className="input" type="number" step="0.01" value={v.netPremium}
onChange={(e) => set("netPremium", e.target.value)} />
</label>
<label className="field">
<span className="field-label">Recargo</span>
<input className="input" type="number" step="0.01" value={v.surcharge}
disabled={!showSurcharge}
onChange={(e) => set("surcharge", e.target.value)} />
<span className="field-hint">
{showSurcharge
? "Lo cotiza la aseguradora — se captura a mano."
: "No aplica en pago anual ni de contado."}
</span>
</label>
<label className="field">
<span className="field-label">Derecho de póliza</span>
<input className="input" type="number" step="0.01" value={v.policyFee}
onChange={(e) => set("policyFee", e.target.value)} />
</label>
<label className="field">
<span className="field-label">IVA ({formatRate(taxRate)})</span>
<input className="input" type="number" step="0.01"
placeholder={computedTax.toFixed(2)} value={v.tax}
onChange={(e) => set("tax", e.target.value)} />
<span className="field-hint">
Calculado: {computedTax.toFixed(2)} sobre base{" "}
{taxableBase(parts).toFixed(2)}
{selectedType?.taxRate == null &&
policy?.taxRate == null &&
" · tasa por omisión, configúrela en Catálogos"}
</span>
</label>
<label className="field">
<span className="field-label">Prima total</span>
<input className="input" type="number" step="0.01"
placeholder={computedTotal.toFixed(2)} value={v.total}
onChange={(e) => set("total", e.target.value)} />
<span className="field-hint">
Calculado: {computedTotal.toFixed(2)}
</span>
</label>
<label className="field">
<span className="field-label">Comisión</span>
<input className="input" type="number" step="0.01" value={v.commission}
onChange={(e) => set("commission", e.target.value)} />
</label>
<label className="field">
<span className="field-label">Honorarios</span>
<input className="input" type="number" step="0.01" value={v.brokerFee}
onChange={(e) => set("brokerFee", e.target.value)} />
</label>
</div>
</div>
+25 -1
View File
@@ -276,6 +276,8 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
policyDate: doc.extractedPolicyDate?.slice(0, 10) ?? "",
currency: doc.extractedCurrency ?? "USD",
netPremium: doc.extractedNetPremium ?? "",
policyFee: doc.extractedPolicyFee ?? "",
tax: doc.extractedTax ?? "",
total: doc.extractedTotal ?? "",
premiumPayment: doc.extractedPremiumPayment ?? "",
coveragePeriodDays: doc.extractedCoveragePeriodDays?.toString() ?? "",
@@ -314,6 +316,8 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
policyDate: v.policyDate || undefined,
currency,
netPremium: numOrUndef(v.netPremium),
policyFee: numOrUndef(v.policyFee),
tax: numOrUndef(v.tax),
total: numOrUndef(v.total),
premiumPayment: trimOrUndef(v.premiumPayment),
coveragePeriodDays: numOrUndef(v.coveragePeriodDays),
@@ -336,6 +340,8 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
policyDate: reviewInput.policyDate,
currency: (currency as "MXN" | "USD" | "EUR" | undefined) ?? undefined,
netPremium: reviewInput.netPremium,
policyFee: reviewInput.policyFee,
tax: reviewInput.tax,
total: reviewInput.total,
premiumPayment: reviewInput.premiumPayment,
coveragePeriodDays: reviewInput.coveragePeriodDays,
@@ -501,7 +507,25 @@ function DocumentRow({ doc, customerIndex, canReview, onSave, onReject }: Docume
onChange={(e) => set("netPremium", e.target.value)}
/>
</Field>
<Field label="Total">
<Field label="Derecho de póliza">
<input
className="input"
type="number"
step="0.01"
value={v.policyFee}
onChange={(e) => set("policyFee", e.target.value)}
/>
</Field>
<Field label="IVA">
<input
className="input"
type="number"
step="0.01"
value={v.tax}
onChange={(e) => set("tax", e.target.value)}
/>
</Field>
<Field label="Prima total">
<input
className="input"
type="number"
+7 -2
View File
@@ -615,8 +615,13 @@ export function getBillingFacets(): Promise<BillingFacets> {
return apiFetch<BillingFacets>("/billing/facets");
}
export function getStatement(customerId: string): Promise<Statement> {
return apiFetch<Statement>(`/billing/customers/${customerId}`);
/** `year` omitted reads the current period; earlier years come from an archive. */
export function getStatement(
customerId: string,
year?: number,
): Promise<Statement> {
const q = year === undefined ? "" : `?year=${year}`;
return apiFetch<Statement>(`/billing/customers/${customerId}${q}`);
}
/** Append a new ledger movement. Booked movements are never edited — fix
+79
View File
@@ -0,0 +1,79 @@
import type { PaymentFrequency } from "./types";
/**
* Client-side twin of apps/api/src/policies/premium.ts. Duplicated rather than
* shared because the API and the web app do not share a package today, and
* both need it: the form computes IVA and Total live as the operator types,
* the API stores what it is sent.
*
* base = prima neta + recargo + derecho de póliza
* IVA = round(base * tasa)
* Total = base + IVA
*
* The recargo is inside the taxable base — that is what reconciles the Access
* books (policy 7006785 prints IVA 52.03 on 610.86 + 8.55 + 31.00; leaving the
* recargo out gives 51.35, which matches nothing on the page).
*/
/** Applied when neither the policy nor its type carries a rate. The single
* row both legacy IMPUESTOS tables held. */
export const DEFAULT_TAX_RATE = 0.08;
/** Paying in more than one exhibición is what earns a recargo. A null
* frequency (every migrated policy) is treated as "unknown, allow it": the
* legacy recargo figures are real and hiding the field would hide them. */
export function surchargeApplies(
frequency: PaymentFrequency | null | undefined,
): boolean {
return frequency !== "ANNUAL" && frequency !== "SINGLE";
}
export function num(v: string | number | null | undefined): number {
if (v === null || v === undefined || v === "") return 0;
const n = typeof v === "number" ? v : Number(String(v).trim());
return Number.isFinite(n) ? n : 0;
}
/** Half-up to cents, matching how the printed policy rounds. */
export function round2(n: number): number {
return Math.round((n + Number.EPSILON) * 100) / 100;
}
export interface PremiumParts {
netPremium: string | number | null | undefined;
surcharge: string | number | null | undefined;
policyFee: string | number | null | undefined;
}
export function taxableBase(p: PremiumParts): number {
return round2(num(p.netPremium) + num(p.surcharge) + num(p.policyFee));
}
export function computeTax(p: PremiumParts, rate: number): number {
return round2(taxableBase(p) * rate);
}
export function computeTotal(p: PremiumParts, rate: number): number {
return round2(taxableBase(p) + computeTax(p, rate));
}
/** Rate ladder: what the policy was issued at, else its line of business, else
* the default. Keeps an old policy reading back at its original rate after
* somebody edits the catalog. */
export function resolveTaxRate(
policyRate: string | number | null | undefined,
policyTypeRate: string | number | null | undefined,
): number {
for (const candidate of [policyRate, policyTypeRate]) {
if (candidate === null || candidate === undefined || candidate === "") continue;
const n = Number(candidate);
if (Number.isFinite(n) && n >= 0) return n;
}
return DEFAULT_TAX_RATE;
}
/** 0.08 -> "8%". Rates are stored as fractions but read as percentages. */
export function formatRate(rate: number): string {
const pct = round2(rate * 100);
return `${pct}%`;
}
+69
View File
@@ -3,6 +3,24 @@
export type Currency = "USD" | "MXN";
/** How the premium is split into payments. Anything other than ANNUAL/SINGLE
* is what earns a recargo. Null on every migrated policy — the original ETL
* dropped Access's FORMA PAGO column entirely. */
export type PaymentFrequency =
| "ANNUAL"
| "SEMIANNUAL"
| "QUARTERLY"
| "MONTHLY"
| "SINGLE";
export const PAYMENT_FREQUENCY_LABELS: Record<PaymentFrequency, string> = {
ANNUAL: "Anual",
SEMIANNUAL: "Semestral",
QUARTERLY: "Trimestral",
MONTHLY: "Mensual",
SINGLE: "Contado",
};
export type Role = "ADMIN" | "MANAGER" | "STAFF" | "VIEWER";
export type Ability =
@@ -174,6 +192,8 @@ export interface IngestFile {
present: boolean;
size: number | null;
modifiedAt: string | null;
/** Year of a prior-period archive (`2025.accdb`); null on the four fixed sources. */
periodYear: number | null;
}
export interface BackupFile {
@@ -289,6 +309,16 @@ export interface Installment {
paidDate: string | null;
checkNumber: string | null;
isCash: boolean;
/** Per-payment premium breakdown — a policy paid in several exhibiciones
* prices each one separately. `amount` is what was actually collected and
* can differ from `total` by rounding; it is not derived from these. */
netPremium: string | null;
surcharge: string | null;
policyFee: string | null;
tax: string | null;
taxRate: string | null;
total: string | null;
commission: string | null;
}
export interface Vehicle {
@@ -398,10 +428,14 @@ export interface PolicyInput {
policyTo?: string;
coveragePeriodDays?: number;
netPremium?: number;
surcharge?: number;
policyFee?: number;
brokerFee?: number;
commission?: number;
tax?: number;
taxRate?: number;
total?: number;
paymentFrequency?: PaymentFrequency;
currency?: Currency;
observations?: string;
notes?: string;
@@ -419,6 +453,13 @@ export interface InstallmentInput {
paidDate?: string;
checkNumber?: string;
isCash?: boolean;
netPremium?: number;
surcharge?: number;
policyFee?: number;
tax?: number;
taxRate?: number;
total?: number;
commission?: number;
}
export interface VehicleInput {
make?: string;
@@ -469,6 +510,10 @@ export interface PolicyTypeRow {
id: string;
name: string;
shortDescription: string | null;
/** IVA fraction for this line of business, 0.08 = 8%. Null means "not
* configured" and the form falls back to DEFAULT_TAX_RATE — it does NOT
* mean the line is untaxed. Serialized as a decimal string by Prisma. */
taxRate: string | null;
_count?: { policies: number };
}
export interface AdjusterRow {
@@ -567,10 +612,14 @@ export interface PolicyDetail {
policyTo: string | null;
coveragePeriodDays: number | null;
netPremium: string | null;
surcharge: string | null;
policyFee: string | null;
brokerFee: string | null;
commission: string | null;
tax: string | null;
taxRate: string | null;
total: string | null;
paymentFrequency: PaymentFrequency | null;
currency: string | null;
observations: string | null;
notes: string | null;
@@ -994,6 +1043,8 @@ export interface BillingFacets {
export interface StatementSummary {
currency: LedgerCurrency;
/** Balance carried in from before the statement year — legacy's BALANCE FORWARD. */
opening: string;
charges: string;
credits: string;
balance: string;
@@ -1007,6 +1058,7 @@ export interface StatementSummary {
export interface StatementDomainRow {
domain: TransactionDomain;
currency: LedgerCurrency;
opening: string;
charges: string;
credits: string;
balance: string;
@@ -1042,6 +1094,15 @@ export interface Statement {
propertyCount: number;
policyCount: number;
};
/** Calendar year the statement covers; movements are scoped to it. */
year: number;
/**
* Periods this customer actually has, newest first. The current year is
* always present; each earlier year comes from an imported archive. Offering
* anything outside this list would render an empty statement that reads as
* "no activity" rather than "not imported".
*/
availableYears: number[];
summary: StatementSummary[];
byDomain: StatementDomainRow[];
byType: StatementTypeRow[];
@@ -1077,6 +1138,8 @@ export interface CustomerDetail {
properties: Property[];
policies: Policy[];
transactions: Transaction[];
/** Calendar year `transactions` covers. */
transactionYear: number;
transactionSummary: TransactionSummaryRow[];
}
@@ -1508,6 +1571,10 @@ export interface PolicyOcrDocument {
extractedNetPremium: string | null;
extractedPolicyFee: string | null;
extractedBrokerFee: string | null;
/** IVA off A.N.A.'s `TAX` cell. Null on GMX — its certificate carries no
* premium, so no tax either. `LOCAL TAX` is not folded in; a non-zero one
* shows up in `matchNote`. */
extractedTax: string | null;
extractedTotal: string | null;
extractedCoveragesJson: PolicyOcrCoverage[] | null;
extractedPremiumPayment: string | null;
@@ -1542,6 +1609,7 @@ export interface PolicyOcrReviewInput {
netPremium?: number;
policyFee?: number;
brokerFee?: number;
tax?: number;
total?: number;
premiumPayment?: string;
coveragePeriodDays?: number;
@@ -1568,6 +1636,7 @@ export interface PolicyOcrConfirmDocument {
netPremium?: number;
policyFee?: number;
brokerFee?: number;
tax?: number;
total?: number;
premiumPayment?: string;
coveragePeriodDays?: number;
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env node
/**
* Delete unused images from the target host after a successful deploy.
*
* This exists because nothing else reclaims them. Every build.yml run pushes a
* new api + web image, every deploy pulls both onto the host, and the previous
* pair is left behind untagged-but-present forever. On galactus that reached
* 63 images / 83.85GB (79.26GB of it unused) and filled the 98GB root
* filesystem to 100% on 2026-08-20 — which surfaced as "re-import is broken",
* because the Operaciones REIMPORT job leads with a mysqldump that could no
* longer write its safety backup.
*
* Two things keep this from eating a live deployment:
*
* - Docker never prunes an image that a container references, running or
* stopped. The five images the prod stacks use are therefore untouchable
* for as long as their containers exist.
* - `until` gives a grace window on top of that, so a rollback target stays
* on disk instead of forcing a re-pull from the registry.
*
* TRAP: `until` filters on the image's CREATION time, not when the host pulled
* it. Rolling back to an old tag pulls an image that is already older than the
* window, so the grace period does NOT protect it — the running-container rule
* is what does. That is why this step must run AFTER the app stack is deployed
* and verified, never before.
*
* Required env:
* PORTAINER_URL, PORTAINER_API_KEY, PORTAINER_ENDPOINT_ID
* Optional:
* KEEP_HOURS grace window in hours (default 168 = 7 days)
*
* TLS: Portainer here is self-signed; the caller sets
* NODE_TLS_REJECT_UNAUTHORIZED=0 for this step.
*/
function required(name) {
const v = process.env[name];
if (!v) {
console.error(`missing required env: ${name}`);
process.exit(1);
}
return v;
}
const PORTAINER_URL = required("PORTAINER_URL").replace(/\/+$/, "");
const API_KEY = required("PORTAINER_API_KEY");
const ENDPOINT_ID = required("PORTAINER_ENDPOINT_ID");
const KEEP_HOURS = process.env.KEEP_HOURS || "168";
const DOCKER = `${PORTAINER_URL}/api/endpoints/${ENDPOINT_ID}/docker`;
// `dangling: ["false"]` is what makes this `docker image prune -a` rather than
// the default, which only collects untagged layers. The tagged-but-superseded
// api/web images are the whole problem, and the default filter walks straight
// past them.
const FILTERS = JSON.stringify({
dangling: ["false"],
until: [`${KEEP_HOURS}h`],
});
function human(bytes) {
if (!bytes) return "0B";
const units = ["B", "KB", "MB", "GB", "TB"];
let i = 0;
let n = bytes;
while (n >= 1024 && i < units.length - 1) {
n /= 1024;
i += 1;
}
return `${n.toFixed(i === 0 ? 0 : 2)}${units[i]}`;
}
async function main() {
const url = `${DOCKER}/images/prune?filters=${encodeURIComponent(FILTERS)}`;
const res = await fetch(url, {
method: "POST",
headers: { "X-API-Key": API_KEY },
});
const body = await res.text();
if (!res.ok) {
throw new Error(`prune -> HTTP ${res.status} ${body.slice(0, 300)}`);
}
let report;
try {
report = JSON.parse(body);
} catch {
throw new Error(`prune returned non-JSON: ${body.slice(0, 300)}`);
}
const deleted = report.ImagesDeleted ?? [];
const reclaimed = report.SpaceReclaimed ?? 0;
console.log(
`pruned images older than ${KEEP_HOURS}h and unused by any container`,
);
console.log(` entries removed : ${deleted.length}`);
console.log(` space reclaimed : ${human(reclaimed)}`);
}
main().catch((err) => {
// Non-fatal by contract: the step that calls this sets continue-on-error, so
// housekeeping never turns a good deploy red. Exit non-zero anyway so the
// failure is visible in the run rather than swallowed.
console.error(`::warning::image prune FAILED: ${err.message}`);
process.exit(1);
});
+23
View File
@@ -179,6 +179,29 @@ Each of these is a known, deliberate stopping point rather than a bug.
be added.
- No `SKIPPED_NO_EMAIL` worklist (see 1.9).
**Captura de pólizas — desglose de primas** (built 2026-08-18)
- **IVA y prima total no existen en los datos legacy.** En Access eran
controles calculados sin campo, así que las 2,378 pólizas migradas leen
`tax` y `total` en null hasta que alguien las edite. No es recuperable: no
hay de dónde.
- **`LOCAL TAX` de A.N.A. no se captura.** El IVA (`TAX`) sí — se guarda desde
2026-08-18 — pero `LOCAL TAX` es un gravamen distinto sin columna destino y
**no** se suma al IVA: sumarlo daría una cifra que ya no divide de vuelta a
una tasa. Imprime 0.00 en todas las pólizas vistas hasta hoy; una distinta
de cero levanta la nota *"impuesto local N no capturado"* y significa que
`total` no cuadra contra `netPremium + policyFee + tax`.
- **`Policy.taxRate` queda en null por la ruta OCR.** A.N.A. imprime el monto
del IVA, no la tasa, y despejarla a la inversa inventaría una tasa que el
documento nunca declaró. El formulario resuelve una desde el ramo.
- **El recargo no se valida contra la forma de pago en datos migrados.** El
formulario lo deshabilita en ANUAL/CONTADO, pero
`backfill_policy_premium_breakdown.py` solo advierte cuando encuentra una
póliza anual con recargo; no la corrige.
- **Las parcialidades 3 y 4 no llevan desglose.** Access solo dibujó la fila
de dinero dos veces, así que una póliza trimestral capturada hoy sí puede
llenar las cuatro a mano, pero no hay nada legacy que migrar a las dos
últimas.
**Policy OCR** — [`POLICY_OCR.md`](POLICY_OCR.md)
- **GMX and A.N.A. only.** The dispatcher is a `[provider, pattern]` table plus
a parser map, so a third carrier is one function and two entries — but no
+6
View File
@@ -440,6 +440,12 @@ take a `policyType` select param. The workflow is *not* MULT-only: the legacy
Params: ramo (with an "todos" option), aseguradora, date range on `policyFrom`.
Columns: póliza, cliente, ramo, aseguradora, vigencia, prima neta, forma de pago.
️ `forma de pago` became a real column on 2026-08-18 (`Policy.paymentFrequency`).
It is **null on every policy migrated before that date** — the original ETL
marked Access's `FORMA PAGO` consumed and then never wrote it anywhere — so the
report must render null as "—" rather than assuming annual. Running
`backfill_policy_premium_breakdown.py` recovers it from the staged Parquet.
Totals: count + prima neta sum per currency (**never collapse MXN and USD** —
same constraint as the billing module).
+21
View File
@@ -365,6 +365,27 @@ A.N.A.'s faces do print one — the `DISCOUNT / PREMIUM / POLICY FEE / TAX /
LOCAL TAX / TOTAL` row is on the same page — so an ANA document reaches the
review queue with `netPremium` populated and `postPremium` already ticked.
Four of those six cells are stored: `PREMIUM``netPremium`, `POLICY FEE`
`policyFee`, `TAX``tax` (`extractedTax` on the document, `Policy.tax` on
confirm), `TOTAL``total`. `DISCOUNT` and `LOCAL TAX` are reported as notes
instead:
- **`DISCOUNT`** has no column, and it prints as a bare `-` when unused, which
is what makes the row positional rather than "find six amounts".
- **`LOCAL TAX`** is a separate levy and is deliberately **not** summed into
`tax`. Folding it in would produce an IVA figure that no longer divides back
to a rate, which is the reason to store it at all. It reads 0.00 on every
A.N.A. policy seen so far; a non-zero one raises
*"impuesto local N no capturado"* and means `total` will not reconcile
against `netPremium + policyFee + tax`.
`Policy.taxRate` is left null by confirm. A.N.A. prints the IVA **amount**, not
the rate, and back-dividing one would mint a rate the document never stated —
the capture form resolves it from the line of business instead
(`PolicyType.taxRate`, see `apps/api/src/policies/premium.ts`). The figures do
agree: 298.61 + 30.00 taxed at 8% is 26.29, totalling 354.90, asserted in
`policy-parser.spec.ts`.
Deductible and loss participation are stored as **strings** (`"5%"`, `"20%"`,
`"USD 1,000"`) — they are printed as a mix of percentages, currency amounts
and free text, and normalising them would lose the distinction.
@@ -0,0 +1,249 @@
"""
Recovers the premium breakdown the original policy transform dropped.
`transform_policies.py` modeled prima neta, derecho de póliza and comisión and
nothing else, which lost two things from every migrated policy:
1. RECARGO — the financing surcharge on a policy paid in more than one
exhibición, plus the whole second money row (`p_neta_2`, `recargo_2`,
`d_pol_2`, `com_2`) that a semestral policy carries because each payment
is priced separately. These were not deleted, they were swept into
`policies.coveragesJson` as loose strings alongside the real coverages —
unqueryable, and mislabeled as coverage amounts.
2. FORMA PAGO — dropped outright. The column was marked "consumed" by the
transform's coverage sweep but never written to any column, so it exists
nowhere in the platform database. It is the field that decides whether a
recargo is legitimate on a row at all, so it cannot be inferred back from
the money.
The transform has been fixed in the same commit, so a full `run_all.py` now
produces all of this directly. This script exists for a database that must not
be re-imported: it reads the same staged Parquet and patches in place.
What it does NOT do: invent IVA or the printed TOTAL. Those were never columns
in the home tables — they were unbound calculated controls on the Access form —
so there is genuinely nothing to recover, and both stay null until a human
edits the policy. The app computes them (apps/api/src/policies/premium.ts).
Idempotent, and never overwrites a non-null value: a figure a human has since
corrected in the app wins over the legacy one.
./.venv/bin/python backfill_policy_premium_breakdown.py --env dev
"""
from __future__ import annotations
import json
import sys
from decimal import Decimal, InvalidOperation
from pathlib import Path
import pandas as pd
from dbenv import connect
from sync import parse_mode
STG = Path(__file__).parent / "output" / "stg_seguros"
LEGACY_DB = "SEGUROS 16_be"
NULL = ""
# Legacy column -> what it is, per source table. Only the tables that actually
# carry a breakdown appear; the auto tables have no recargo and no second row.
HOME_TABLES = ("mult", "incendio", "m_empr")
# Keys the transform used to dump into coveragesJson that are now real columns.
# Stripped once migrated so the blob stops pretending they are coverages.
MIGRATED_COVERAGE_KEYS = (
"recargo", "recargo_2", "p_neta_2", "d_pol_2", "com_2",
)
_FREQ = {
"ANNUAL": "ANNUAL",
"ANUAL": "ANNUAL",
"SEMESTRAL": "SEMIANNUAL",
"TRIMESTRAL": "QUARTERLY",
"MENSUAL": "MONTHLY",
"CONTADO": "SINGLE",
}
def s(v):
if v is None or pd.isna(v):
return None
v = str(v).strip()
return None if v in ("", NULL) else v
def dec(v):
v = s(v)
if v is None:
return None
try:
return Decimal(v.replace(",", ""))
except (InvalidOperation, ValueError):
return None
def freq(v):
return _FREQ.get((s(v) or "").upper())
def load(name):
df = pd.read_parquet(STG / f"{name}.parquet").sort_values("_row_num").reset_index(drop=True)
for c in df.columns:
if c != "_row_num":
df[c] = df[c].astype("string").str.strip()
return df
def main():
env, _sync_mode = parse_mode()
# Fails closed rather than reporting a clean run over nothing: an empty
# staging directory and a policy set with no recargo look identical from
# the database side, and "0 rows updated" would read as success.
if not STG.exists():
print(f"[policy-premium] staged Parquet missing at {STG} — run extract/load first.")
return 3
conn = connect(env)
c = conn.cursor()
print(f"[policy-premium] target env: {env}")
# Every policy that came from the insurance ETL, keyed by provenance. The
# id is needed to reach the installments, coveragesJson to strip the keys.
c.execute(
"SELECT legacySourceTable, legacyId, id, coveragesJson "
"FROM policies WHERE legacySourceDb = %s AND legacyId IS NOT NULL",
(LEGACY_DB,),
)
by_key = {(t, lid): (pid, cov) for t, lid, pid, cov in c.fetchall()}
print(f" {len(by_key)} migrated polic(ies) in the target database")
pol_updates = [] # (surcharge, paymentFrequency, coveragesJson, policyId)
inst_updates = [] # (netPremium, surcharge, policyFee, commission, policyId, seq)
seen_tables = 0
for table in HOME_TABLES + (
"tabla_autos", "tabla_autos_ampl", "tabla_autos_limit",
"tabla_autos_ampl_r", "tabla_autos_rc_r", "mca2", "licencias",
):
path = STG / f"{table}.parquet"
if not path.exists():
continue
seen_tables += 1
df = load(table)
home = table in HOME_TABLES
for _, row in df.iterrows():
key = (table, str(int(row["_row_num"])))
hit = by_key.get(key)
if not hit:
continue
pid, cov_raw = hit
surcharge = dec(row.get("recargo")) if home else None
frequency = freq(row.get("forma_pago"))
# Strip the now-modeled keys out of the coverage blob. Rewritten
# only when something actually changes, so a policy whose blob a
# human has edited is left byte-identical.
cov_new = None
if cov_raw:
try:
cov = json.loads(cov_raw) if isinstance(cov_raw, str) else cov_raw
except (TypeError, ValueError):
cov = None
if isinstance(cov, dict):
kept = {k: v for k, v in cov.items() if k not in MIGRATED_COVERAGE_KEYS}
if len(kept) != len(cov):
cov_new = json.dumps(kept, ensure_ascii=False) if kept else None
if surcharge is not None or frequency is not None or cov_new is not None:
pol_updates.append((surcharge, frequency, cov_new, cov_new is not None, pid))
# Per-payment breakdown. Slot 1 is the unsuffixed money row, slot 2
# the _2 twin; the auto tables have a single slot and no recargo.
if home:
slots = [
(1, "p_neta", "recargo", "d_pol", "com"),
(2, "p_neta_2", "recargo_2", "d_pol_2", "com_2"),
]
else:
pn = "prima1" if table == "mca2" else "prima_neta"
dp = "d_poliza1" if table == "mca2" else "d_poliza"
slots = [(1, pn, None, dp, None)]
for seq, pn, rc, dp, cm in slots:
vals = (
dec(row.get(pn)) if pn else None,
dec(row.get(rc)) if rc else None,
dec(row.get(dp)) if dp else None,
dec(row.get(cm)) if cm else None,
)
if all(v is None for v in vals):
continue
inst_updates.append((*vals, pid, seq))
if not seen_tables:
print(f"[policy-premium] no policy tables staged under {STG} — nothing to do.")
return 3
# COALESCE on every target: a column a human has already filled in the app
# keeps its value, the legacy figure only lands where there is a hole.
for surcharge, frequency, cov_new, rewrite_cov, pid in pol_updates:
c.execute(
"UPDATE policies SET "
" surcharge = COALESCE(surcharge, %s), "
" paymentFrequency = COALESCE(paymentFrequency, %s), "
" coveragesJson = IF(%s, %s, coveragesJson) "
"WHERE id = %s",
(surcharge, frequency, 1 if rewrite_cov else 0, cov_new, pid),
)
for netp, surch, fee, comm, pid, seq in inst_updates:
c.execute(
"UPDATE policy_payment_installments SET "
" netPremium = COALESCE(netPremium, %s), "
" surcharge = COALESCE(surcharge, %s), "
" policyFee = COALESCE(policyFee, %s), "
" commission = COALESCE(commission, %s) "
"WHERE policyId = %s AND sequence = %s",
(netp, surch, fee, comm, pid, seq),
)
conn.commit()
print(f" policies : {len(pol_updates)} row(s) touched")
print(f" installments: {len(inst_updates)} row(s) touched")
# --- validation ---------------------------------------------------------
c.execute("SELECT COUNT(*) FROM policies WHERE surcharge IS NOT NULL AND surcharge <> 0")
n_surch = c.fetchone()[0]
c.execute("SELECT COUNT(*) FROM policies WHERE paymentFrequency IS NOT NULL")
n_freq = c.fetchone()[0]
c.execute(
"SELECT COUNT(*) FROM policies "
"WHERE paymentFrequency IN ('ANNUAL','SINGLE') AND surcharge IS NOT NULL AND surcharge <> 0"
)
n_bad = c.fetchone()[0]
c.execute(
"SELECT COUNT(*) FROM policies WHERE coveragesJson IS NOT NULL "
"AND JSON_EXTRACT(coveragesJson, '$.recargo') IS NOT NULL"
)
n_left = c.fetchone()[0]
print(f" -> policies with a recargo : {n_surch}")
print(f" -> policies with a forma pago : {n_freq}")
print(f" -> recargo still in coverages : {n_left}")
# A surcharge on an annual policy contradicts the rule the capture form
# enforces, so it is worth surfacing rather than leaving for someone to
# find in a total. It is a warning, not a failure: the books are the books.
if n_bad:
print(f" !! {n_bad} annual/contado polic(ies) carry a non-zero recargo — review by hand")
return 0
if __name__ == "__main__":
sys.exit(main() or 0)
+57
View File
@@ -18,6 +18,7 @@ the four Access source files.
"""
import os
import re
from pathlib import Path
# The folder holding the four Access source files. Overridable via INGEST_DIR so
@@ -95,3 +96,59 @@ SOURCES = {
},
},
}
# --- prior-period archives ----------------------------------------------
#
# Legacy ran a year-end *corte*: it summed the closing year, wrote that total
# back as each customer's Jan-1 BALANCE FORWARD, and started the next year
# clean. Access keeps the closed year as a whole-database snapshot named for
# the period it holds — `2025.accdb` is UTILITIES as it stood when 2025 was
# cut — and the office archives one per year.
#
# Only the ledger is staged out of a snapshot. Everything else in it (DATMEX,
# PROFILE, EFECTIVO, ...) is a year-stale copy of a table the live
# UTILITIES.accdb already provides, and staging all ~50 of them would triple
# the extract time to import data we would then have to ignore. DATGRAL comes
# along solely to check that a NUMid still means the same customer it did that
# year; see the recycle guard in transform_transactions.py.
#
# The cash side is deliberately NOT taken from the snapshot: `EFECTIVO` is a
# lifetime journal, so the snapshot's copy is a subset of the live one and
# importing it would double-book every prior-year receipt.
PERIOD_FILE_RE = re.compile(r"^(\d{4})\.accdb$", re.IGNORECASE)
PERIOD_TABLES = {"datos2", "DATGRAL"}
def period_schema(year: int) -> str:
return f"stg_period_{year}"
def discover_periods(root: Path) -> dict[str, dict]:
"""Find every `YYYY.accdb` archive sitting in the ingest folder.
Discovery is by filename because that is the whole upload contract: the
operator drops `2025.accdb` on the Operaciones page and the period is 2025.
Nothing inside the file names the year — a snapshot's `datos2` looks
identical to the live one — so the name is the only declaration of intent
we get, and it is what the allowlist on the upload endpoint enforces.
"""
found: dict[str, dict] = {}
if not root.is_dir():
return found
for path in sorted(root.iterdir()):
m = PERIOD_FILE_RE.match(path.name)
if not m:
continue
year = int(m.group(1))
found[f"period_{year}"] = {
"path": path,
"schema": period_schema(year),
"exclude": set(),
"include": set(PERIOD_TABLES),
"period_year": year,
}
return found
SOURCES.update(discover_periods(SOURCE_ROOT))
+11
View File
@@ -40,6 +40,17 @@ def stage_source(source_name: str, source_cfg: dict, sink) -> None:
tables = extract.list_tables(cnxn)
excluded = source_cfg["exclude"]
# A source may name the only tables it is worth staging. Prior-period
# archives do: they are whole-database snapshots, but everything in them
# except the ledger is a year-stale copy of a live table, so staging the
# rest costs minutes per file to produce data nothing reads.
include = source_cfg.get("include")
if include is not None:
missing = include - set(tables)
if missing:
print(f" [WARN] {source_name}: missing expected table(s) {sorted(missing)}", file=sys.stderr)
tables = [t for t in tables if t in include]
for table_name in tables:
if table_name in excluded:
print(f" [exclude] {table_name}")
+12
View File
@@ -59,6 +59,12 @@ STEPS = [
# touches.
"backfill_statement_match_fields.py",
"transform_policies.py",
# Premium breakdown (recargo, per-payment figures, forma de pago).
# transform_policies.py now writes these directly, so on a full rebuild
# this is a no-op that re-asserts them; on a database migrated before the
# breakdown existed it is what recovers them out of coveragesJson.
# Must follow transform_policies.py, which truncates the installments.
"backfill_policy_premium_breakdown.py",
"transform_transactions.py",
"prune_empty_customers.py",
# Seeds the Scotiabank chequera that every SCOTHIA movement is booked into;
@@ -78,6 +84,12 @@ SYNC_STEPS = [
# touches.
"backfill_statement_match_fields.py",
"transform_policies.py",
# Premium breakdown (recargo, per-payment figures, forma de pago).
# transform_policies.py now writes these directly, so on a full rebuild
# this is a no-op that re-asserts them; on a database migrated before the
# breakdown existed it is what recovers them out of coveragesJson.
# Must follow transform_policies.py, which truncates the installments.
"backfill_policy_premium_breakdown.py",
"transform_transactions.py",
# Manual-safe prune: drops legacy-owned empties that the customer upsert
# re-creates from Parquet, but leaves manually-added customers alone.
+72 -12
View File
@@ -18,6 +18,17 @@ Design (validated against staged data):
policies are skipped and counted (required FK).
- Payment slots: c_1er_pago is the first amount, pago_subsec the recurring
amount for slots 2-4; efectivo is a cash flag, no_cheque the check ref.
- Premium breakdown: a policy paid in more than one exhibicion prices EACH
payment separately, which is why the home tables carry the whole money row
twice (p_neta/recargo/d_pol/com and their _2 twins). The unsuffixed set is
the policy header, the suffixed one belongs to payment 2, and both are
written per installment as well. This used to be lost: `recargo` and the
_2 columns fell into coveragesJson as loose strings and forma_pago was
marked consumed but never written anywhere at all.
- IVA and the printed TOTAL are NOT in Access for the home tables. They were
unbound calculated controls on the form, so there is nothing to migrate;
the app computes them (apps/api/src/policies/premium.ts) from
(p_neta + recargo + d_pol) * rate.
- Any source column not explicitly modeled (coverage amounts: edificio,
contenidos, robo, cristales, ...) is preserved verbatim in coveragesJson,
so nothing is lost in consolidation.
@@ -93,20 +104,55 @@ def truthy(v):
return (s(v) or "0").lower() in {"1", "-1", "true", "si", "", "yes", "x"}
# Access FORMA PAGO -> PaymentFrequency. The whole staged corpus holds exactly
# four spellings (ANNUAL 1851, SEMESTRAL 29, semestral 2, CONTADO 1); anything
# else is left null rather than guessed, because the value decides whether a
# recargo is legitimate on the row.
_FREQ = {
"ANNUAL": "ANNUAL",
"ANUAL": "ANNUAL",
"SEMESTRAL": "SEMIANNUAL",
"TRIMESTRAL": "QUARTERLY",
"MENSUAL": "MONTHLY",
"CONTADO": "SINGLE",
}
def freq(v):
return _FREQ.get((s(v) or "").upper())
def slot_dec(row, slot, key):
"""One figure of a payment's premium breakdown, or None when the source
table has no such column. Deliberately not zero: a zero d_pol on the second
payment is a real figure in the books and must stay distinguishable from
'this table never had that column'."""
col = slot.get(key)
return dec(row.get(col)) if col else None
# --- per-table config ------------------------------------------------------- #
# fields: policy column -> source column. installments: list of slot dicts.
# vehicles: 'trip_underscore' | 'single' | 'mca2' | None. drivers: 'mca2' |
# 'licencias' | None.
# `pneta`/`recarg`/`dpol`/`com` on a slot are that payment's own share of the
# premium. Only the first two payments have one in Access — the form only ever
# drew the money row twice — so slots 3 and 4 carry none and keep just the
# amount actually collected.
HOME_INST = [
dict(seq=1, amt="c_1er_pago", cu="moned", d="fecha_pago", ck="no_cheque", cash="efectivo"),
dict(seq=2, amt="pago_subsec", cu="moned_2", d="fecha_pago_2", ck="no_cheque_2", cash="efectivo_2"),
dict(seq=1, amt="c_1er_pago", cu="moned", d="fecha_pago", ck="no_cheque", cash="efectivo",
pneta="p_neta", recarg="recargo", dpol="d_pol", com="com"),
dict(seq=2, amt="pago_subsec", cu="moned_2", d="fecha_pago_2", ck="no_cheque_2", cash="efectivo_2",
pneta="p_neta_2", recarg="recargo_2", dpol="d_pol_2", com="com_2"),
dict(seq=3, amt="pago_subsec", cu="moned_3", d="fecha_pago_3", ck="no_cheque_3", cash="efectivo_3"),
dict(seq=4, amt="pago_subsec", cu="moned_4", d="fecha_pago_4", ck="no_cheque_4", cash="efectivo_4"),
]
HOME_FIELDS = dict(polno="no_poliza", agent="agent", comp="comp", desde="desde", hasta="hasta",
forma="forma_pago", curcol="moned", pneta="p_neta", dpol="d_pol", com="com",
forma="forma_pago", curcol="moned", pneta="p_neta", recarg="recargo",
dpol="d_pol", com="com",
liquidada="liquidada", numliq="num_liquidacion", fliq="f_liquida1", renov="renovacion")
AUTO_SINGLE_INST = [dict(seq=1, amt="total", cu="moneda", d="fecha_pago", ck="no_cheque", cash="efectivo")]
AUTO_SINGLE_INST = [dict(seq=1, amt="total", cu="moneda", d="fecha_pago", ck="no_cheque", cash="efectivo",
pneta="prima_neta", dpol="d_poliza")]
CONFIGS = {
"incendio": dict(ptype="INCENDIO", idcol="num_id", fields={**HOME_FIELDS, "curcol": "moneda"},
@@ -157,7 +203,8 @@ CONFIGS = {
forma="forma_pago", curcol="moneda", pneta="prima_neta", dpol="d_poliza",
total="total", liquidada="liquidada", numliq="num_liquidacion",
fliq="f_liquida1", renov="renovacion"),
inst=[dict(seq=1, amt="total", cu="moneda", d="fecha_pago", ck="no_cheque", cash="efectivo")],
inst=[dict(seq=1, amt="total", cu="moneda", d="fecha_pago", ck="no_cheque",
cash="efectivo", pneta="prima_neta", dpol="d_poliza")],
veh=None, drv="licencias"),
}
@@ -200,6 +247,10 @@ def main():
consumed = {cfg["idcol"], *F.values()}
for slot in cfg["inst"]:
consumed |= {slot["amt"], slot["cu"], slot["d"], slot["ck"], slot["cash"]}
# The per-payment premium columns are now modeled, so they must
# leave the coveragesJson sweep — otherwise every recargo would be
# written twice, once as a column and once as a fake coverage.
consumed |= {slot[k] for k in ("pneta", "recarg", "dpol", "com") if slot.get(k)}
for _, row in df.iterrows():
cid = cust.get(norm_id(row[cfg["idcol"]]))
@@ -237,9 +288,11 @@ def main():
dt(row.get(F.get("desde", ""))) if F.get("desde") else None,
dt(row.get(F.get("hasta", ""))) if F.get("hasta") else None,
dec(row.get(F.get("pneta", ""))) if F.get("pneta") else None,
dec(row.get(F.get("recarg", ""))) if F.get("recarg") else None,
dec(row.get(F.get("dpol", ""))) if F.get("dpol") else None,
dec(row.get(F.get("com", ""))) if F.get("com") else None,
dec(row.get(F.get("total", ""))) if F.get("total") else None,
freq(row.get(F.get("forma", ""))) if F.get("forma") else None,
cur(row.get(F.get("curcol", ""))) if F.get("curcol") else "MXN",
s(row.get("observaciones")),
json.dumps(cov, ensure_ascii=False) if cov else None,
@@ -255,9 +308,12 @@ def main():
pdate = dt(row.get(slot["d"]))
if amt is None and pdate is None:
continue
# Slot breakdown, where the source table has one.
insts.append((str(uuid.uuid4()), pid, slot["seq"], amt,
cur(row.get(slot["cu"])), pdate, s(row.get(slot["ck"])),
1 if truthy(row.get(slot["cash"])) else 0))
1 if truthy(row.get(slot["cash"])) else 0,
slot_dec(row, slot, "pneta"), slot_dec(row, slot, "recarg"),
slot_dec(row, slot, "dpol"), slot_dec(row, slot, "com")))
# vehicles
def add_vehicle(make, model, body, engine, plate, year=None, state=None):
@@ -328,16 +384,19 @@ def main():
1 if truthy(r["concluido"]) else 0, s(r["resolucion"])))
pol_cols = ("id,policyNumber,customerId,policyTypeId,insuranceProviderId,agentName,policyDate,"
"policyFrom,policyTo,netPremium,policyFee,commission,total,currency,observations,"
"policyFrom,policyTo,netPremium,surcharge,policyFee,commission,total,paymentFrequency,"
"currency,observations,"
"coveragesJson,liquidated,liquidationNumber,liquidationDate,legacySourceDb,"
"legacySourceTable,legacyId,updatedAt")
ph = ",".join(["%s"] * 23)
ph = ",".join(["%s"] * 25)
pol_upsert = (
f"INSERT INTO policies ({pol_cols}) VALUES ({ph}) ON DUPLICATE KEY UPDATE "
"customerId=VALUES(customerId),policyNumber=VALUES(policyNumber),policyTypeId=VALUES(policyTypeId),"
"insuranceProviderId=VALUES(insuranceProviderId),agentName=VALUES(agentName),policyDate=VALUES(policyDate),"
"policyFrom=VALUES(policyFrom),policyTo=VALUES(policyTo),netPremium=VALUES(netPremium),policyFee=VALUES(policyFee),"
"commission=VALUES(commission),total=VALUES(total),currency=VALUES(currency),observations=VALUES(observations),"
"policyFrom=VALUES(policyFrom),policyTo=VALUES(policyTo),netPremium=VALUES(netPremium),"
"surcharge=VALUES(surcharge),policyFee=VALUES(policyFee),"
"commission=VALUES(commission),total=VALUES(total),paymentFrequency=VALUES(paymentFrequency),"
"currency=VALUES(currency),observations=VALUES(observations),"
"coveragesJson=VALUES(coveragesJson),liquidated=VALUES(liquidated),liquidationNumber=VALUES(liquidationNumber),"
"liquidationDate=VALUES(liquidationDate),updatedAt=VALUES(updatedAt),archivedAt=NULL")
@@ -399,8 +458,9 @@ def main():
c.executemany("INSERT INTO policy_payment_installments "
"(id,policyId,sequence,amount,currency,paidDate,checkNumber,isCash) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s)", insts)
"(id,policyId,sequence,amount,currency,paidDate,checkNumber,isCash,"
"netPremium,surcharge,policyFee,commission) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", insts)
c.executemany("INSERT INTO vehicles (id,customerId,policyId,make,model,modelYear,bodyType,"
"engineNumber,licensePlate,stateCode,legacySourceTable,legacyId) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", vehicles)
+151 -4
View File
@@ -90,6 +90,62 @@ def load(src, name):
return df
def verify_corte(c, year: int) -> None:
"""Assert legacy's corte identity: SUM(period Y) == BALANCE FORWARD(Y+1).
This is the whole reason a year can be shown on its own. Legacy closed each
year by summing it and writing that total back as every customer's Jan-1
opening row for the next one, so if an archive is the right file, complete,
and attached to the right customers, its per-customer total lands exactly on
the next year's BALANCE FORWARD. A truncated export, a file dropped under
the wrong year, or a botched customer match all break the identity loudly
here instead of quietly six months from now.
Reported, never fatal. Legacy publishes on its own schedule, so a handful of
customers legitimately drift between the snapshot and the cut — the run that
established this reconciled 1,160 of 1,170.
"""
c.execute(
"""
SELECT sums.customerId, sums.total, bf.amount
FROM (
SELECT customerId, ROUND(SUM(amount), 2) AS total
FROM transactions
WHERE legacySourceTable = %s AND voidedAt IS NULL
GROUP BY customerId
) sums
LEFT JOIN (
SELECT t.customerId, ROUND(SUM(t.amount), 2) AS amount
FROM transactions t
JOIN type_transactions tt ON tt.id = t.typeId
WHERE tt.nameEn = 'BALANCE FORWARD' AND t.voidedAt IS NULL
AND t.transactionDate >= %s AND t.transactionDate < %s
GROUP BY t.customerId
) bf ON bf.customerId = sums.customerId
""",
(f"datos2@{year}", f"{year + 1}-01-01", f"{year + 1}-01-02"),
)
rows = c.fetchall()
matched = mismatched = 0
missing = 0
drift = Decimal(0)
for _cid, total, amount in rows:
if amount is None:
missing += 1
continue
if abs(Decimal(str(total)) - Decimal(str(amount))) < Decimal("0.02"):
matched += 1
else:
mismatched += 1
drift += abs(Decimal(str(total)) - Decimal(str(amount)))
checked = matched + mismatched
pct = (100 * matched / checked) if checked else 0
print(
f" corte {year} -> BF {year + 1}: {matched}/{checked} match ({pct:.1f}%)"
f", {mismatched} off by {drift:,.2f}, {missing} with no BF row"
)
def main():
env, sync_mode = parse_mode()
conn = connect(env)
@@ -230,9 +286,13 @@ def main():
message=s(r["conepto"]), check=s(r[check_col]) if check_col else None,
src_db="UTILITIES", src_tbl=legacy_tbl, legacy=str(int(r["_row_num"])))
def billing(name, legacy_tbl):
def billing(name, legacy_tbl, *, src="stg_utilities", skip_numids=None):
"""Load a DATOS2-shaped billing ledger.
`src` names the staging schema, so a prior-period archive
(stg_period_2025) loads through this same path: the snapshot's `datos2`
is the identical eleven-column shape, one year older.
NOPAGO is the legacy "still owed" flag. The website reads it directly —
`account.statement.php` splits the statement on `NOPAGO = 0` vs
`NOPAGO = 1` and renders the latter as the "Outstanding Bills Requiring
@@ -241,10 +301,13 @@ def main():
Only these three tables carry it (76 rows set in DATOS2 today); the
EFECTIVO/FM3 cash streams have no such column and stay 0.
"""
nonlocal skip_cust, skip_date
df = load("stg_utilities", name)
nonlocal skip_cust, skip_date, skip_recycled
df = load(src, name)
for _, r in df.iterrows():
cid = util_cust.get(norm_id(r["numid"]))
numid = norm_id(r["numid"])
if skip_numids and numid in skip_numids:
skip_recycled += 1; continue
cid = util_cust.get(numid)
if not cid:
skip_cust += 1; continue
td = dt(r["date"])
@@ -257,6 +320,53 @@ def main():
legacy=str(int(r["_row_num"])),
outstanding=1 if s(r["nopago"]) == "1" else 0)
skip_recycled = 0
recycle_report: list[tuple[int, str, str, str]] = []
def period_numid_guard(year: int) -> set[str]:
"""NUMids whose prior-period owner is not today's customer.
Prior-period rows attach by NUMid and nothing else, so a number the
office retired and reissued would file one customer's ledger under
another's name — the one error this feature must never make, because it
shows a stranger's charges to whoever holds the number now.
Reuse is real but rare: comparing each archive's DATGRAL against the
live one, 13 names moved since 2025 and 40 since 2024. Most are the same
customer re-described — a typo fixed (VIKIE -> VICKIE), a spouse added
or dropped (STEWART, ALAN R. -> STEWART, ALAN & JENNIFER). A few are
genuinely a different household (STRONKS, BOB -> SWEET, DONALD E.).
Sharing any word of three or more characters separates the two cleanly:
a rename keeps the surname, a reissue keeps nothing. Names are compared
legacy-to-legacy, archive DATGRAL against live DATGRAL, deliberately not
against `customers.name` — that column has been through the blank-name
recovery pass, and comparing to it reported 121 drifts where there are
13, every extra one a false positive that would have discarded good
history.
"""
try:
arch = load(f"stg_period_{year}", "datgral")
live = load("stg_utilities", "datgral")
except (FileNotFoundError, OSError):
return set()
def toks(v) -> set[str]:
return {w for w in "".join(ch if ch.isalnum() else " " for ch in (s(v) or "").upper()).split() if len(w) >= 3}
live_names = {norm_id(r["num_id"]): s(r["nombre"]) for _, r in live.iterrows()}
blocked: set[str] = set()
for _, r in arch.iterrows():
numid = norm_id(r["num_id"])
was, now = s(r["nombre"]), live_names.get(numid)
if not numid or not was or not now:
continue
if toks(was) & toks(now):
continue
blocked.add(numid)
recycle_report.append((year, numid, was, now))
return blocked
def iva():
nonlocal skip_cust
df = load("stg_utilities", "iva_2015")
@@ -286,6 +396,31 @@ def main():
billing("fee_anual", "FEE ANUAL")
billing("fee15", "fee15")
iva()
# --- prior periods -----------------------------------------------------
#
# Legacy kept each closed year in its own table and opened the next one with
# a Jan-1 BALANCE FORWARD carrying the closing total. The platform has one
# `transactions` table, so the period a row belongs to has to travel with
# the row: it rides in legacySourceTable as `datos2@2025`.
#
# That tag, not the date, is what a year view should filter on. The archives
# are not cleanly bounded — 2025's ledger carries ten undated rows and two
# dated into 2026 — and legacy itself never filtered by date either: its
# reader is `SELECT ... FROM \`2025\``. Keying on provenance reproduces the
# legacy period exactly and strands nothing.
#
# The tag also keeps the unique key safe. legacyId is a positional row
# ordinal, so every archive restarts it at 0 and would collide with the live
# `datos2` row-for-row if they shared a source-table name.
periods = sorted(
int(d.name.rsplit("_", 1)[1])
for d in STG.glob("stg_period_*")
if d.is_dir() and d.name.rsplit("_", 1)[1].isdigit()
)
for year in periods:
billing("datos2", f"datos2@{year}", src=f"stg_period_{year}",
skip_numids=period_numid_guard(year))
# Same record shape in the seguros DB. Labelled for consistency in the
# platform's own UI; unverifiable against the site, which only ever reads
# domain='UTILITY', so no customer-facing behaviour depends on it.
@@ -334,6 +469,7 @@ def main():
print(f" skipped (unresolved customer): {skip_cust}")
print(f" skipped (unparseable date) : {skip_date}")
print(f" skipped (EFECTIVO_BACKUP dup): {skip_dupe}")
print(f" skipped (reissued NUMid) : {skip_recycled}")
print(f" -> transactions : {count('transactions')}")
print(f" by domain : {dict(by_dom)}")
for src, n in by_src:
@@ -342,6 +478,17 @@ def main():
print(f" -> exchange_rates : {count('exchange_rates')}")
print(f" orphan transactions (bad customer FK): {orphans}")
assert orphans == 0, "transaction customer FK invariant failed"
if recycle_report:
print(f" ! reissued NUMids, prior-period rows NOT imported: {len(recycle_report)}")
for year, numid, was, now in recycle_report[:8]:
print(f" {year} NUMid {numid}: '{was}' -> '{now}'")
if len(recycle_report) > 8:
print(f" ... {len(recycle_report) - 8} more")
for year in periods:
verify_corte(c, year)
print(" validation: OK")
conn.close()
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "jorgecuadros-platform",
"version": "1.0.22",
"version": "1.0.26",
"private": true,
"workspaces": [
"apps/*",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@jorgecuadros/database",
"version": "1.0.22",
"version": "1.0.26",
"private": true,
"main": "generated/client/index.js",
"types": "generated/client/index.d.ts",
@@ -0,0 +1,37 @@
-- Premium breakdown the Access capture form had and this schema did not:
-- RECARGO, IVA and PRIMA TOTAL on the policy header, the same six figures per
-- payment on the installments, and the FORMA PAGO that decides whether a
-- surcharge applies at all.
--
-- IVA and TOTAL were never columns in Access — they were unbound calculated
-- controls on the form — so there is nothing to backfill for them here and
-- every migrated row stays null until somebody edits the policy. RECARGO and
-- the per-installment figures DO exist in the legacy data; they are currently
-- stranded inside `policies.coveragesJson` (the migration swept every column
-- it did not model into that blob) and are recovered by
-- `migration/backfill_policy_premium_breakdown.py`, not by this migration.
ALTER TABLE `policy_types`
ADD COLUMN `taxRate` DECIMAL(6, 4) NULL;
ALTER TABLE `policies`
ADD COLUMN `surcharge` DECIMAL(12, 2) NULL,
ADD COLUMN `tax` DECIMAL(12, 2) NULL,
ADD COLUMN `taxRate` DECIMAL(6, 4) NULL,
ADD COLUMN `paymentFrequency` ENUM('ANNUAL', 'SEMIANNUAL', 'QUARTERLY', 'MONTHLY', 'SINGLE') NULL;
ALTER TABLE `policy_payment_installments`
ADD COLUMN `netPremium` DECIMAL(12, 2) NULL,
ADD COLUMN `surcharge` DECIMAL(12, 2) NULL,
ADD COLUMN `policyFee` DECIMAL(12, 2) NULL,
ADD COLUMN `tax` DECIMAL(12, 2) NULL,
ADD COLUMN `taxRate` DECIMAL(6, 4) NULL,
ADD COLUMN `total` DECIMAL(12, 2) NULL,
ADD COLUMN `commission` DECIMAL(12, 2) NULL;
-- Seed the rate the books actually use. The legacy IMPUESTOS and
-- IMPUESTOS_AUTOS tables each held exactly one row, both 0.0800, covering the
-- home and auto lines respectively; applying it to every existing type
-- reproduces current behaviour rather than changing it. Types created later
-- start null and fall back to the API default.
UPDATE `policy_types` SET `taxRate` = 0.0800 WHERE `taxRate` IS NULL;
@@ -0,0 +1,9 @@
-- A.N.A. prints IVA on the policy face and the parser already read it, but
-- `ParsedPolicy` had no field for it, so the figure only ever reached a review
-- note and the confirmed Policy was written with `tax` null. This gives it a
-- column, matching the premium fields beside it.
--
-- GMX stays null: its certificate carries no premium at all, so there is no
-- tax on it to read either.
ALTER TABLE `policy_ocr_documents`
ADD COLUMN `extractedTax` DECIMAL(12, 2) NULL;
+65
View File
@@ -158,10 +158,31 @@ model InsuranceProvider {
@@map("insurance_providers")
}
/// How the premium is split into payments. Drives whether a surcharge
/// applies at all: the legacy books only ever charge `recargo` on a policy
/// paid in more than one exhibición, never on an annual one. Values come from
/// the Access `FORMA PAGO` column (ANNUAL / SEMESTRAL / CONTADO) plus the
/// quarterly option Jorge sells today but never recorded in Access.
enum PaymentFrequency {
ANNUAL
SEMIANNUAL
QUARTERLY
MONTHLY
/// Legacy "CONTADO" — the whole premium in one payment, no schedule.
SINGLE
}
model PolicyType {
id String @id @default(uuid())
name String @unique
shortDescription String?
/// IVA rate charged on this line of business, as a fraction (0.08 = 8%).
/// Replaces the legacy one-row IMPUESTOS / IMPUESTOS_AUTOS tables, which
/// held exactly one rate each — per line of business, editable without a
/// deploy, because the rate is a tax rule and tax rules change. Null falls
/// back to DEFAULT_TAX_RATE in the API rather than to "no tax", so a type
/// nobody has configured still computes the same 8% the books use today.
taxRate Decimal? @db.Decimal(6, 4)
policies Policy[]
@@map("policy_types")
@@ -185,10 +206,32 @@ model Policy {
policyTo DateTime?
coveragePeriodDays Int? @default(365)
netPremium Decimal? @db.Decimal(12, 2)
/// "Recargo" — the financing surcharge for paying in installments. Entered
/// by hand, never derived: it is quoted by the carrier, not computed here.
/// Only ever set when `paymentFrequency` is not ANNUAL/SINGLE, and it IS
/// part of the taxable base (verified against the Access books: policy
/// 7006785 only reconciles as (610.86 + 8.55 + 31.00) * 0.08 = 52.03).
surcharge Decimal? @db.Decimal(12, 2)
policyFee Decimal? @db.Decimal(12, 2)
brokerFee Decimal? @db.Decimal(12, 2)
commission Decimal? @db.Decimal(12, 2)
/// IVA. Access never stored this — it was an unbound calculated control on
/// the form — so every legacy row starts null and is filled going forward.
/// Stored rather than computed on read because the printed policy is the
/// record of truth and its rounding must survive a later rate change.
tax Decimal? @db.Decimal(12, 2)
/// The rate actually applied when `tax` was written, as a fraction. Kept on
/// the row so a policy issued at 8% still reads back as 8% after somebody
/// edits the PolicyType to a new rate.
taxRate Decimal? @db.Decimal(6, 4)
/// Prima total = netPremium + surcharge + policyFee + tax. Populated by the
/// capture form from now on. NOTE the legacy rows: `total` is 0 or null on
/// all but 2 of 2378 migrated policies, so list/sort code must keep using
/// netPremium as the headline (see policies.service.ts).
total Decimal? @db.Decimal(12, 2)
/// ANNUAL on all but 31 legacy rows — and the migration used to drop the
/// column entirely, so every pre-2026 policy reads null here.
paymentFrequency PaymentFrequency?
currency Currency @default(MXN)
observations String? @db.Text
notes String? @db.Text
@@ -267,6 +310,23 @@ model PolicyPaymentInstallment {
checkNumber String?
isCash Boolean @default(false)
// Per-payment premium breakdown. A policy paid in more than one exhibición
// prices EACH payment separately — its own net premium, its own surcharge,
// its own IVA — which is why the Access form printed the whole money row
// twice (P NETA / RECARGO / D POL / IVA / TOTAL / COM, once per pago) and
// why these cannot live on the policy header alone. `amount` stays the
// authoritative figure actually collected: it is what the cheque was
// written for and it drifts from `total` by a peso or two in the books
// (policy 7006785: amount 702.73 vs total 702.44), so it is deliberately
// NOT recomputed from this breakdown.
netPremium Decimal? @db.Decimal(12, 2)
surcharge Decimal? @db.Decimal(12, 2)
policyFee Decimal? @db.Decimal(12, 2)
tax Decimal? @db.Decimal(12, 2)
taxRate Decimal? @db.Decimal(6, 4)
total Decimal? @db.Decimal(12, 2)
commission Decimal? @db.Decimal(12, 2)
@@map("policy_payment_installments")
}
@@ -443,6 +503,11 @@ model PolicyOcrDocument {
extractedNetPremium Decimal? @db.Decimal(12, 2)
extractedPolicyFee Decimal? @db.Decimal(12, 2)
extractedBrokerFee Decimal? @db.Decimal(12, 2)
/// IVA off A.N.A.'s `TAX` cell. Null on GMX, whose certificate carries no
/// premium at all. The adjacent `LOCAL TAX` is a separate levy with no
/// column of its own and is NOT summed in — it would make the figure stop
/// dividing back to a rate; the parser reports a non-zero one as a note.
extractedTax Decimal? @db.Decimal(12, 2)
extractedTotal Decimal? @db.Decimal(12, 2)
/// Per-coverage rows from the GMX "Material damages" / "Additional risk"
/// tables and ANA's numbered risk sections — preserved verbatim so a
+550
View File
@@ -0,0 +1,550 @@
#!/usr/bin/env node
/**
* Corte (year-end cut) audit — READ ONLY. Writes nothing, voids nothing.
*
* Legacy Access ran a corte every year: it moved the year's utility movements
* into a per-year table and stamped one BALANCE FORWARD row per customer,
* dated Jan 1, carrying the closing balance. The platform inherited the ROWS
* (1,170 of them, dated 2026-01-01 — legacy's last cut before the extract) but
* not the PROCESS, and BillingService uses those rows as a per-customer floor
* (BALANCE_FLOOR_JOIN / NOT_SUPERSEDED in apps/api/src/billing/billing.service.ts).
*
* This script reports the two populations that floor does not cover:
*
* A. FLOORLESS customers — no BALANCE FORWARD row at all, so their balance
* is a raw lifetime sum. The platform only migrated the CURRENT-year
* charge ledger (datos2); the per-year charge tables live in DreamHost
* and were never staged. What survives before the cutover is therefore
* a cash journal — receipts with no matching charges — so those sums read
* as the office owing money it does not owe.
*
* READ THE COMPOSITION LINE BEFORE ACTING ON THIS. After the prior-period
* import the group is mostly insurance-only customers whose rows come from
* the seguros database's own EFECTIVO, which is that line's ONLY ledger.
* Flooring those deletes receipts instead of removing a double count. The
* "floor them, never carry" argument holds for the utilities rows alone.
*
* B. DOUBLE-BOOKED 2026 RECEIPTS — one cash receipt appearing twice, once in
* EFECTIVO with folio `N` and once in datos2 with reference `CN`.
*
* This is NOT an office data-entry defect, which is what it looked like
* while the pair count kept growing at ~40/month. EFECTIVO is the paper
* receipt book and every receipt in it is POSTED to the datos2 ledger by
* design — verified against the live legacy database, 296 of the 297
* receipts written in 2026 carry a matching posting. Legacy summed the
* ledger alone. The duplication was the migration flattening a journal and
* its postings into one table, and since 1.0.26 the application drops the
* journal from every balance. Section B now reports what the journal holds
* and asserts that none of it still reaches a balance.
*
* The folio alone neither proves nor disproves a pair, so it is used as a
* lead and never as the verdict. Folios are reused, so `C13483` can collide
* with an unrelated receipt; folios are also mistyped, so a genuine pair can
* carry two different numbers. Detection therefore runs twice — once on the
* `CN` cross-reference, once over the C-refs that pass left orphaned, this
* time on proximity alone (same customer, within three days) — and BOTH
* passes are then judged on the money: identical amount when the two legs
* share a currency, or an implied USD->MXN rate inside the band the
* exchange_rates table actually observed that year. Anything that fails is
* reported apart and must not be counted as duplicated money.
*
* The second pass is not a refinement. Jorge Jr's own account carries
* `C13647` against EFECTIVO folio `13649` — same day, same 3,500.00 — and
* POWERS carries `C135808` against `13508`. Folio matching alone reports
* both accounts as clean.
*
* node scripts/corte-audit.mjs # summary + both sections
* node scripts/corte-audit.mjs --cutover 2026-01-01
* node scripts/corte-audit.mjs --csv-a # per-customer table, section A
* node scripts/corte-audit.mjs --csv-b # per-pair table, section B
* node scripts/corte-audit.mjs --limit 40 # rows printed per section
*
* Needs DATABASE_URL. Point it at PROD — a stale copy answers about itself.
* On a database imported before the BALANCE FORWARD type was minted those rows
* carry typeId NULL instead (see numid.service.ts:80), so the floor is matched
* in BOTH shapes here; matching only the type name reports every customer as
* floorless on such a copy.
*/
import pkg from "../packages/database/generated/client/index.js";
const { PrismaClient } = pkg;
/** Prisma hands raw DECIMAL back as Decimal|string|null; counts as BigInt. */
const d = (v) => (v == null ? 0 : Number(v));
const money = (v) => d(v).toFixed(2).padStart(13);
/**
* Raw DATE/DATETIME columns arrive as JS Date objects. String() would render
* them in the host's local zone, which turns a row stored at 2026-01-01 00:00
* UTC into "Dec 31" on a US Pacific laptop — the ledger is keyed on UTC dates
* everywhere else, so format in UTC and nowhere else.
*/
const day = (v) => (v == null ? "—" : new Date(v).toISOString().slice(0, 10));
function arg(args, name, fallback = null) {
const i = args.indexOf(name);
return i === -1 ? fallback : args[i + 1];
}
/**
* A row is a balance-forward marker in either of two shapes. Keep in step with
* EMPTY_NUMID_SQL in apps/api/src/customers/numid.service.ts.
*/
const BF_PREDICATE = `(
tt.nameEn = 'BALANCE FORWARD'
OR (t.typeId IS NULL AND MONTH(t.transactionDate) = 1 AND DAY(t.transactionDate) = 1
AND t.legacySourceTable = 'datos2')
)`;
/**
* The rest of what a balance query drops, over and above the floor. Mirrors
* NOT_CASH_JOURNAL and archiveIsHistorySql in billing.service.ts — an audit
* that computes a different book than the application is worse than no audit,
* because its numbers look authoritative and diff cleanly against yesterday's.
*
* The database qualifier is not decoration. `SEGUROS 16_be` keeps its own table
* called EFECTIVO and that one is the insurance line's only ledger; matching on
* the table name alone would report 55,444.95 USD of real receivables as
* duplicate cash. See efectivo-is-a-journal-not-a-ledger.
*/
const NOT_CASH_JOURNAL = `(
t.legacySourceDb IS NULL
OR t.legacySourceDb <> 'UTILITIES'
OR t.legacySourceTable IS NULL
OR t.legacySourceTable NOT IN
('EFECTIVO', 'EFECTIVO_BACKUP', 'EFECTIVO FM3', 'CHEQUE FM3', 'IVA 2015')
)`;
/**
* An imported period counts as history below the year start and is dropped at
* or above it. Spelled as a positive OR: `NOT (col LIKE ... AND ...)` is NULL
* for an app-captured row, which would silently drop every one.
*
* The bound is the running calendar year, matching currentYearStart() in the
* application rather than the cutover — the app's current period is "this
* year", whatever cut the data happens to reflect.
*/
const yearStart = `${new Date().getUTCFullYear()}-01-01`;
const ARCHIVE_IS_HISTORY = `(
t.legacySourceTable IS NULL
OR t.legacySourceTable NOT LIKE 'datos2@%'
OR t.transactionDate < '${yearStart}'
)`;
/** Everything a balance drops apart from the floor itself. */
const READ_SCOPE = `(${NOT_CASH_JOURNAL} AND ${ARCHIVE_IS_HISTORY})`;
async function main() {
const args = process.argv.slice(2);
const limit = Number(arg(args, "--limit", "25"));
const prisma = new PrismaClient();
try {
// ---- cutover -----------------------------------------------------------
// Default to the newest balance-forward date actually in the book rather
// than to the current year: the cut the data reflects is a fact, not a
// preference, and hardcoding 2026 would silently lie on any other copy.
const [bfDates] = await prisma.$queryRawUnsafe(`
SELECT MAX(t.transactionDate) AS newest, MIN(t.transactionDate) AS oldest,
COUNT(*) AS rows_, COUNT(DISTINCT t.customerId) AS custs
FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.voidedAt IS NULL AND ${BF_PREDICATE}
`);
const cutover =
arg(args, "--cutover") ??
(bfDates.newest ? new Date(bfDates.newest).toISOString().slice(0, 10) : null);
if (!cutover) {
console.log("No BALANCE FORWARD rows in this database and no --cutover given.");
return;
}
console.log(`corte audit — cutover ${cutover}`);
console.log(
` balance-forward rows: ${d(bfDates.rows_)} across ${d(bfDates.custs)} customers` +
`, dated ${day(bfDates.oldest)}..${day(bfDates.newest)}`,
);
// ---- book totals -------------------------------------------------------
const [book] = await prisma.$queryRawUnsafe(
`
WITH bfloor AS (
SELECT t.customerId, MAX(t.transactionDate) AS floorDate
FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.voidedAt IS NULL AND ${BF_PREDICATE}
GROUP BY t.customerId
)
SELECT
ROUND(SUM(CASE WHEN t.currency='MXN' THEN t.amount ELSE 0 END), 2) AS rawMxn,
ROUND(SUM(CASE WHEN t.currency='USD' THEN t.amount ELSE 0 END), 2) AS rawUsd,
ROUND(SUM(CASE WHEN t.currency='MXN' AND (b.floorDate IS NULL OR t.transactionDate >= b.floorDate)
THEN t.amount ELSE 0 END), 2) AS floorOnlyMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND (b.floorDate IS NULL OR t.transactionDate >= b.floorDate)
THEN t.amount ELSE 0 END), 2) AS floorOnlyUsd,
ROUND(SUM(CASE WHEN t.currency='MXN' AND (b.floorDate IS NULL OR t.transactionDate >= b.floorDate)
AND ${READ_SCOPE} THEN t.amount ELSE 0 END), 2) AS todayMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND (b.floorDate IS NULL OR t.transactionDate >= b.floorDate)
AND ${READ_SCOPE} THEN t.amount ELSE 0 END), 2) AS todayUsd,
ROUND(SUM(CASE WHEN t.currency='MXN' AND t.transactionDate >= ?
THEN t.amount ELSE 0 END), 2) AS flooredMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND t.transactionDate >= ?
THEN t.amount ELSE 0 END), 2) AS flooredUsd
FROM transactions t
LEFT JOIN bfloor b ON b.customerId = t.customerId
WHERE t.voidedAt IS NULL AND t.outstanding = 0
`,
cutover,
cutover,
);
// ---- section A: floorless customers ------------------------------------
const floorless = await prisma.$queryRawUnsafe(
`
WITH nobf AS (
SELECT c.id, c.name
FROM customers c
WHERE EXISTS (SELECT 1 FROM transactions t WHERE t.customerId = c.id AND t.voidedAt IS NULL)
AND NOT EXISTS (
SELECT 1 FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.customerId = c.id AND t.voidedAt IS NULL AND ${BF_PREDICATE}
)
)
SELECT n.id, n.name,
COUNT(*) AS rows_,
SUM(t.transactionDate < ?) AS preRows,
SUM(t.transactionDate >= ?) AS postRows,
MIN(t.transactionDate) AS firstTx,
MAX(t.transactionDate) AS lastTx,
SUM(t.domain = 'UTILITY') AS utilRows,
SUM(t.domain = 'INSURANCE') AS insRows,
ROUND(SUM(CASE WHEN t.currency='MXN' AND t.outstanding=0 THEN t.amount ELSE 0 END), 2) AS todayMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND t.outstanding=0 THEN t.amount ELSE 0 END), 2) AS todayUsd,
ROUND(SUM(CASE WHEN t.currency='MXN' AND t.outstanding=0 AND t.transactionDate >= ?
THEN t.amount ELSE 0 END), 2) AS afterMxn,
ROUND(SUM(CASE WHEN t.currency='USD' AND t.outstanding=0 AND t.transactionDate >= ?
THEN t.amount ELSE 0 END), 2) AS afterUsd
FROM nobf n
JOIN transactions t ON t.customerId = n.id AND t.voidedAt IS NULL
GROUP BY n.id, n.name
ORDER BY ABS(SUM(CASE WHEN t.currency='MXN' AND t.transactionDate < ? THEN t.amount ELSE 0 END)) DESC
`,
cutover, cutover, cutover, cutover, cutover,
);
// Direction of the pre-cutover history, which is the whole argument for
// flooring rather than carrying it: a corte carries a NET, and a net built
// from receipts whose charges were never migrated is not one.
const [split] = await prisma.$queryRawUnsafe(
`
WITH nobf AS (
SELECT c.id FROM customers c
WHERE EXISTS (SELECT 1 FROM transactions t WHERE t.customerId = c.id AND t.voidedAt IS NULL)
AND NOT EXISTS (
SELECT 1 FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.customerId = c.id AND t.voidedAt IS NULL AND ${BF_PREDICATE}
)
)
SELECT SUM(t.amount < 0) AS charges, ROUND(SUM(CASE WHEN t.amount < 0 THEN t.amount ELSE 0 END), 2) AS chargeMxn,
SUM(t.amount > 0) AS credits, ROUND(SUM(CASE WHEN t.amount > 0 THEN t.amount ELSE 0 END), 2) AS creditMxn
FROM transactions t JOIN nobf n ON n.id = t.customerId
WHERE t.voidedAt IS NULL AND t.transactionDate < ?
`,
cutover,
);
// What the floorless population's balance is actually MADE OF.
//
// "Floor them, never carry" was written when this group looked like
// utilities cash receipts whose charges were never migrated. It is not that
// any more. After the prior-period import the group is 99 customers, and
// almost all of them are insurance-only — their rows come from the seguros
// database's own EFECTIVO, which is that line's ONLY ledger. Nothing posts
// it a second time, so flooring it does not remove a double count, it
// deletes receipts. Split the two so the remedy is chosen per population
// rather than for the group.
const [floorlessMix] = await prisma.$queryRawUnsafe(
`
WITH nobf AS (
SELECT c.id FROM customers c
WHERE EXISTS (SELECT 1 FROM transactions t WHERE t.customerId = c.id AND t.voidedAt IS NULL)
AND NOT EXISTS (
SELECT 1 FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.customerId = c.id AND t.voidedAt IS NULL AND ${BF_PREDICATE}
)
)
SELECT
COUNT(DISTINCT CASE WHEN t.legacySourceDb = 'SEGUROS 16_be' THEN t.customerId END) AS insCusts,
ROUND(SUM(CASE WHEN t.legacySourceDb = 'SEGUROS 16_be' AND t.currency='MXN'
THEN t.amount ELSE 0 END), 2) AS insMxn,
ROUND(SUM(CASE WHEN t.legacySourceDb = 'SEGUROS 16_be' AND t.currency='USD'
THEN t.amount ELSE 0 END), 2) AS insUsd,
COUNT(DISTINCT CASE WHEN t.legacySourceDb <> 'SEGUROS 16_be' THEN t.customerId END) AS utilCusts
FROM transactions t JOIN nobf n ON n.id = t.customerId
WHERE t.voidedAt IS NULL AND t.outstanding = 0 AND t.transactionDate < ?
`,
cutover,
);
// ---- section B: double-booked receipts ---------------------------------
const pairs = await prisma.$queryRawUnsafe(
`
SELECT d.id AS datos2Id, e.id AS efectivoId, c.name,
DATE(d.transactionDate) AS datos2Date, DATE(e.transactionDate) AS efectivoDate,
d.amount AS datos2Amount, d.currency AS datos2Currency,
e.amount AS efectivoAmount, e.currency AS efectivoCurrency,
d.reference AS datos2Ref, e.reference AS efectivoRef,
fx.lo AS rateLo, fx.hi AS rateHi
FROM transactions d
JOIN transactions e
ON e.customerId = d.customerId
AND e.legacySourceTable = 'EFECTIVO'
AND e.voidedAt IS NULL
AND e.reference = SUBSTRING(d.reference, 2)
JOIN customers c ON c.id = d.customerId
LEFT JOIN (
SELECT YEAR(effectiveDate) AS y, MIN(rate) AS lo, MAX(rate) AS hi
FROM exchange_rates GROUP BY YEAR(effectiveDate)
) fx ON fx.y = YEAR(d.transactionDate)
WHERE d.voidedAt IS NULL
AND d.legacySourceTable = 'datos2'
AND d.reference REGEXP '^C[0-9]+$'
ORDER BY d.transactionDate
`,
);
// Pass two — the leads pass one could not follow. Same shape of row, judged
// by the same money rules below, so a folio typo costs nothing.
const nearby = await prisma.$queryRawUnsafe(
`
SELECT d.id AS datos2Id, e.id AS efectivoId, c.name,
DATE(d.transactionDate) AS datos2Date, DATE(e.transactionDate) AS efectivoDate,
d.amount AS datos2Amount, d.currency AS datos2Currency,
e.amount AS efectivoAmount, e.currency AS efectivoCurrency,
d.reference AS datos2Ref, e.reference AS efectivoRef,
fx.lo AS rateLo, fx.hi AS rateHi
FROM transactions d
JOIN transactions e
ON e.customerId = d.customerId AND e.legacySourceTable = 'EFECTIVO'
AND e.voidedAt IS NULL AND e.amount > 0
AND ABS(DATEDIFF(e.transactionDate, d.transactionDate)) <= 3
JOIN customers c ON c.id = d.customerId
LEFT JOIN (
SELECT YEAR(effectiveDate) AS y, MIN(rate) AS lo, MAX(rate) AS hi
FROM exchange_rates GROUP BY YEAR(effectiveDate)
) fx ON fx.y = YEAR(d.transactionDate)
WHERE d.voidedAt IS NULL AND d.legacySourceTable = 'datos2'
AND d.reference REGEXP '^C[0-9]+$'
AND NOT EXISTS (
SELECT 1 FROM transactions x
WHERE x.customerId = d.customerId AND x.legacySourceTable = 'EFECTIVO'
AND x.voidedAt IS NULL AND x.reference = SUBSTRING(d.reference, 2)
)
ORDER BY d.transactionDate
`,
);
const [unpaired] = await prisma.$queryRawUnsafe(
`
SELECT COUNT(*) AS n
FROM transactions d
WHERE d.voidedAt IS NULL AND d.legacySourceTable = 'datos2'
AND d.reference REGEXP '^C[0-9]+$'
AND NOT EXISTS (
SELECT 1 FROM transactions e
WHERE e.customerId = d.customerId AND e.legacySourceTable = 'EFECTIVO'
AND e.voidedAt IS NULL AND e.reference = SUBSTRING(d.reference, 2)
)
`,
);
// REGRESSION GUARD. Since 1.0.26 the application drops the whole cash
// journal from every balance, so none of section B's rows should reach one
// any more. This counts the ones that still do: it is 0 while the exclusion
// holds, and goes non-zero the moment someone reintroduces a balance query
// that forgets it. A defect list that cannot tell you whether the defect is
// still live is just history.
const [stillCounted] = await prisma.$queryRawUnsafe(
`
WITH bfloor AS (
SELECT t.customerId, MAX(t.transactionDate) AS floorDate
FROM transactions t LEFT JOIN type_transactions tt ON tt.id = t.typeId
WHERE t.voidedAt IS NULL AND ${BF_PREDICATE}
GROUP BY t.customerId
)
SELECT COUNT(*) AS n,
ROUND(SUM(CASE WHEN t.currency='MXN' THEN t.amount ELSE 0 END), 2) AS mxn,
ROUND(SUM(CASE WHEN t.currency='USD' THEN t.amount ELSE 0 END), 2) AS usd
FROM transactions t
LEFT JOIN bfloor b ON b.customerId = t.customerId
WHERE t.voidedAt IS NULL AND t.outstanding = 0
AND t.legacySourceDb = 'UTILITIES'
AND t.legacySourceTable IN
('EFECTIVO', 'EFECTIVO_BACKUP', 'EFECTIVO FM3', 'CHEQUE FM3', 'IVA 2015')
AND (b.floorDate IS NULL OR t.transactionDate >= b.floorDate)
AND ${READ_SCOPE}
`,
);
// ---- CSV escapes -------------------------------------------------------
if (args.includes("--csv-a")) return dumpCsv(floorless);
if (args.includes("--csv-b")) return dumpCsv([...pairs, ...nearby]);
// ---- report ------------------------------------------------------------
console.log("\nBOOK (voided and outstanding rows excluded)");
console.log(` raw lifetime sum, no floor ${money(book.rawMxn)} MXN ${money(book.rawUsd)} USD`);
console.log(` BF floor only (pre-1.0.26) ${money(book.floorOnlyMxn)} MXN ${money(book.floorOnlyUsd)} USD`);
console.log(` TODAY, as the app computes it ${money(book.todayMxn)} MXN ${money(book.todayUsd)} USD`);
console.log(` flat floor at ${cutover} ${money(book.flooredMxn)} MXN ${money(book.flooredUsd)} USD`);
console.log(
" (the middle line is the floor alone, kept only so older runs of this\n" +
" script still diff against something. The app has dropped the cash\n" +
" journal and windowed the archives since 1.0.26; USD going to zero on\n" +
" the utilities side is correct, that ledger is peso-denominated.)",
);
const preRowsTotal = floorless.reduce((s, r) => s + d(r.preRows), 0);
const wouldZero = floorless.filter((r) => d(r.postRows) === 0);
const deltaMxn = floorless.reduce((s, r) => s + (d(r.todayMxn) - d(r.afterMxn)), 0);
const deltaUsd = floorless.reduce((s, r) => s + (d(r.todayUsd) - d(r.afterUsd)), 0);
console.log(`\nA. FLOORLESS CUSTOMERS — ${floorless.length}`);
console.log(` pre-cutover rows they still count: ${preRowsTotal}`);
console.log(` of those rows: ${d(split.charges)} charges (${d(split.chargeMxn).toFixed(2)})` +
` vs ${d(split.credits)} credits (${d(split.creditMxn).toFixed(2)})`);
console.log(` balance moved by flooring: ${money(-deltaMxn)} MXN ${money(-deltaUsd)} USD`);
console.log(` customers left with NO rows at all after the cut: ${wouldZero.length}` +
` (their balance becomes 0 — an assertion, not a migrated figure)`);
console.log(
` of the ${floorless.length}: ${d(floorlessMix.insCusts)} carry insurance-line cash` +
` (${d(floorlessMix.insMxn).toFixed(2)} MXN / ${d(floorlessMix.insUsd).toFixed(2)} USD)` +
`, ${d(floorlessMix.utilCusts)} carry utilities rows`,
);
console.log(
` READ THAT LINE BEFORE FLOORING ANYONE. The seguros EFECTIVO is that\n` +
` line's only ledger — nothing posts it twice — so flooring those\n` +
` customers deletes receipts rather than removing a double count.\n` +
` The argument for flooring holds for the utilities rows alone.`,
);
console.log(
`\n ${"customer".padEnd(30)} ${"pre".padStart(4)} ${"post".padStart(4)}` +
` ${"today MXN".padStart(13)} ${"after MXN".padStart(13)} ${"first tx".padStart(10)}`,
);
for (const r of floorless.slice(0, limit)) {
console.log(
` ${(r.name || "(sin nombre)").slice(0, 30).padEnd(30)}` +
` ${String(d(r.preRows)).padStart(4)} ${String(d(r.postRows)).padStart(4)}` +
` ${money(r.todayMxn)} ${money(r.afterMxn)} ${day(r.firstTx).padStart(10)}`,
);
}
if (floorless.length > limit) console.log(` ... ${floorless.length - limit} more (--csv-a)`);
// A folio match is a hypothesis; the money is the evidence. The band is
// widened by 10% either side of what exchange_rates observed that year,
// because the office keys receipts at its own counter rate, not at a
// published one, and a pair should not be called false over a few centavos.
const classify = (p) => {
const dAmt = d(p.datos2Amount);
const eAmt = d(p.efectivoAmount);
if (p.datos2Currency === p.efectivoCurrency) {
return Math.abs(dAmt - eAmt) < 0.005 ? "confirmed" : "suspect";
}
if (p.efectivoCurrency !== "USD" || p.datos2Currency !== "MXN") return "suspect";
if (!eAmt || !p.rateLo) return "suspect";
const implied = dAmt / eAmt;
return implied >= d(p.rateLo) * 0.9 && implied <= d(p.rateHi) * 1.1
? "confirmed"
: "suspect";
};
for (const p of pairs) {
p.pass = "folio";
p.verdict = classify(p);
}
for (const p of nearby) {
p.pass = "proximity";
p.verdict = classify(p);
}
pairs.push(...nearby);
const confirmed = pairs.filter((p) => p.verdict === "confirmed");
const suspect = pairs.filter((p) => p.verdict === "suspect");
const byCust = new Set(confirmed.map((p) => p.name));
const sameCur = confirmed.filter((p) => p.datos2Currency === p.efectivoCurrency);
const converted = confirmed.filter((p) => p.efectivoCurrency === "USD" && p.datos2Currency === "MXN");
const efecMxn = confirmed.reduce((s, p) => s + (p.efectivoCurrency === "MXN" ? d(p.efectivoAmount) : 0), 0);
const efecUsd = confirmed.reduce((s, p) => s + (p.efectivoCurrency === "USD" ? d(p.efectivoAmount) : 0), 0);
console.log(`\nB. DOUBLE-BOOKED RECEIPTS — ${confirmed.length} confirmed pairs across ${byCust.size} customers`);
const byFolio = confirmed.filter((p) => p.pass === "folio").length;
console.log(` candidates examined: ${pairs.length} (confirmed ${confirmed.length}, rejected on money ${suspect.length})`);
console.log(` found by folio cross-reference: ${byFolio}, by proximity after a folio miss: ${confirmed.length - byFolio}`);
console.log(` confirmed same-currency, amount equal to the cent: ${sameCur.length}`);
console.log(` confirmed USD receipt posted to datos2 in MXN: ${converted.length}`);
console.log(` datos2 C-refs with no EFECTIVO partner at all: ${d(unpaired.n)}`);
console.log(` EFECTIVO side of the confirmed pairs: ${money(efecMxn)} MXN ${money(efecUsd)} USD`);
console.log(
` still reaching a balance after the 1.0.26 exclusion: ${d(stillCounted.n)} rows` +
` (${d(stillCounted.mxn).toFixed(2)} MXN / ${d(stillCounted.usd).toFixed(2)} USD)` +
`${d(stillCounted.n) === 0 ? " <- 0 is the passing value" : " <- REGRESSION"}`,
);
console.log(
` The rows below still exist and always will; the ledger's own C<folio>\n` +
` posting is the copy that counts. This section is now a record of what\n` +
` the journal holds, not a list of money being double-counted.`,
);
console.log(
`\n ${"customer".padEnd(28)} ${"datos2".padStart(10)} ${"efectivo".padStart(10)}` +
` ${"datos2 amt".padStart(13)} ${"efectivo amt".padStart(13)} ref`,
);
for (const p of confirmed.slice(0, limit)) {
console.log(
` ${(p.name || "(sin nombre)").slice(0, 28).padEnd(28)}` +
` ${day(p.datos2Date).padStart(10)} ${day(p.efectivoDate).padStart(10)}` +
` ${money(p.datos2Amount)} ${p.datos2Currency}` +
` ${money(p.efectivoAmount)} ${p.efectivoCurrency} ${p.datos2Ref}/${p.efectivoRef}`,
);
}
if (confirmed.length > limit) console.log(` ... ${confirmed.length - limit} more (--csv-b)`);
if (suspect.length) {
console.log(`\n REJECTED — folio matched, money did not. Not duplicates on this evidence:`);
for (const p of suspect.slice(0, limit)) {
const implied =
d(p.efectivoAmount) && p.datos2Currency !== p.efectivoCurrency
? ` implied ${(d(p.datos2Amount) / d(p.efectivoAmount)).toFixed(2)}`
: "";
console.log(
` ${(p.name || "(sin nombre)").slice(0, 28).padEnd(28)}` +
` ${day(p.datos2Date).padStart(10)} ${day(p.efectivoDate).padStart(10)}` +
` ${money(p.datos2Amount)} ${p.datos2Currency}` +
` ${money(p.efectivoAmount)} ${p.efectivoCurrency} ${p.datos2Ref}${implied}`,
);
}
if (suspect.length > limit) console.log(` ... ${suspect.length - limit} more (--csv-b)`);
}
console.log(
"\nNOTE: nothing above has been changed. Section A is a proposal to move the\n" +
"floor, not a carried-forward balance: the pre-cutover charge ledger was\n" +
"never migrated, so no true opening balance can be computed from this\n" +
"database. It exists in the DreamHost per-year tables. Section B is no\n" +
"longer an open defect — it was fixed read-side in 1.0.26 — and the line\n" +
"that matters there is the regression count, which must stay at 0.",
);
} finally {
await prisma.$disconnect();
}
}
function dumpCsv(rows) {
if (!rows.length) return;
const cols = Object.keys(rows[0]);
console.log(cols.join(","));
for (const r of rows) {
console.log(cols.map((c) => JSON.stringify(r[c] == null ? "" : String(r[c]))).join(","));
}
}
main().catch((err) => {
console.error(err);
process.exit(1);
});