75dcbc11b801188ccc403311702900980b5968f4
100
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  rmancinasandClaude Opus 5
|
75dcbc11b8 |
feat(billing): audit the two populations the balance floor misses
Read-only. Reports what a corte would touch without touching it. The platform inherited legacy's BALANCE FORWARD rows (1,170, all dated 2026-01-01) but not the yearly process that produces them, and BillingService floors balances per customer on those rows. Two populations fall outside that floor, and they are unrelated defects that happen to surface as the same symptom — a customer whose balance reads as a credit the office does not owe. A. 102 customers have no BALANCE FORWARD row, so their balance is a raw lifetime sum. Only the current-year charge ledger (datos2) was migrated; the per-year charge tables stayed in DreamHost. What survives before the cutover is the EFECTIVO cash journal, and it shows: 199 of their 201 pre-cutover rows are credits. Flooring them at 2026-01-01 moves the book by -82,297.78 MXN and -52,020.20 USD. 91 of the 102 are left with no rows at all, so their balance becomes zero — an assertion, not a figure recovered from anywhere, which is why this script proposes and does not apply. B. 276 cash receipts booked twice in 2026 — once in EFECTIVO under folio N, once in datos2 as reference CN — across 120 customers, 986,240.00 MXN and 88,252.00 USD on the EFECTIVO side. Both rows sit after the floor so both count. The statement already hides them via STATEMENT_EXCLUDED_SOURCE_TABLES; the balances worklist, the movement browser and the /clientes/:id card do not. The folio alone does not establish a pair — folios are reused. Every pair is corroborated on money too: equal amounts when both legs share a currency, or an implied USD->MXN rate inside the band exchange_rates observed that year. Two of 278 folio matches fail that test and are reported apart rather than counted, both same-day partial applications that need a human. Matches the balance-forward row in both shapes, as numid.service.ts does. Databases imported before the type was minted carry those rows with typeId NULL, and name-only matching reports every customer as floorless on such a copy — including the dev database, which the API itself currently reads as +20,653,109.15 MXN against a floored -9,194.61. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
9929a9a3ac |
feat(customers): the customer file's estado de cuenta follows the same year rule
`/clientes/:id` carries its own "Estado de cuenta" card, fed by CustomersService.detail rather than by BillingService.statement, so the previous commit left it reading the old way: the last 100 movements of all time, newest first. Two pages, one label, two orders. It now covers the current calendar year oldest-first, like the statement and like the sheet the office prints. The `take: 100` is gone with it — capping a descending list hid the oldest rows, but capping an ascending one would hide the newest, and a single year is small (365 rows for the heaviest customer in the book). The per-domain totals above the list are untouched and still historical: they are an unfloored groupBy, so they double-count every customer's opening balance. That is a separate bug from this one; the card now says out loud that those figures are lifetime, not this year's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
bc749055e7 |
feat(statements): scope the estado de cuenta to the current year, oldest-first
The office's EDO CUENTA sheet has always been a *year* statement: a balance
forward line dated January 1st, then that year's movements in the order they
happened. Both of ours read the other way — every year the customer ever had,
newest first — so staff comparing the screen against the printed sheet were
reading two different documents.
Movements are now bounded to the calendar year and returned ascending, on the
screen (/estado-cuenta/[id]) and in the printable `edo-cuenta-datos` report
alike.
Earlier rows are dropped from the *list*, not from the arithmetic. The balance
floor normally lands on January 1st already, so for most customers nothing
extra is dropped at all; when it doesn't — a customer the last legacy publish
skipped, or one that never had an opening balance — the earlier rows are
folded into a carried balance and shown as a single "saldo anterior" line.
Discarding them instead would restart every balance at zero on January 1st and
nothing would throw; the numbers would just be wrong, which is how the
double-counting bug survived for years. `opening` is exposed per currency and
per business line so the totals still reconcile against the last running
balance printed.
Two things the report was missing on its own are fixed while it is being
touched, since it must agree with the screen to the peso:
- it never applied the balance floor, so every pre-cutover row was counted
twice — once inside the opening balance and once as itself;
- its source-table exclusion used a bare `notIn`, and `NULL NOT IN (...)` is
NULL rather than true, so every app-captured row (which has no
legacySourceTable) silently vanished from the printout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
75e9f582b4 |
feat(policy-ocr): store the IVA A.N.A. already prints
The parser has been reading A.N.A.'s `TAX` cell since the ANA layout landed,
but `ParsedPolicy` had nowhere to put it, so the figure only ever reached a
review note and every OCR-confirmed policy was written with `tax` null — even
though the paper states it.
`TAX` now flows parser -> `extractedTax` -> `Policy.tax`, alongside the premium
fields beside it. GMX stays null: its certificate carries no premium at all, so
there is no tax on it either.
The other two cells stay out, for reasons worth keeping:
- `LOCAL TAX` is a separate levy with no destination column, and summing it
into `tax` would produce an IVA that no longer divides back to a rate —
which is the whole reason to store the figure. It reads 0.00 on every
policy seen so far; a non-zero one now raises a note saying the total will
not reconcile, instead of quietly inflating the IVA.
- `DISCOUNT` has no column and prints as a bare "-" when unused, which is
what makes the row positional rather than "find six amounts".
`taxRate` is left null by confirm. A.N.A. prints the amount, not the rate, and
back-dividing it would mint a rate the document never stated; the capture form
resolves one from the line of business instead.
The review screen gains derecho de póliza next to the new IVA field. It was
already parsed and already written on confirm, but never shown — and an IVA
with no fee beside it leaves the reviewer unable to see why premium + fee + tax
equals the printed total.
The spec's assertion moved off the note and onto the field, plus a check that
the row reconciles: 298.61 + 30.00 at 8% is 26.29, totalling 354.90. That
agreement is what proves the positional mapping landed on the right cells
rather than merely on six numbers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
48e01ddd21 |
feat(policies): capture the full premium breakdown
The capture form only ever had prima neta, derecho de póliza and comisión.
The Access form it replaces has seven figures, and the four that were missing
are the ones that make a policy paid in installments add up.
Adds recargo, IVA, prima total and forma de pago to the policy header, the
same breakdown per installment, and a per-line-of-business IVA rate.
IVA and prima total are the only derived figures:
base = prima neta + recargo + derecho de póliza
IVA = round(base * tasa)
total = base + IVA
The recargo is inside the taxable base. That is not a guess — policy 7006785
prints IVA 52.03 on 610.86 + 8.55 + 31.00, and leaving the recargo out gives
51.35, which matches nothing on the page. Both of its money rows are asserted
in premium.spec.ts. The recargo itself is never derived: the carrier quotes it,
so staff key it in, and the field is disabled on ANNUAL/SINGLE. Both derived
figures are stored rather than recomputed on read, and stay editable, because
the printed policy is the record of truth and a later rate change must not
silently restate what was issued.
The rate lives on PolicyType (seeded to 0.08, editable in Catálogos), which is
the legacy one-row IMPUESTOS / IMPUESTOS_AUTOS tables made configurable. The
rate applied is stamped on the policy so an old one reads back at its original
rate.
Per-installment, not two fixed slots on the header: a policy split into several
exhibiciones prices each payment separately — that is why the Access form drew
the money row twice — and a trimestral policy needs four, which the Access
layout could not hold.
Also fixes two losses in the ETL, which is how these went missing:
- `forma_pago` was marked consumed by the coverage sweep and then never
written to any column, so FORMA PAGO existed nowhere in the platform.
- `recargo` and the whole second money row fell into `coveragesJson` as
loose strings, mislabeled as coverage amounts.
transform_policies.py now writes all of it directly;
backfill_policy_premium_breakdown.py recovers it on a database that must not be
re-imported, and strips the migrated keys back out of coveragesJson. Both are
COALESCE-only, so a figure a human has corrected in the app wins.
IVA and TOTAL are NOT backfilled: they were unbound calculated controls on the
Access form, never columns, so there is nothing to recover and every migrated
policy reads null until it is edited.
The backfill warns on 5 annual policies that carry a non-zero recargo — a
contradiction that predates this change and is left for a human, not silently
corrected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
4f2f064955 |
fix(web): OCR batch review returns to capture, not the policy list
"Volver a pólizas" on /polizas/captura/[id] dropped the reviewer at /polizas, so getting back to the batch queue meant navigating in again. Points at /polizas/captura and reads "Volver a captura", matching the statement review screen, which returns to /recibos. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
2f99bd5f98 |
fix(web): define the layout utilities the screens were already using
The policy OCR review header read "Para revisarPágina 1700489616· PAMELA DENISE WAGONERLICENCIASANA". `.row`, `.stack`, `.tag`, `.page-sub` and `.state-warn` are used across the app but no rule ever defined them. Without `display: flex` the `gap` those call sites pass does nothing, and JSX drops the newline between sibling elements, so the header's spans concatenated. `.tag` rendered as prose rather than as a chip for the same reason. Defined against the existing design tokens: `.tag` takes `.badge`'s shape, `.state-warn` is the gold sibling of `.state-error`, and `p.page-sub` keeps the block margin while the inline form drops it so a flex row still centres. Also dropped the hand-rolled "· " separator and margin from the OCR header — the flex gap does that now, and the literal dot was left floating in it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
81938877ed |
feat(policy-ocr): suggest the customer from the printed insured name
The office books customers surname-first ("WAGONER, PAMELA") and carriers
print them given-name-first ("PAMELA DENISE WAGONER"), so the review screen
made staff retype a name the parser had already read. Comparing normalized
token sets makes the two orderings the same thing.
Only on the zero-hit path, where the policy number found nothing and a human
has to pick a customer anyway. The suggestions are written to a new
`customerSuggestions` column rather than `matchCandidates`, which the review
screen reads as policy-number hits, and they never set `matchedCustomerId` or
`confident` — matching on `Policy.policyNumber` is unchanged.
Two tiers, drawn where the real book has cliffs: EXACT (identical token sets)
and PARTIAL (containment, >=2 shared tokens, surname present). Of 1536
customers, 1487 have a distinct token set, so EXACT cross-person collisions
are ~0; loosen to surname + first given name and 131 (8.5%) collide, and 185
surnames are shared by 524 customers, which is why one token is never enough
and the surname must be printed explicitly. Replaying every book row as a
carrier would print it: 97.9% top-ranked correct, 1.2% a different row, all
but two of those the same human on a duplicate or variant row.
Normalization folds accents (OCR's MUNOZ reaches the book's MUÑOZ), drops
initials, Spanish particles, JR/S.A. DE C.V., and any token with a digit —
ANA prints the phone hard against the name as `Ph.3102001538`. Names over 8
tokens or 80 characters are refused outright, because GMX's especificación
has no field labels and the parser has handed its whole first page over as
`insuredName`.
Not used for utility statements: there the registrant genuinely is not the
customer, so the same trick would be wrong rather than noisy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
19864f16f2 |
fix(ocr): keep the printed layout when rebuilding text from word boxes
The parsers are written against `pdftotext -layout`, and every policy-ocr
fixture is a verbatim excerpt of it. The runtime does not use it: it reads
`-bbox-layout` and rebuilds the page from word boxes, and that rebuild
collapsed all white space — no blank lines between blocks, one space
between columns. White space is the only thing marking a cell boundary on
these borderless forms, so the fixtures could not see any of it.
What it cost, on the GMX PVL especificación and the ANA driver's policy:
- `espectBlock` walks a wrapped cell until a blank line. With no blank
line it ran to the end of the page, so the insured's name came back as
the entire first page of the specification.
- `INSURED\s{2,}` and its siblings matched nothing, and the phone that
shares the name cell rode along with it ("PAMELA DENISE WAGONER
Ph.3102001538"), which matches no customer.
- `parseAnaDriverCoverages` splits SUM INSURED from PREMIUM by the
header's own column offsets. Without offsets, every premium was filed
as a sum insured.
So `toVisualRows` now emits a blank line where the reader sees one (a
vertical gap over 1.6 line heights — the two populations measure 0.3-1.1
and 2.1+, so the threshold sits in empty space) and pads each word to its
own column, using one space wherever words merely follow each other so
rounding drift cannot sprinkle false cell boundaries through prose.
Two independent guards, so neither failure can come back silently: the
ANA phone splits on a single space, and the especificación's cell walk is
capped at the one wrap the longest cell on that document actually uses.
Verified against the real PDFs: the especificación reads "EMMER .
KATHLEEN" with all 18 coverages named (they were "(sin nombre)"), and the
ten born-digital gas invoices parse byte-identically to before. The
scanned statements are untouched — they come through tesseract, not this
path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
022d1935ad |
feat(policy-ocr): set policyTypeId and insuranceProviderId on confirm
The BACKLOG claimed this was blocked on incomplete `policy_types` rows.
Querying the dev database says otherwise: AUTO (1316 policies) and LICENCIAS
(306) are both live and healthy, so ANA's two faces were never blocked at
all. Three separate things had been conflated.
What the parser now emits is a NAME, not an id -- it is a pure function over
text and must not reach for the database:
ANA AUTOMOBILE -> AUTO
ANA DRIVER'S POLICY -> LICENCIAS
GMX (both documents) -> MULT
`resolveLookups()` turns that into a foreign key at confirm, and does the
same for the carrier off the parser's provider code. It resolves, never
creates: a missing `policy_types` row means a human deleted it, and silently
recreating it would undo that with no record. An explicit `policyTypeId` /
`insuranceProviderId` on the confirm payload always wins.
GMX is MULT rather than INCENDIO because the caratula's own header reads
"Multiple Policy / Home" and the especificación is "PVL Hogar" -- one product,
two artifacts. MULT is the live row carrying 769 of them; INCENDIO is fire-only
and no policy in the book has ever used it.
The parser's provider code is not the carrier's row name, so PROVIDER_ROW_NAME
maps ANA onto "ANA SEGUROS", which is where the office's 738 ANA policies
already are.
--- the actual defect underneath -----------------------------------------
`policies.policyTypeId`, `policies.insuranceProviderId` and
`claims.adjusterId` are all ON DELETE SET NULL, and the lookups screen deleted
unconditionally. So deleting a lookup row returned 200 and silently blanked
the field on every row referencing it -- no error, nothing in the UI. That is
how M_EMPR disappeared and left 5 policies with no ramo, found months later
only by querying.
All three deletes now refuse while the row is in use, naming it and the count
("El tipo de póliza «M_EMPR» está en uso por 5 póliza(s)"). The schema-level
`onDelete: Restrict` the spec once recommended is deliberately not used: a raw
FK error is not something the operator can act on.
`20260815160000_policy_type_repair` cleans up what already happened:
- restores M_EMPR and re-points its 5 policies, scoped to
`policyTypeId IS NULL AND legacySourceTable = 'm_empr'` so it can never
claim a policy blanked for some other reason
- merges the duplicate "ANA" carrier (1 policy) into "ANA SEGUROS" (738).
OCR is about to start assigning the carrier automatically and two rows
would keep splitting the book. Written as joins, not subqueries, so both
statements are no-ops when either row is absent -- a subquery form would
resolve to NULL and blank the carrier off every ANA policy.
- does NOT restore INCENDIO. It is the other row the migration would have
produced, but the legacy INCENDIO table has 1 row that never loaded, so
the type has zero policies and restoring it would only put a dead option
in the type picker.
Verified by running the repair against the real broken dev data inside a
transaction and rolling back: 5 orphans -> 0, ANA/ANA SEGUROS -> one row with
739, and a second run in the same transaction changes nothing. The DDL half
matches `prisma migrate diff` exactly.
186 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
5a277f4885 |
feat(deploy): apply migrations at api container start
`prisma migrate deploy` ran in one place only: a workflow step on the Gitea
runner, which has to reach the target host's MySQL on 3306 directly. Two
paths went around it:
- `skip_migrate=true`, the documented answer for when the runner cannot
reach 3306, left the schema a release behind with nothing to catch it.
The mismatch surfaced later as a column-not-found at runtime rather than
as a failed deploy.
- A container brought back by `restart: unless-stopped` after a host
reboot, or a stack re-applied by hand in Portainer, never runs the
workflow at all.
docker/api-entrypoint.sh becomes the api image's ENTRYPOINT: migrate, then
exec node. If the migration fails the container exits non-zero and the API
never listens — serving against a schema that does not match the code is
worse than being down, because the failures are partial and silent (a write
to a missing column breaks one feature while the rest looks healthy).
This does not replace the workflow step and is not a substitute for it. That
step still runs FIRST, while the old code is serving, which is the order
expand/contract migrations are designed around. `migrate deploy` is
idempotent, so on the normal path the container's run is a no-op query.
Behaviour:
RUN_MIGRATIONS=false skip and start anyway; plumbed through both app
stack files, for a schema moved by hand
DATABASE_URL unset refuse to start, and say why
P1001 (unreachable) retry, default 20 x 3s -- a cold db container, and
galactus's MagicDNS lookup right after a reboot
anything else exit at once; retrying a broken migration only
delays the same error. P3005 prints the
`migrate resolve --applied 0000_init` hint the
workflow step already printed.
Only P1001 retries, so a genuinely broken migration is not buried under a
minute of noise.
Both stacks are replicas: 1 and must stay so for an unrelated reason (the
servicios email sweep has no DB lock). The old comment claiming migrations
must not run per-container because "N replicas would race" is dropped: they
would not corrupt anything, since Prisma takes a database advisory lock and
the losers find nothing pending -- they would only each pay the wait.
The prisma CLI is already in the runtime layer (the image copies
/repo/node_modules wholesale), but which of the two plausible .bin paths
carries it is an implementation detail of pnpm's hoisted linker, so the
entrypoint accepts either and the Dockerfile asserts one exists at BUILD
time. A missing CLI breaks the image build, not a production boot.
Verified by running the entrypoint against stubbed prisma binaries: clean
run, P3005, P1001-to-exhaustion, P1001-then-recovery, RUN_MIGRATIONS=false,
missing DATABASE_URL, missing CLI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
d645ba51d3 |
feat(policy-ocr): read A.N.A. Seguros' two policy faces
A.N.A. is the Rosarito office's tourist auto book and the second carrier
the policy OCR pipeline reads. It ships two unrelated faces, and the split
is different from GMX's: GMX ships two documents about one policy, A.N.A.
ships two products.
AUTOMOBILE (SPECIAL POLICY FOR TOURISTS) insures a car; vehicle table,
9 numbered sections, one
LIMIT OF LIABILITY column
DRIVER'S POLICY (the office: "licencia") insures up to 5 named drivers;
no vehicle at all, 6 unnumbered
sections in a different order,
SUM INSURED + PREMIUM columns
The four automobile products the office sells (amplia / responsabilidad
civil, annual / by-the-day) are the same layout with different numbers, so
they get one parser rather than four.
These are born-digital portal PDFs, so pdftotext -layout returns exact
columns and the driver's-policy parser uses that: its two value columns
print the same shape (100,000.00 usd. / 18.70 usd.) with no per-row label,
so horizontal position is the only thing separating them. The split comes
from the header's own offsets, not a constant, because they shift between
products; when it can't be read every amount is reported as a sum insured
and the reviewer is told, rather than half the premiums being filed as
coverage limits.
Three things the layout will punish a naive read for:
- Each PDF prints its face two or three times (ORIGINAL, AGENT COPY, then
a receipt and three travel cards) and the pipeline concatenates every
page before parsing. The coverage walk is bounded to the first copy and
the driver list to the first POLICY HOLDER block. Unbounded, the licencia
returns the same person three times, which reads as a three-driver policy
rather than as a bug.
- The money row is read positionally off its header. An unused DISCOUNT
prints as a bare "-", so "find the six amounts" shifts every value one
column left on a discounted policy.
- Two five-digit numbers sit in the header band and only one is the agent
clave; the agent's street address is "BENITO JUAREZ 25 No.50 INT 38",
three lines above the No. cell holding the policy number.
Sections 6-8 print a PREMIUM where the others print a limit, so
ParsedCoverage gains an optional `premium` (GMX never fills it) and the
review table a column: $40 is what legal aid cost, not a $40 liability
limit. Exclusions follow the GMX rule and go in the risk label with a null
amount -- which matters more here, since a responsabilidad-civil policy
prints 0.00 for material damage and the two are identical on the page.
Also in this change:
- coveragePeriodDays is parsed and written. A.N.A. sells 3- and 4-day
policies; Policy.coveragePeriodDays defaults to 365, so a weekend policy
left at the default sits in the renewals window a year out. Derived from
the dates, cross-checked against the printed DAYS cell, disagreement
noted not resolved.
- Vehicles and named drivers are parsed, shown read-only in review, and
written as Vehicle / InsuredDriver rows on confirm, skipping any already
on the policy (VIN then plate; licence then name). The case that forces
the skip is confirming a renewal onto an existing policy. Nothing is ever
updated or deleted -- a changed plate lands as a second row for a human.
- Batch.provider is set from what the parsers actually claimed instead of
being hardcoded "GMX", so a mixed upload is labelled as mixed and the
header can never contradict its own documents. PolicyDocument.documentType
follows the same rule (was hardcoded GMX_POLICY).
- matchNote becomes TEXT. It was VARCHAR(191) and the note trail was sliced
to 190 chars, which cut the tail notes -- the "could not read X" ones.
- The policy detail page renders an array coveragesJson as a table. Both
shapes have always been possible there, but the object renderer was the
only one, so an OCR-confirmed policy showed a row per array index labelled
"0", "1", "2" with [object Object] as the value. ANA makes that routine.
GMX is untouched behaviourally; its two parsers now spread a shared empty
base instead of listing every null field. 29 new parser cases against
verbatim pdftotext output of three real ANA PDFs, 53 in the suite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
14c4d44acb |
docs(policy-ocr): vigencia/agente/prima are keyed in by hand on the PVL layout
Confirmed with Luz, who handles GMX policies at the office: the three
fields the especificación does not carry are entered manually. The review
screen already supports it — all three are editable and `postPremium`
enables off the typed premium, so no code change was needed.
The parser's note said "esos datos están en la carátula de la póliza",
which now sends the reviewer looking for the wrong document. It says
"captúrelos a mano" instead, and names the consequence of leaving the
vigencia blank: `Policy.policyTo` is nullable and the renewals window
filters `policyTo: { gte, lte }`, so a policy confirmed without one never
matches and never gets a renewal notice — silently, permanently, with
nothing downstream erroring.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
45be0ad77d |
feat(policy-ocr): read GMX's Spanish PVL especificación layout
GMX ships two unrelated documents for the same policy and the office downloads both from the same portal. The parser only knew the English caratula, so a `…-CondicionesParticulares.pdf` parsed to an almost entirely empty row — including the policy number, which the matcher needs. `parseGmx` becomes a dispatcher over `parseGmxCaratula` (unchanged behaviour) and the new `parseGmxEspecificacion`. Both still report `provider: "GMX"`: the matcher keys on the policy number alone and must not care which artifact was uploaded. The especificación has no tables. Coverages are found by anchoring on `Límite … Responsabilidad:` and walking backwards for the heading, where a heading is a short line *preceded by a blank line* — length alone cannot tell one from the wrapped tail of the paragraph above it, and without that condition coverages get named after the last word of the preceding prose. Also fixed, both pre-existing: - The policy number's group widths are not the same across the two families (`007-037-…-0000-02` vs `07-037-…-00000-01`). The pinned-width regex is replaced by a shape, so both read. - The caratula's ZIP fallback pushed a note saying it had read the ZIP from the address, then never assigned it. Verified against the full ten-page real document: all 17 coverages, amounts, deductibles and the excluded earthquake section match what is printed. 24 parser tests (was 8), four of them regressions for ways this layout can silently attach the *wrong* value rather than none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinas
|
7be897ef2b | chore(release): v1.0.18 | ||
|
rmancinasandClaude Opus 5
|
cf40cd22ef |
fix(deploy): stop pinning API_ORIGIN, and probe one origin not the list
Two leftovers from making the browser derive the API origin. The stack env still injected API_ORIGIN from a repo secret, which pinned the origin again on every deploy and would have re-broken an https front door with mixed active content. Drop it from both env_data blocks; the secret stays, now purely as the URL the verify step probes. That verify step was also about to break on its own: WEB_ORIGIN is a comma-separated CORS list now, and `curl "$WEB_ORIGIN/version"` on a list retries thirty times and fails a deploy whose app is perfectly healthy. Probe the first entry, so keep the runner-reachable origin first in the secret. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinas
|
3b02c6944f | chore(release): v1.0.17 | ||
|
rmancinasandClaude Opus 5
|
683fd37b08 |
docs(deploy): stop documenting API_ORIGIN as required
The swarm stack still hard-failed on an unset API_ORIGIN, and both the env template and the README told the reader to pin it — the exact habit the derived origin was meant to end. Make it an optional override everywhere, and say that WEB_ORIGIN is now a list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
14c6183aa2 |
feat(deploy): derive the API origin from the page, not from a pinned env var
The browser hard-required API_ORIGIN, so every move of the server — tailnet today, the 192.168.1.0 office LAN later, a temporary demo domain in between — meant editing the deploy env and redeploying. Worse, an http:// API origin on a page served over TLS is blocked outright as mixed active content, which is what broke the demo on https://jorgecuadros.freakma.com. The browser now derives the origin from window.location the way a PHP app would: same host on port 3001 over plain HTTP, or the same-origin /api path under https (the reverse proxy strips the prefix). API_ORIGIN survives as an optional override for a deployment that genuinely splits the two hosts, and SSR still reads process.env because a derived origin is browser-only. WEB_ORIGIN becomes a comma-separated list to match: one deployment is now reached under several origins, and a credentialed fetch from an unlisted one gets no CORS headers and fails. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
2169ffa78d |
feat(ops): verify the replica against the master, not just its own status
Every field the replication card showed was self-reported by the replica, and the two most reassuring ones lie in the same failure. Seconds_Behind_Source reads 0 when the I/O thread is disconnected — with no incoming event there is nothing to measure staleness against — and Replica_IO_Running only says the network thread is alive, not that it is receiving. Two checks that ask the master instead: - GTID drift, folded into the polled status. GTID_SUBTRACT(master, replica) counts transactions the master executed that the replica has not, so a silent disconnect shows up as a number that climbs instead of a lag that stays 0. It also isolates transactions carried under the replica's OWN server UUID — writes that exist nowhere on the master. There are currently 518 of them, residue of the seed dump load; inert while log_replica_updates is off, and a real divergence the day anyone promotes that box. - A full row-by-row comparison behind a button, over the eight tables my.jorgecuadros.com reads. GTIDs prove the replica applied everything the master sent; they say nothing about rows changed here by another route, which is the one failure the rest of the card cannot see. The comparison hashes CONVERT(col USING binary), not CAST(col AS CHAR). CAST transcodes into the connection character set, and the two servers do not agree on it: the client inside the master's container negotiates latin1, the replica's utf8mb4. Every accented character in a Mexican name, street or note then hashes differently and the tool reports a permanent mismatch on exactly the tables that hold free text. Caught by building it and running it — customers.name gave 3344437324815 against 3339150372121 under CAST, and 3339150372121 on both under CONVERT. All eight tables now match byte for byte. Verify is POST and audited despite reading nothing: it full-scans both servers, so a prefetch or a refresh must not be able to start one. Tests cover the GTID interval arithmetic, which is inclusive at both ends and easy to get wrong by one in the direction that hides a gap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
17d83291c3 |
feat(migration): refuse a full re-import that would delete native rows
A full run_all.py pass truncates and rebuilds every table it owns from the Access extract. That was harmless while the platform was a read-only mirror -- every row came from the extract, so wiping and rebuilding lost nothing. It stopped being harmless once the platform started minting rows Access has never heard of: allocated portal NUMids, customers created in the staff UI, OCR-captured policies, app-booked ledger rows, uploaded documents. REIMPORT is a button in /operaciones, so that was one click away. native_guard.py counts what only exists here and exits 3; run_all.py runs it before the first truncate and stops. Detecting an allocated NUMid needs the staged Parquet -- the customer holds an ordinary-looking (utilities, DATGRAL, '1172') ref, so "customer has no refs" cannot see it and only comparing against the extract can. Missing staging is therefore treated as blocking rather than as "nothing to protect". The guard does not teach full mode to preserve anything: --sync already upserts legacy rows against the existing refs and leaves the rest alone, and rebuilding that inside full mode would re-implement it. --force-full (checkbox in the REIMPORT confirm, recorded in the audit log) deletes them deliberately. Verified against dev: clean before, exit 3 listing utilities/1172 with a synthetic ref present, clean again after removing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
6a97242fc3 |
feat(customers): allocate portal NUMids, with an audit for reusable ones
Customers created in the staff UI had no NUMid and so could not log in to my.jorgecuadros.com at all: the id is a CustomerLegacyRef row, not a column, and create() deliberately writes none. Allocation is a staff action (POST /customers/:id/portal-access, MANAGER) rather than part of create, because insurance is expected to move to the platform before utilities and an insurance-only customer has no reason to spend a utilities id. The audit that decides which ids are reusable took three passes. "Owns no rows" matches nobody -- migration gave all 1,171 NUMids a property and a transaction. "No transaction in N years" also matches nobody -- every customer carries a synthetic Jan-1 opening-balance row, so everyone looks active this year. Subtracting that row is what makes dormancy measurable, and it leaves 4 never-used ids and 10 dormant ones on dev. Two further traps are encoded in the queries: insurance/DATGRAL is a separate id space that reuses the sourceTable name and runs past 4,000, and ACCOUNT CANCELED is a transaction line type, not an account state -- all 8 customers carrying it have current-year activity. Recycling ships switched off (numid.recycleEmpty, default false). Every reusable id still exists in Access DATGRAL, and a --sync run reassigns refs with ON DUPLICATE KEY UPDATE customerId, so an id recycled before the utilities cutover is silently handed back to its Access owner. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
d173c9e9a0 |
fix(billing): stop double-counting history a BALANCE FORWARD already carries
BALANCE FORWARD rows are not movements. Access materialized one per customer per year, dated Jan 1, holding the closing balance of everything before it — that is what let the portal keep each year in its own table and still show a correct running balance from one year's rows. The platform imported those rows AND the real pre-cutover history they summarize, and every balance aggregate summed the lot. NUMid 501 read -10,469.29 on the receivables worklist against -14,065.29 on the customer's own statement and on the legacy portal; the gap was two cash receipts from 2009 and 2012 that the 2026 opening balance had already absorbed. The scale settles what it is: summed the old way the whole book came to +20,605,447.86 MXN — the office owing its customers 20.6 million pesos. Floored, it is -56,855.90. A receivables ledger cannot be 20M in credit. Adds BALANCE_FLOOR_JOIN + NOT_SUPERSEDED and applies them to balances() (page and count queries, which must agree), to stats()'s per-currency and per-domain figures, and to the owing/in-credit split. The four stats() aggregates moved from Prisma groupBy to raw SQL because groupBy cannot express a per-customer floor. statement() takes the same floor as a scalar, which is also what stops FEE ANUAL and fee15 leaking in. Those are not in STATEMENT_EXCLUDED_SOURCE_TABLES — that list reproduces legacy's DATOS2-only `datosfreak` — and they were putting 2,092 pre-cutover fee rows across 1,062 customers into the statement, skewing it by -5,129,764 against the number those customers have been quoted for years. Dating rather than source is the right test: a fee row *after* the opening balance is a real charge and still counts. movements() is deliberately left alone. It is a browser over captured rows — "how much water did we capture in April" — and staff need the historical rows visible, so it keeps totalling everything, the same asymmetry NOT_OUTSTANDING already has. stats() now separates the two questions it was mixing: movements, ledgerCustomers, crossLineCustomers and the date range stay unfloored inventory; everything under byCurrency/byDomain is a balance and is floored. BillingService had no tests. Adds 13 covering the floor's failure modes — it fails silently, so MIN-vs-MAX, `>` vs `>=`, the NULL branch for customers with no opening balance, and the join/predicate alias pairing are each pinned, plus the 501 arithmetic as a regression. Verified through the real service against the live ledger: balances() and statement() both return -14,065.29 for 501, matching the portal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
e9a5ee9e90 |
fix(migration): carry NOPAGO into transactions.outstanding
datosfreak's NOPAGO is the legacy "still owed" flag, and the website reads it directly — account.statement.php splits the statement on NOPAGO = 0 vs NOPAGO = 1 and renders the latter as "Outstanding Bills Requiring Attention". transform_transactions.py hardcoded 0, so all 40,421 rows came across settled and that section renders empty for anyone served off the platform. Not a missing column: a missing section, with no error. Only the three DATOS2-shaped tables carry the flag (76 rows set in datos2, 0 in FEE ANUAL and fee15); the EFECTIVO/FM3 cash streams have no such column and keep the 0 default. Sync mode gets outstanding=VALUES(...) too, so an additive sync corrects rows already loaded rather than leaving them settled forever. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
b12382b436 |
ci: move the galactus deploy chain to a tag-only workflow
The deploy was a job inside build.yml gated by `if: startsWith(github.ref, 'refs/tags/v')`. Gitea draws every job into the run graph before it evaluates that `if`, so an ordinary push to master showed a pending "Deploy to galactus" — indistinguishable from prod being about to be redeployed off an unreleased commit, and the only safe reaction is to cancel the run, which takes the images down with it. The gate itself was never wrong (no deploy-galactus run has ever been created from a branch ref), but a guarantee you cannot see is not much of a guarantee. `on: push: tags: ["v*"]` in a workflow of its own makes it structural: the deploy cannot appear on a master build because the workflow does not exist there. It replaces `needs: build` by polling the Actions API for the build.yml run at this tag and requiring it green, so both images are still known to be in the registry before anything is pulled. AUTO_DEPLOY_GALACTUS still cuts the chain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
ed19f51a52 |
fix(ops): re-stage before an additive sync
SYNC ran `run_all.py --sync` without `--stage`, so it depended on staged
Parquet under migration/output. That directory is part of the image, not a
volume, so any redeploy wiped it and the job died on the first transform:
FileNotFoundError: '/repo/migration/output/stg_utilities/datgral.parquet'
Re-staging is also what makes the job's own label true — without it a sync
would replay whatever upload staged last, not the files currently in the
ingest folder.
Staging now counts as a numbered step when it runs, so the Operaciones
progress bar moves during the slowest phase instead of sitting empty.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
e85db73dbc |
ci: deploy to galactus automatically when a tag build goes green
Cutting a release then had one manual step left: watch build.yml and dispatch "Deploy to galactus" by hand with the version. Chain it. build.yml gains a `deploy` job, `needs: build` and gated on refs/tags/v*, that dispatches deploy-galactus.yml against the tag with tag=<version> scope=app bootstrap=false skip_migrate=false. `needs` waits for both matrix legs, so api and web are both in the registry before prod pulls either — deploy-galactus.yml only pulls, and a half-pushed pair leaves prod running one new image and one old one. A dispatch rather than a `workflow_run:` trigger (which Gitea has supported since 1.24) because deploy-galactus.yml reads github.event.inputs.* in ten places; under workflow_run all of them are empty strings, so the deploy would run with no tag. The dispatch keeps that workflow's contract intact and keeps it hand-runnable, which is how rollbacks work. The dispatch is confirmed the same way release.yml confirms the build started: snapshot the existing deploy-galactus run ids first, then require a new one to appear. An accepted dispatch that creates no run is the failure mode that cost v1.0.3 its images, and a plain "is there a deploy run" check would be satisfied by the previous release. Kill switch: repo variable AUTO_DEPLOY_GALACTUS=false prints the manual command instead of deploying. Needs the existing RELEASE_TOKEN secret; preflight fails loudly and names the manual command if it is unset. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
fe761e119e |
feat(ops): show relay apply progress on the replication card
Seconds_Behind_Source cannot answer "is it moving?". While the SQL thread works through one large transaction the lag counter holds still — often at 0 — even though the replica is not caught up. The relay backlog does move, and it comes out of the SHOW REPLICA STATUS the panel already runs, so this costs no extra query and no connection to the source. Adds applyProgress(), which reads Source_Log_File / Read_Source_Log_Pos vs Relay_Source_Log_File / Exec_Source_Log_Pos and reports the fetched-but-not- applied byte delta plus a percentage. Both positions are source binlog coordinates, so they are only comparable while the two threads are on the same file; across files the delta is meaningless (positions restart at ~4 in each new file) and is reported as null rather than as a huge negative number. The percentage deliberately stops at 99.99 while any backlog remains — binlog positions are large enough that a real backlog of a few KB rounds to 100% and would render a lagging replica as caught up. Not folded into `healthy`: a non-zero backlog is the normal state of a working replica between fetch and apply, so alarming on it would cry wolf. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
66d0d071b0 |
feat(ops): show step progress for reimport and sync jobs
A REIMPORT takes ~110 seconds and, until now, showed only a scrolling log — there was no way to tell "halfway" from "wedged", which mattered the day one actually did wedge. run_all.py emits "[paso i/N] name" before each step and the API derives progress from the job log. Emitting the marker from the Python rather than having the UI count STEPS itself means the step count is stated in exactly one place; adding a step cannot desync the display. Progress is derived, not stored, for the same reason: the log is already the record of what happened, and a separate counter could contradict it, which is precisely the confusion a progress display exists to remove. While RUNNING, step i is IN PROGRESS rather than finished, so only i-1 count as done. Counting i would show 100% while the final step was still working — and the final step (blob_extract) is the slowest, so the bar would sit at "100%" for the longest stretch of the job. BACKUP and RESTORE are a single mysqldump with no steps and deliberately render no bar; a fabricated percentage would be worse than none. The safety backup that precedes a REIMPORT is likewise named explicitly instead of showing 0%, which reads as stuck. Pinned by job-progress.spec.ts, including the literal line run_all.py emits, so a change to the Python format fails a test rather than silently blanking the panel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
eef9a5f4c8 |
fix(ops): stop the replica field parser reading the next line
The panel reported "Error SQL: Replicate_Ignore_Server_Ids:" against a replica that was healthy — both threads running, zero lag. `\s` matches newlines in JavaScript, so `^\s*NAME:\s*(.*)$` let the `\s*` after the colon walk past an EMPTY field's line break and capture the following line. Last_SQL_Error is blank on a healthy replica and Replicate_Ignore_Server_Ids happens to be printed immediately after it, so the blank error field returned the next field's name as its value. Every empty field was affected; the visible damage was that a healthy replica rendered as broken, which is the worst direction for a health panel to fail. Fixed with `[^\S\n]` — horizontal whitespace only — on both sides of the field name. Extracted as replicaField() and pinned by replication.spec.ts against the verbatim output of the live replica, keeping the empty Last_SQL_Error adjacent to Replicate_Ignore_Server_Ids because that exact adjacency is what broke. Also covers the literal "NULL" lag surviving as a distinct value from empty, and a field name that is a suffix of another (Last_Error vs Last_SQL_Error) not matching the wrong line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
7797c45e9f |
fix(ops): fail orphaned RUNNING jobs at startup
Ops jobs run as a child of the API process, so no job can outlive it. When a deploy landed 110 seconds into a REIMPORT, the child died and nothing was left to finalize the row — it stayed RUNNING forever. Because startJob() refuses to start while any RUNNING row exists, that one interrupted job wedged the panel permanently with no way out from the UI; recovering it took a manual UPDATE against the production database. A fresh boot is proof that nothing survived, so this is unconditional rather than filtered on age: "started recently" does not imply "still alive" here. Rows are updated one at a time rather than with updateMany so the reason can be APPENDED to the log. A job whose log simply stops mid-step with no explanation is what made the first occurrence hard to diagnose. Failure to reconcile is logged and swallowed: a wedged panel is bad, an API that will not boot is worse. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
dac1f1982f |
feat(ops): show read-replica health on the Operaciones screen
my.jorgecuadros.com serves customer balances from the Oracle VPS replica. A replica whose SQL thread has stopped does not error — it keeps answering, with data frozen at the moment it stopped — so nothing on the customer site looks wrong and the only signal is a customer complaining about a stale balance. This puts the failure somewhere a human sees it. Deliberately does not trust the two fields an operator reaches for first. Replica_IO_Running reports Yes while the SQL thread is stopped, because the network thread keeps downloading binlog it will never apply; verified by stopping SQL_THREAD and watching IO stay Yes. Seconds_Behind_Source reads NULL whenever EITHER thread is down, so the card renders "sin dato" rather than "0 s" — showing zero there would report an outage as perfect health. The problem string is resolved most-specific-first for the same reason. Shells out to the mysql client because the API has no MySQL driver and the image already ships one. --ssl is required (the replica sets require_secure_transport); --ssl-verify-server-cert=0 is deliberate and is NOT the trade-off the website makes: this hop never leaves Tailscale and the replica's firewall admits only this host, so WireGuard authenticates the peer, whereas the DreamHost leg crosses the public internet and pins the CA. The account behind it holds REPLICATION CLIENT and nothing else — it cannot read a single row. REPLICA_DB_* unset is a supported state and renders "no configurada", which is correct in dev and before cutover. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
1d689d8f46 |
fix(migration): label EFECTIVO cash rows as CASH DEPOSIT
The EFECTIVO ledgers have no type column — in Access the transaction type
is implied by which table a row lives in — so unlike DATOS2 there was no
string to map and typeId came out NULL on all 13,496 rows.
That is not just a blank label. handleGetAccountDetails in
my.jorgecuadros.com identifies payments by matching TYPEOFTRX against
('PAYMENT THANK YOU', 'PAYPAL', 'CASH DEPOSIT', 'CHECK DEPOSIT') to reset
the running balance in mode=current. An unlabelled payment is not
recognised, so the balance silently diverges from legacy — 285 rows across
129 customers in the current year alone.
"CASH DEPOSIT" is measured, not chosen: matching the unlabelled rows to the
live site on (NUMid, date, amount) resolves unanimously to that label —
66/66 in the current-year `datosfreak` and 100/100 in the prior-year `2025`
table, the only two periods the site allowlists.
The FM3 fee streams (EFECTIVO FM3 627, CHEQUE FM3 157) have the same
missing-type problem and are deliberately left NULL: every row predates both
exposed periods, so nothing can be matched against a legacy label and none
can reach a customer. Guessing "CHECK DEPOSIT" there would feed the
payment-detection list on no evidence.
type_id_for(None) returns None, so call sites without a label are unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
7bec2a13d8 |
feat(deploy): add a replication health check for the read replica
my.jorgecuadros.com reads customer data from the Oracle VPS replica, and a replica that has silently stopped applying serves stale balances rather than erroring — so "is it replicating" needed an answer that is not a human squinting at SHOW REPLICA STATUS. Runs entirely against the replica over ssh, so it needs no credentials for the galactus master, and exits non-zero on failure so it can be driven from cron or a monitor. It deliberately does not trust the two fields an operator reaches for first. Replica_IO_Running reports Yes while the SQL thread is stopped, because the network thread is still downloading binlog it will never apply — verified by stopping SQL_THREAD and watching IO stay Yes. Seconds_Behind_Source reads 0 both when there is nothing to apply and when nothing is connected. The trustworthy signal is GTID_SUBTRACT(Retrieved, Executed): binlog fetched but not applied. NULL lag means either thread is down, so it is reported as "not applying" rather than blamed on a specific thread — the thread fields above already say which, and guessing there produced a wrong diagnosis. Uses sed rather than `head -n1`; on this machine `head` resolves to LWP's HTTP head(1), which mangles the pipeline instead of failing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
7226772c22 |
fix(migration): recover transaction type labels and minimum balance
Two fields the customer-facing site reads were being dropped on the way in from Access. transform_transactions.py mapped DATOS2's type string through the Access `TYPE OF TRX` table and stored NULL on a miss. That table is a stale pick-list rather than a constraint — staff free-text straight into DATOS2 — so 78 distinct values covering 3,939 rows never resolved, including BALANCE FORWARD (1,188) and ANNUAL FEE (1,116). Nothing else on `transactions` carries the type text, so those rows lost their label outright and rendered blank. Now mints a type_transactions row from the literal string when the lookup lacks it; nameEs stays NULL since only the lookup has translations. transform_customers.py never carried DATGRAL.TIPO, leaving customers.minimumBalance empty on every row despite the column existing. TIPO is the minimum-balance threshold (100/200/300/500; 1,017 of 1,172 customers carry one), not an account type as the name suggests — the customer app shows it as `minBalance`. Added to the insert list and to the ON DUPLICATE KEY UPDATE clause, without which --sync would silently skip it on existing rows. Both land on the next `run_all.py --sync` reload. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
e77e5546d8 |
docs: SES secrets created, ship blocker cleared
Five documents asserted the SES_* secrets were unset in Gitea. They now exist, so all five are corrected rather than leaving the claim to rot in whichever one a reader opens first. Replaces the blocker with the two things creating the secrets does NOT establish, since both fail in ways that look identical to a missing config: SES_FROM must be a verified identity in SES_REGION, and the account must be out of the SES sandbox — in sandbox SES only delivers to verified recipients, so a sweep across 815 policyholders would fail almost every send while the configuration reads as correct. Recommends running the first sweep with debug on, which diverts every recipient and, on the pólizas side, leaves the avisos pending so a failed test consumes nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
3e12597204 |
docs: add BACKLOG.md, one list of everything outstanding
Open work was spread across six documents: PLAN's per-step status, RESUME §6, two specs' collected open questions, and the "Not built" sections of the two OCR docs. Nothing tracked the two live data defects except a paragraph inside INSURANCE_FEATURES_SPEC, and nothing at all recorded that master is 14 commits and 5 migrations past the last tag. Compiled by reading those six, then checking each claim against the code and the dev database rather than trusting the prose — which is how the dead-table finding surfaced and how both insurance defects were confirmed still open. Leads with the ship blocker: SES_* is unset in Gitea while the pólizas sweep defaults to enabled at 06:00, so deploying current master gives a nightly sweep that fails every run. Set the secrets or disable the schedule before cutting v1.0.7. Findings not previously written down anywhere: - policy_types still holds only AUTO/LICENCIAS/MULT and 5 policies still have a NULL policyTypeId; policyTypeId is still `String?` with Prisma's default SetNull, so the spec's recommended Restrict was never applied. - EmailTemplate / EmailCampaign / EmailLog have zero references in apps/api/src or apps/web/src. Scaffolded for step 10's "email campaigns"; notificaciones shipped against email_notification_log instead. Either wire them or drop them. - Customer.customerNumber does not exist, so recycling is not merely unbuilt but unstarted at the schema level. Linked from PLAN.md and README so it is findable from either entry point. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
ec139737be |
docs: as-built reference for the statement OCR capture
Gives receipt capture the same treatment policy OCR just got: a doc that records what is in the code, separate from the spec that records what was designed. RECEIPT_CAPTURE_SPEC.md §2 had accumulated three BUILT notes totalling ~120 lines of findings, which is the right place for the evidence but the wrong place to look up how the matcher picks a column. docs/STATEMENT_OCR.md covers the pipeline, the OCR seam and its text-layer-first rule, all eight parsers and the ordering constraints between them, the matcher's two governing rules and the scopedRefField table, confirm-through-BillingService, the learning write-back, and the API surface. Weight goes to the things that are load-bearing and invisible from the code shape: brand detection must run to completion before layout because Tijuana bills predial and zona federal off the same treasury header; scopedRefField is exported because three call sites must agree or a reference gets learned into a column nothing searches; FEDERAL_ZONE's accountNumber holds a peso amount, so it fails the null-guards as well as the lookup; a misread `$` is the dangerous failure, not a missing one. Also records that CFE/CESPT/Telnor have no unit suite — they predate the gas/predial extension and were only verified end to end. Cross-linked from the spec, POLICY_OCR.md, PLAN.md, README and RESUME.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
872a661051 |
docs: document policy OCR capture, the feature no spec proposed
Policy OCR shipped 2026-08-01 (
|
||
|
rmancinasandClaude Opus 5
|
6331481f82 |
docs: record notificaciones as built, flags global, schedules editable
The docs still described the state before the last five commits: the insurance spec called for a `@Cron` literal and a manual mark-as-sent mutation, PLAN.md had step 12 as "NOT STARTED", and README's module and route lists predated seven modules. - MASS_EMAIL_NOTIFICATIONS.md: new "Send flags", "API surface" and "Scheduled runs" sections; "Cron (future)" removed — it exists. The flags table says which flags apply where, and why a debug renewal send must skip both the RenewalNotice row and `lastSuccessfulAt`. - INSURANCE_FEATURES_SPEC.md: §1 BUILT note listing the three places the build diverged from the spec; §1.1 and §1.4 marked superseded in place rather than deleted, so the reasoning stays readable. - PLAN.md: step 12 renewal emails DONE with the divergences; status paragraph rewritten. - README.md: current module/route lists, plus a "Scheduled jobs" section — a reader cloning this repo had no way to know the API sends mail on a timer. - DEPLOY_AND_MIGRATIONS.md: the cadence lives in app_settings and survives an image rollback, and the servicios sweep has no multi-replica lock. - RESUME.md: session record for the whole notificaciones arc. - RENEWAL_NOTICES.md: pointer that this is the legacy record, not what shipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
89611da202 |
feat(notificaciones): global send flags + editable schedules
The "Flags del envío" panel lived inside the Servicios tab and only
governed the four bulk jobs. The pólizas half had no debug at all, so
there was no way to test a renewal notice without mailing a real
customer. The panel now lives in the /notificaciones shell above the
tabs and both halves read it.
`debug` on the renewal path diverts to the same override inbox as the
servicios jobs and deliberately does NOT write the `RenewalNotice` row
or advance the sweep's `lastSuccessfulAt` — the customer was not
notified, so nothing may gate the letter they are still owed.
`ignoreDayRestriction` and `useEmailLimit` stay estado-de-cuenta-only
and are labelled as such.
Both automatic sweeps are now operator-editable. The renewal cadence
was a `@Cron("0 6 * * *")` literal and servicios had no automatic run
at all; both now resolve through `NotificationScheduleService`, which
stores the cadence in `app_settings` and reinstalls the cron job on
save — no redeploy, no restart. Defaults preserve current behaviour:
pólizas 06:00 daily, servicios off. A scheduled run never inherits the
UI flags; it always sends for real.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
a491ef3eed |
feat(notificaciones): edit summary recipients in the UI
NOTIFICATION_ADMIN_EMAILS made "add Beto to the summaries" a redeploy — the wrong unit of work for a list that changes when office staff change. Adds `app_settings`, a key/value table for the configuration staff must be able to change without a deploy, and `SettingsService`, which resolves every key db -> env -> default and reports which of the three a value came from. That ladder is what makes the move safe: a deployment behaves exactly as before until somebody saves in the UI, and the screen can say "this is still coming from the deployment" rather than implying somebody chose it. - new ability `setting:manage` (ADMIN) — deliberately above `notification:send`, since redirecting the audit summaries is how someone would quietly stop them being read - GET/PUT /notifications/settings/admin-emails; read is open to any logged-in user so the UI can display the list, write is gated - resolved per job, not cached at boot, or we would reintroduce exactly the restart-to-apply behaviour being removed - a saved empty list means "nobody" and does NOT fall through to the env, or clearing the field would keep mailing the people just removed Credentials stay in env — see the model doc for where the line is drawn. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
f4b92fa7a5 |
fix(deploy): pass SES config through to the app stack
The stack env is assembled from Gitea repo secrets by the deploy workflows' `env_data` block — there is no .env file on the host for the app stack. SES was in neither, so `MailService` came up unconfigured on every deployment and, with NODE_ENV=production killing the stdout dev fallback, every notification and renewal aviso failed. Wire SES_REGION / SES_FROM / SES_FROM_NAME / SES_ACCESS_KEY / SES_SECRET_KEY / SES_CONFIGURATION_SET / NOTIFICATION_ADMIN_EMAILS through both galactus and cubex. No `_GALACTUS` suffix: one SES identity serves every deployment. Kept out of the required-secrets preflight — mail is not needed to boot, and failing a deploy over it would be wrong. Preflight warns instead, since the failure is otherwise invisible until someone clicks "Ejecutar". Also corrects the comments added in the previous commit, which claimed these belonged in a host env file rather than in CI secrets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
33833c3af9 |
feat(notificaciones): one send log across servicios and pólizas
Renewal avisos left behind only a `RenewalNotice` row, whose sole job is gating: a row with `sentAt` drops the policy off the pending list. It cannot represent a failed send or a customer with no address, so the Pólizas tab had no "Registro de envíos" to show and a sent notice simply vanished from the list. Renewals now write `email_notification_log` — the same table the four bulk jobs write — as `RENEWAL_NOTICE` / `POLICIES`, with rows for failures and no-email skips too. `RenewalNotice` keeps its gating role unchanged; the two are complementary, not redundant. - extend `EmailNotificationType` (+RENEWAL_NOTICE) and `EmailNotificationServicio` (+POLICIES); `level` now carries the aviso generation on renewal rows, so every reader must branch on the type first (`notificationLevelLabel()` is the one place that lives) - backfill emailed notices (`channel = 'EMAIL'`) into the log; MAIL-channel rows are legacy printed letters and are deliberately left out - extract `NotificationLogService`/`NotificationLogModule` as the single writer, so a feature that sends mail records it without pulling the bulk-job pipelines into its module - `GET /notifications/log` and `/stats` take a comma-separated `servicio` list; each tab reads its own slice. This also fixes the "Omitidos" view, which mapped to no filter at all and showed every row - share one `NotificationLogPanel` between both tabs - pass SES_* / NOTIFICATION_ADMIN_EMAILS through the galactus compose, which was missing them entirely — mail is runtime config, not a CI secret, and the prod image sets NODE_ENV=production so a blank config fails loudly instead of falling back to stdout Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinas
|
c0cc0d2ac2 |
feat(renovaciones): send renewal notices from the list, drop manual marking
The Pólizas tab now sends. Each pending row gets an "Enviar aviso" button backed by POST /renewals/send, which renders, mails and records the notice through the same path the daily sweep uses — so a hand-sent letter is marked exactly like a swept one and drops off the pending list. Sending is now the only way a notice gets marked as sent. Remove the manual "Marcar impreso" / "Marcar EMAIL" buttons and the endpoint behind them (POST /policies/:id/renewal-notices, PoliciesService.markRenewalNotice, MarkRenewalNoticeDto): they wrote a sentAt with no mail behind it, which let the list claim a customer was notified when nothing was sent. sendOne refuses a generation that already has a sentAt (409) so a double click cannot mail the customer twice, and 400s when the customer has no email on file. Sweep and single send share the new deliver() helper. |
||
|
rmancinas
|
53a5fe8076 |
feat(notificaciones): ejecutar todos for servicios jobs
Add POST /notifications/run-all: runs the four notification jobs (outstanding, payment confirmation, account status, trust confirmation) sequentially with one shared set of flags from "Flags del envío". Sequential rather than parallel — the jobs share the SES transport and account status can self-throttle via useEmailLimit. A job that throws is captured and the sweep continues, so one bad query cannot swallow the other three envíos; the aggregate response carries per-job results plus summed sent/skipped/failed and an errors count. Audited as a single notification.run-all.run entry so one staff click is one audit row. UI adds the button to the flags card, with a confirm when debug is off, and a per-job summary in "Última respuesta". |
||
|
rmancinasandClaude Opus 5
|
0332292ae9 |
fix(notificaciones): merge renewals into one screen, fix MailModule DI
MailModule's provider used a `useFactory` with no `inject`, so the factory received `undefined` and `new MailService(config)` threw on `config.get`, taking the whole API down at boot. The module also wasn't actually `@Global()` even though both NotificationsModule and RenewalsModule inject MailService without importing it — that would have failed next. Replaced the factory with a plain provider (ConfigModule is already `isGlobal`) and marked the module global. On the web side, mass email and renewal notices were two menu entries doing the same job — telling a customer something by email. They are now two tabs of `/notificaciones` (Servicios and Pólizas), following the Captura pattern: `/renovaciones` still resolves, opening the same screen on its Pólizas tab so existing bookmarks keep working. The notifications page was also the last screen written in raw inline styles, with blue buttons and filter pills that appear nowhere else in the app. It now uses the shared design system: btn-primary/btn-outline, the seg segmented control, card, tx-table, pager, and the servicios/fideicomiso badges. Two supporting fixes found on the way: NOTIFICATION_STATUS_COLORS hardcoded hex instead of the theme's positive/negative/muted vars, and `.small` was referenced in 19 places across the app but never defined in globals.css. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinas
|
ec0e9c2a5d |
Merge branch 'massive-email-notification' into master
# Conflicts: # .env.example # apps/api/src/app.module.ts |
||
|
rmancinas
|
a52e59cbc5 |
feat(notificaciones): mass email notifications over SES
Replaces the four legacy PHP scripts under email.notifications/send*.php with a single NestJS module. Four jobs (outstanding payments, payment confirmations, account-status alerts with day-of-week gates, trust payment confirmations) share one MailService modelled on StorageService: env-driven SES client, null fallback in dev with console logging, refuses to send in production when unconfigured. Schema adds email_notification_log (every attempt, sent/failed/skipped) and account_status_history (one row per threshold hit, Job 3). Enums encode the legacy wire shape so external log scrapers keep parsing notificationType keys verbatim. Web adds /notificaciones with four trigger cards, a flags panel, and a paginated log browser. New notification:send ability gates all four endpoints at MANAGER, matching the renewal:send trust tier. |
||
|
rmancinas
|
87d8743251 |
feat(renovaciones): renewal notification emails over SES
INSURANCE_FEATURES_SPEC §1. The office printed and mailed renewal letters from the legacy CONTROL <ramo> RENEW[2/3] paper log; 91% of policyholders have an email on file, so send the notice instead and keep the paper log as the fallback. A daily cron (06:00 America/Tijuana) sweeps three generations off policyTo — 30 and 15 days before expiry, 7 days after — sends each through SES, and upserts RenewalNotice by [policyId, generation] so a policy is never notified twice for the same milestone. RenewalNotice now records providerMessageId, so a later bounce or complaint webhook can be traced back to the row that sent it. - customers.emailOptOut excludes a customer from every sweep; editable from the customer form - scheduled_job_states holds the sweep's lock and last successful run; the window is widened to cover days the job did not run, so a weekend outage does not silently drop a generation - SES unconfigured is not an error outside production — messages are logged and skipped, so dev and CI never send - /renovaciones (renewal:send, MANAGER+) lists what is pending per generation, runs the sweep by hand, and marks a notice sent by mail for the customers with no email - POST /policies/:id/renewal-notices records that manual mark - the aviso-renovacion report and the emails now share one projection (reports/renewal-letter.ts) instead of two copies of the mapping |
||
|
rmancinas
|
3125b52057 |
feat(ocr): discard abandoned capture batches
A bad scan, the wrong PDFs or a duplicate upload used to leave a batch sitting in READY_FOR_REVIEW forever, because the only exits were confirm (posts to the books) or rejecting every page one at a time. Add a DISCARDED terminal status to both OCR domains and a single endpoint per domain that rejects every page still pending in one shot. Discarding is refused once anything has landed: statements once a page is POSTED, policies once a page is APPLIED. Those batches did real work and have to be settled page by page. - POST /statements/batches/:id/discard - POST /policy-ocr/batches/:id/discard - shared DiscardBatchCard on both review screens, gated the same way |
||
|
rmancinasandClaude Opus 5
|
5e9cb12fba |
feat(polizas): OCR capture for insurance policy PDFs
Mirrors the utility statement intake on the insurance side: a policy_ocr batch/document pair of tables, a GMX parser, a matcher keyed on Policy.policyNumber, and a "Captura" screen under /polizas that proposes policy -> customer for staff to confirm. Lifts the OCR seam out of StatementsModule into its own OcrModule so PolicyOcrModule can inject OCR_PROVIDER without taking on the rest of the statement pipeline; StatementsModule now imports it and binds nothing itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
5bce0e4c94 |
feat(recibos): OCR capture for zona federal (ZOFEMAT Tijuana)
Adds the ZONA FEDERAL TIJUANA parser to the statement intake, measured against 8 pages of real "Zona Federal Marítimo Terrestre" receipts — the federal maritime-zone occupancy fee the municipality bills on beachfront lots. Provider read on 8/8, amount on 8/8 (each verified against the paper), concession clave on 6/8, period on 8/8, deadline on 2/8. Four things the corpus forced: - Tijuana bills predial and zona federal from the same treasury: same header, same Paseo del Centenario address, same ATB-541201 RFC. Every predial discriminator matches a zona federal page too, so whichever rule is asked first wins it. The only words exclusive to this layout are "Marítimo Terrestre", so its brand rule is asked ahead of all three predial ones — and its structural rule, anchored on the stub's "Derechos de ocupación", ahead of theirs. - FEDERAL_ZONE.accountNumber is an amount, not a reference. It holds DATMEX.zfed, whose 77 values include 246.06, 2369.09, 22653.94 and a negative -1679, while the concession claves these receipts are keyed by appear nowhere in the database. Matching on that column could never hit — and because every row already has a value, the `[field]: null` guards on learnAccountRefs and on the review blank-service fill would never fire either, so every page would return to the queue every bimester forever. The clave moves to meterNumber, joining gas and Tijuana predial, and the first confirm teaches the match. - The payable figure is not the printed subtotal. The municipality rounds to whole pesos and prints the difference on its own "Ajuste Ley Hacienda Mpal" line (-$0.05 against a 591.05 subtotal, $0.21 against 2,872.79). The "Total a pagar" box carrying the rounded figure sits on a grey fill and OCR'd on 1 of 8 pages; the SubTotal row read on 8 of 8. So the amount is the rounded subtotal, cross-checked against the printed box wherever it survives — where it did, it agreed. - The clave is 2 digits, a letter and 3 digits (12-T -012), not the cadastral shape, and the letter is kept as printed: toDigits maps D to 0, which turns a real 14-D -014 into 140014. It is printed twice, which rescued a page whose heading was struck through by the office's own highlighter — the failure mode behind both missing claves. Deriving the deadline from the bimester is deliberately not attempted: it is the 17th of the month after the bimester closes on a current bill, but four of these eight are late (a $1,000 Multa) and print a recalculated date, so a derived date would be wrong on exactly the pages a human most wants to see. Re-ran the earlier corpora (25 pages: predial Tijuana/Rosarito/Ensenada, CFE, CESPT, Telnor) through detection to confirm the new rules steal nothing — all 25 still read as their original provider, including the five Tijuana predial pages that share the RFC. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
d6501f1d74 |
feat(recibos): OCR capture for gas butano and municipal predial
Adds four parsers to the statement intake — GAS TIJUANA plus one per municipality, because Tijuana, Rosarito and Ensenada issue three completely different predial documents — and a text-layer fast path for the born-digital invoices the gas company sends. Measured against a new corpus of 14 documents / 29 pages: provider read on 29/29, amount on 26/29, and 21/29 auto-matched against the dev database (22/29 identified). The eight review cases are all legitimate. Five things the corpus forced: - Not every statement is a scan. The gas invoices are born-digital CFDIs whose text layer is exact; rasterising them only loses information (one sample turned `MEDIDOR: VM01014426` into `ar (LTR): 014420`). The new `OcrProvider.textPages` reads the embedded layer via `pdftotext -bbox-layout` — same poppler package as `pdftoppm`, so no new dependency — and OCR stays the fallback for real scans. Poppler's own `<line>` grouping follows text flow rather than the page, so words are regrouped by vertical position; without that, a two-column header leaves every label separated from the value printed beside it. - The clave catastral is not two letters and six digits. Position three is a letter in 15 of the 932 stored claves, and digitising the whole tail mapped a real `MMB01041` to a nonexistent `MM801041`. - Tijuana predial prints no clave at all. Its only identifier is an 8-digit municipal account carried in a 32-digit payment barcode, which the legacy database never held, so it goes in `meterNumber` alongside gas — `accountNumber` holds `DATMEX.predial`, which is not a per-property key and must not be overwritten. Those pages start cold and are taught by the first confirm. - On Rosarito and Ensenada the clave is the primary key, not a fallback: those receipts print nothing else, so a unique hit auto-matches. On a utility bill that merely happens to print one it stays a review hint. - A misread `$` is the dangerous failure. An Ensenada receipt for $2,203.00 OCR'd as `82,203.00`, which would post a charge 37x too large and look ordinary in the ledger. Predial amounts now require a literal `$` and a page that cannot produce one goes to review. The scoped match field is now one exported function rather than three copies of `kind === "GAS" ? ... : ...`, since the lookup, the blank-service fill and the confirm write-back have to agree or a reference gets learned into a column nothing searches. First tests in this package: 23 specs over the parsers and the text-layer reader, every fixture a verbatim OCR excerpt from a real receipt. Adds the jest config they need and a build tsconfig so they stay out of dist. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinas
|
216309190c |
feat(recibos): live OCR progress bar on review page
Backend already returns per-status counts via byStatus; render a real progress bar (X% / N de M / en cola) while PENDING_OCR pages remain, using the existing progress-track CSS. Falls back to indeterminate when no docs have been reported yet. |
||
|
rmancinasandClaude Opus 5
|
e589bda28b |
ci(build): skip the redundant master build when a release is cut
release.yml pushes the release commit and its tag in a single `git push`, so Gitea created two build.yml runs for the same commit. Only the tag run matters: it emits the X.Y.Z and X.Y image tags, and since it is the same commit it publishes `latest` and `sha-<short>` as well. The master run was pure duplicate work that had to be waited out or cancelled by hand. Guard the build job with an `if` that skips a branch push whose head commit message starts with `chore(release):`. Ordinary pushes to master are unaffected, and tag pushes and manual dispatches always build. The skipped master run keeps the release commit's sha, which would have let release.yml's "Verify build.yml started" check go green on it alone even if the tag run were never created — the exact failure that check exists to catch. It now also requires the run's ref to be the tag, falling back to the sha match only when the API reports no ref. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
898cf48c80 |
fix(migration): re-import died on the last step because blob_extract required deploy/.env.prod
Every transform resolves its target through dbenv.database_url(), which lets a
DATABASE_URL in the process environment win — that is how the API container
drives a re-import against its own database with no deploy/ directory present.
blob_extract.py was the one step that bypassed it and called load_env()
directly for the MinIO credentials, so the "Operaciones" re-import loaded all
the data and then exited 1 on:
missing /repo/deploy/.env.prod — deploy the 'prod' DB stack and write its
.env first
Give the S3 settings the same resolution as the DB URL: load_env() now returns
{} for an absent file, and setting()/require() layer the process environment on
top of it. blob_extract reads S3_ENDPOINT / S3_BUCKET and accepts either
S3_ACCESS_KEY/S3_SECRET_KEY or MINIO_ROOT_USER/MINIO_ROOT_PASSWORD, matching
the fallback order in storage.service.ts and the vars the api service already
sets in deploy/galactus/jorgecuadros-app.compose.yml. A genuinely missing
setting still fails fast, now naming the variable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
567b033c46 |
fix(docker): re-import failed because the Access CLI tools were never installed
The API image installed Alpine's `mdbtools` package, which ships only the
shared library. The command-line tools that migration/extract.py actually
shells out to -- `mdb-tables` and `mdb-export` -- are in the separate
`mdbtools-utils` subpackage, so the build succeeded and the re-import in the
"Operaciones" admin panel failed at run time with:
RuntimeError: mdbtools not found on PATH (need mdb-tables and mdb-export)
Install `mdbtools-utils` instead; it pulls the library in as a dependency.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
1934470d53 |
ci(release): dispatch the fallback build with a fully qualified ref
Gitea's workflow dispatch API 404s on a bare `v1.0.3` and accepts only
`refs/tags/v1.0.3`, so the fallback added in
|
||
|
rmancinasandClaude Opus 5
|
fdbe9fdb88 |
ci(release): fail the release when the build never starts
Gitea creates workflow runs from the post-receive hook. When that hook errors the refs still land, git prints `remote: error: Internal Server Error` and exits 0 — a post-receive failure does not fail a push. v1.0.3 was cut exactly that way: tag pushed, no build run created, no images published, and the release step green. It surfaced two steps later as a 404 when the deploy tried to pull 1.0.3. Capture the push output and warn on `remote: error`, then verify a build.yml run actually exists for the new commit, dispatching it against the tag if not. Fail the release if that does not take either, so a release that publishes nothing is red instead of green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
860d483bad |
fix(ops): backup failed on the MariaDB client shipped in the API image
Every backup on galactus died with: mysqldump: unknown variable 'set-gtid-purged=OFF' respaldo incompleto eliminado Alpine's mysql-client is MariaDB's, so `mysqldump` inside the API container is a shim over `mariadb-dump`, which has no --set-gtid-purged. That took out BACKUP and, because they take a safety dump first, SYNC and REIMPORT too. Probe `mysqldump --help` and pass the flag only when it is advertised, calling `mariadb-dump` directly otherwise — MariaDB writes no GTID state unless asked with --gtid, so there is nothing to suppress. Testing whether mariadb-dump merely exists would be wrong: on a host carrying both clients it would shadow a perfectly good MySQL mysqldump. The probe uses a command substitution rather than `--help | grep -q` because PIPEFAIL is in effect for these commands and grep closing the pipe early would report a supported flag as unsupported. pre-migrate-backup.mjs is unaffected — it dumps from a real mysql:8.4 image, not from the API container. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
783ec83464 |
feat(ops): show upload percent, speed and ETA for ingest files
The ingest upload used fetch(), which cannot report request-body progress, so the only feedback was a static "Cargando…" label — no way to tell a stalled 2 GB upload from a working one. Switch uploadFile() to XMLHttpRequest and expose an optional onProgress callback reporting loaded/total bytes, a smoothed transfer rate and a remaining-time estimate. The Operaciones ingest table renders a progress bar row under the file being uploaded. Once the bytes are all sent the server still has to write the file, so that tail reads "Procesando en el servidor…" rather than parking at 100%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
a8afd87c3f |
ci: add a "Cut release" dispatch workflow
Stamps every package.json, commits chore(release): vX.Y.Z, tags and pushes both refs in one dispatch — patch/minor/major, or an explicit number. Cutting a release from a laptop is how a manifest bump gets forgotten or a tag lands on an unpushed commit; the only input here is the number. Guards: refuses a version that already exists as a tag (releases are immutable), a no-op bump, a leading `v`, and a malformed number. Checkout is full-depth because the duplicate-tag check is meaningless against a shallow clone. Pushes with a RELEASE_TOKEN PAT rather than the built-in Actions token — whether a push made with that token re-triggers build.yml depends on the Gitea version, and a release that quietly publishes no images is worse than one that fails outright. Builds and deploys stay separate: the tag push triggers build.yml, and deploying remains a deliberate dispatch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
b59abda895 |
feat(captura): fold recibo OCR into Captura as an automatic mode
Scanning a stack of bills and keying them in are the same daily job, ending in the same ledger path, so OCR intake becomes a mode of the capture screen instead of a second menu entry: - components/Captura.tsx holds the mode switch; the manual check form moves verbatim to components/ManualCheckCapture.tsx and the OCR intake to components/StatementIntake.tsx. - /estado-cuenta/lote opens on manual, /recibos on automatic — both render Captura, so batch-review links and old bookmarks still land right. - Nav drops "Recibos (OCR)"; "Captura" covers both, with a NavLink.aliases field so /recibos still highlights it. Also fixes the "El almacenamiento de documentos no está configurado" failure staff hit on upload. Uploading with no object storage configured used to succeed, then die on the first put minutes later, leaving a FAILED batch whose only explanation was that string. createBatch now refuses up front, GET /statements/status reports storageAvailable alongside ocrAvailable, and the intake tab explains the situation instead of offering an upload that cannot work. S3_* documented in .env.example (deploy stacks already set it). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
4d5008b545 |
feat(statements): OCR intake for scanned utility bills
Staff key 300+ utility statements per company per month by hand. This adds the ingest -> split -> OCR -> match -> review pipeline that proposes customer and amount per page instead (RECEIPT_CAPTURE_SPEC §2), posting through the existing BillingService.createBatch seam with source=OCR and a per-document captureRef so machine and hand capture share one write path and audit trail. Everything was designed against 10 real scanned statements (46 pages of CFE, CESPT and Telnor bills) rather than from the sample-free spec. The scans have no text layer at all — they are camera images — so OCR is mandatory, and they arrive bundled one customer per page. Measured on those pages the parser identifies the provider 46/46 and reads an account reference 43/46; against the dev database that is 39/46 (85%) exact auto-match, 40/46 identified, with the rest genuine review cases. That closes the OCR-provider question in favour of self-hosted Tesseract: it clears the bar for a queue where a human confirms every row, and OcrProvider keeps a managed API a one-line swap. The samples corrected three things the spec had wrong or unknown: - Clave catastral is NOT predial. DATMEX.clave (934 rows) is what CESPT and predial bills print; DATMEX.predial, which PROPERTY_TAX.accountNumber holds, has 663 distinct values across 1135 rows and appears on no statement. The clave now lives on Property.cadastralKey as the matcher's secondary key; predial is left untouched. This had been blocking predial matching. - Gas was recoverable: 160 of 334 DATMEX.gas values are real account numbers (the rest are ESTACIONARIO/CILINDRO descriptors), now in GAS.meterNumber. - Phone is one billed line per property (534/18/1 across phone1/2/3), so the new TELEPHONE ServiceKind backfills from phone1 only, not three rows. Matching is scoped to one column per service kind and never reads the customer name — a CESPT receipt prints ARNAIZ ROSAS ELSA AURORA for an account this office holds under CATT, RANDY, because the printed name is the registrant, not the current owner. Where a provider prints a payment barcode it beats the printed label (one CFE label OCR'd a digit too many while its barcode was correct) and the two cross-check, with disagreement forcing review. Confirming a document whose service had no reference writes it back, so gas and any other cold start is a one-time cost rather than a permanent queue. Verified end to end against the live dev API and MinIO: real scans uploaded over HTTP, matched, confirmed against a check, and the resulting rows checked in MySQL (negative amounts, captureSource=OCR, concept derived from the batch kind, captureRef linking back to each page). Re-confirming a posted batch is refused. Test data was removed afterwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
121952fdc1 |
chore(release): v1.0.1
v1.0.0's images were built from
|
||
|
rmancinasandClaude Opus 5
|
15f533b984 |
fix(web): show the full commit hash in the build footer
The footer abbreviated to 7 characters, so the line read "v master · 19f0319". That line exists to be pasted into `git show` or compared against a registry tag, and an abbreviation makes both a manual step — while the full 40-char value was already baked into the image (build.yml passes `github.sha` whole, and /version returns it untouched). `shortSha` had no other caller, so it goes with it. The span gets `overflow-wrap: anywhere` and `min-width: 0`: hex offers no break opportunity, and the api/web mismatch branch renders two of these hashes side by side, which would otherwise push a phone into horizontal scroll. Measured at a simulated 360px with both hashes present — the span wraps, and documentElement.scrollWidth stays equal to clientWidth. Verified in the browser against the dev database: footer renders "v1.0.0 · db2bd54c0ffee1234567890abcdef0123456789a", hash length 40. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
db2bd545a1 |
chore(release): v1.0.0
Every manifest still read 0.1.0 while the deployed images were addressed by the moving tag `latest`. That combination is what hid the stale-image bug: a checkout could not be placed against a running container, and `latest` silently kept serving two-commit-old web code through a green deploy. Tagging v1.0.0 makes docker/metadata-action publish immutable `1.0.0` and `1.0` image tags, so deploys can name a version instead of a moving target. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
30dfc7dc3e |
fix(ops): run backups as an admin login, and stop recording failed dumps as good
The Operaciones panel (backup, restore, sync, re-import) shelled out to mysqldump as the application user, parsed straight out of DATABASE_URL. `--single-transaction` issues FLUSH TABLES, which needs the global RELOAD privilege, and the app user is granted only ALL ON jorgecuadros.* plus USAGE ON *.*. BACKUP failed outright; SYNC and REIMPORT failed with it, since both take a safety backup first. An admin credential is now supplied out of band via OPS_DB_ADMIN_USER / OPS_DB_ADMIN_PASSWORD, mirroring what deploy/scripts/pre-migrate-backup.mjs already does, rather than permanently elevating the user the API serves requests as. Host, port and database still come from DATABASE_URL, so the override can only change who logs in, never which server. Unset, it falls back to the DATABASE_URL credentials and warns — local development is unaffected. Two defects in the dumps themselves, both shared with the deploy backup before it was rewritten: - No --set-gtid-purged=OFF. The production server is the replication source with GTID on, so every dump embedded SET @@GLOBAL.GTID_PURGED and was unrestorable onto the server it came from — the one thing the restore screen is for. - The pipeline's exit status was gzip's, and gzip succeeded. A mysqldump that died on its first statement left a small, perfectly valid archive that the job recorded as SUCCESS and the restore screen listed as an ordinary restore point. Dumps now run under `set -o pipefail`, assert a CREATE TABLE count, and delete their own output on failure. Verified with a stubbed mysqldump: a failing dump exits 1, surfaces the real error, removes the partial file, and — critically — stops SYNC/REIMPORT before the ETL touches anything. Restores gained pipefail too: a corrupt archive made gunzip fail while mysql, fed a truncated stream, could still exit 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
d5ebb86cae |
fix(deploy): dump from a dedicated container as root, and prove the dump is real
The pre-migrate backup ran INSIDE the API container, which made it depend on that image's toolchain — and deadlocked: the running image shipped a MySQL client that could not authenticate, so the backup failed, which blocked the very deploy that would have replaced the broken image. A backup must not depend on the thing being deployed. The dump now runs in a throwaway container built from mysql:8.4 with the API's backup volume mounted. The volume name is discovered from the API container's mounts, so the file still lands where the Operaciones restore screen looks. As a container rather than an exec, its logs can simply be read — no more failures reported as a bare exit code. The image is pulled if the host lacks it, since a scope:app deploy never touches the db stack. Three further defects found while verifying, none of which would have surfaced without dumping against the real database: - The dump now runs as root. mysqldump --single-transaction issues FLUSH TABLES, needing the global RELOAD privilege; the MySQL image grants the application user only ALL ON `<db>`.*, and --skip-lock-tables does not avoid it. Elevating the app's own runtime user would have been the worse trade. - --set-gtid-purged=OFF. galactus is the replication SOURCE with GTID on, so a default dump embeds SET @@GLOBAL.GTID_PURGED and is unrestorable onto the server it came from. Verified: 0 GTID_PURGED lines in the output. - Verification was too weak to be worth having. `test -s` plus `gzip -t` passes on a 372-byte gzip containing no tables, which is exactly what a dump that died on its first statement produces. It now asserts a CREATE TABLE count and logs it. A failed attempt also deletes its own output, so a truncated file never appears in the restore list. Verified against live prod, both paths: success writes a 31-table dump the API container can see; a wrong password fails with mysqldump's own error quoted and leaves the volume empty. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
19f03198d6 |
fix(docker): install the MySQL 8.4 auth plugin; report why a dump fails
The pre-migrate backup failed with "mysqldump exited 2" and nothing else.
Reproduced on the host with stderr captured:
ERROR 1045: Plugin caching_sha2_password could not be loaded:
/usr/lib/mariadb/plugin/caching_sha2_password.so: No such file or directory
Alpine's `mysql-client` is MariaDB's client and ships an EMPTY plugin
directory, so it cannot perform caching_sha2_password — MySQL 8.4's default and
effectively only auth method. `mariadb-connector-c` provides the plugin.
This was never about the deploy backup alone. Every mysqldump/mysql call from
the API container was broken, which means the whole Operaciones panel — backup,
restore, sync, re-import — could not work in a container. It went unnoticed
because that feature had only ever been run with the API on a developer
machine, where the Oracle client is installed. Verified after the fix: dump
exits 0, gzip valid, 31 CREATE TABLEs.
Also fixed, both found while chasing the above:
- The backup script reported an exit code and nothing else, because a detached
exec captures no output — which is precisely why this needed a manual
reproduction. mysqldump's stderr is now redirected to a file and read back
through a short attached exec on failure, so the deploy log states the cause.
Verified against live prod: the log now carries the 1045 line itself.
- Listing ONLY 100.100.100.100 as the containers' resolver costs them public
DNS, since MagicDNS does not forward upstream unless the tailnet defines
global nameservers. Nothing at runtime needed it, but `apk` inside the
container stopped resolving, and anything outbound would have too. A public
fallback resolver is now listed after MagicDNS.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
b2cdcbe2cd |
fix(api): session cookie never issued over HTTP; ship the seed script
Prod came up with nobody able to log in, in two separate ways.
1. No sign-in account exists. `prisma migrate deploy` creates tables, never
rows, and nothing in the deploy path seeds one — deliberately, since making
an administrator should not be a side effect of shipping code. But
apps/api/scripts was not in the runtime image either, so the only way to
create the first account was to run the script from a developer machine
against a production DATABASE_URL. Ship scripts/ in the image so it can be
run on the host with docker exec. Still never run automatically.
2. Login could not establish a session at all. cookie.secure followed NODE_ENV,
the image sets NODE_ENV=production, and the app is served over plain HTTP —
express-session then silently emits NO Set-Cookie header. POST /auth/login
still answered 200 with the full user object, no session was created, every
later request 403'd, and the UI would have looped back to /login. It reads
as an auth bug and is really a transport mismatch.
The flag is now driven by SESSION_COOKIE_SECURE, still defaulting to
NODE_ENV. An EMPTY value counts as unset rather than false, because compose
turns an absent `${SESSION_COOKIE_SECURE:-}` into the empty string and the
naive check would have quietly dropped Secure on any deployment that merely
passed the variable through.
galactus sets it to "false". That is acceptable ONLY because the host is
reachable exclusively over Tailscale, so WireGuard already encrypts the
wire. It must go back to "true" when the app is served over TLS or exposed
off-tailnet; behind a TLS-terminating proxy, set trust proxy instead.
Verified against live prod: seeded an admin, POST /auth/login returns 200 with
full ADMIN abilities, a wrong password is rejected with 401, and no Set-Cookie
was present before this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
7e3b530174 |
fix(deploy): pull images explicitly, and detect api/web drift by commit
The first successful galactus deploy came up all-green while the web tier was running a build from two commits earlier. The registry held web:latest from 3ff56e6; the host still had a web:latest cached from 4ee7ec7; the deploy reported success and served the old one. The API was only current because it had been pulled by hand during earlier debugging. Two independent failures, both fixed here. 1. Images are not pulled. The deploy action's `pull: true` does not reliably refresh an already-cached moving tag on a standalone endpoint. Added a Pull images step (deploy/scripts/pull-images.mjs) that pulls each image through Portainer's Docker API with registry credentials and fails the deploy if a pull fails — note the endpoint answers 200 even when the pull errored, so the stream body has to be inspected, not just the status. 2. The drift check could not see it. Both the verify step and the web footer compared APP_VERSION, but on a branch build BOTH tiers report "master", so equality proved nothing. They now compare gitSha, which is the only field that differs between two builds of the same branch. api and web come from one matrix run, so a difference can only mean an image was not replaced. This needed a /version on the web tier too — previously its build identity was only readable by scraping window.__APP_BUILD__ out of the HTML. pull-images.mjs builds the X-Registry-Auth header as URL-safe base64 WITH padding: Node's "base64url" omits the padding and Portainer's Go decoder rejects it with "Illegal base64 data at input byte N". Verified against galactus: pulls both images, and exits non-zero on a nonexistent tag. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
1cba9bfc32 |
fix(galactus): give containers Tailscale's resolver so MagicDNS names resolve
With the image fixed, the API got as far as connecting and then died with
Prisma P1001 "can't reach database server". The cause is DNS, not routing.
galactus runs systemd-resolved, whose 127.0.0.53 stub is unreachable from
inside a container, so Docker falls back to the upstream resolver in
/run/systemd/resolve/resolv.conf — the LAN router, which knows nothing about
the tailnet. Verified from a probe container on galactus: resolving
galactus.tail01aa2.ts.net fails outright, while `nc 100.103.77.46 3306` is
OPEN. Only the lookup was broken.
Pin the api and web services to Tailscale's own resolver (100.100.100.100,
the same anycast address on every tailnet) with this tailnet's search suffix.
Both are overridable via TAILSCALE_DNS / TAILNET_SUFFIX. db and minio need
nothing — they make no outbound calls.
Verified end to end: the published image, unmodified, with only these DNS
settings, boots on galactus against the real database and serves
/health {"status":"ok"}
/version {"service":"api","version":"master","gitSha":"3ff56e6b..."}
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
3ff56e6b72 |
fix(docker): API image could never boot — missing workspace link and Prisma engine
Two independent defects in docker/api.Dockerfile, both found by booting the published image on galactus rather than by reading it. Neither had ever been observed because no deploy had previously got far enough to start the API. 1. "Cannot find module '@jorgecuadros/database'". node-linker=hoisted flattens EXTERNAL dependencies into /repo/node_modules, but the workspace dependency stays linked per-package at apps/api/node_modules/@jorgecuadros/database -> ../../../../packages/database. The runtime stage copied only /repo/node_modules, so the link was dropped. Copy the @jorgecuadros scope dir as well — not the whole directory, whose only other contents are devDependencies. 2. "Prisma Client could not locate the Query Engine for runtime linux-musl-openssl-3.0.x ... generated for linux-musl". Prisma picks its engine by sniffing the build environment. The build stage had no openssl so it generated for plain "linux-musl", while the runtime stage demanded the openssl-3.0.x variant and refused to start. Fixed at both ends: binaryTargets now names the musl target explicitly in schema.prisma, so the shipped engine no longer depends on what happens to be installed at build time, and openssl is installed in the deps stage (generate) and the runtime stage (Prisma needs it regardless). Verified by running the published image on galactus with each fix patched in by hand, against the real database, until it got past both failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
27f04f1073 |
fix(deploy): preflight missing secrets instead of failing opaquely
The first deploy attempt (run 705) died on "Input required and not supplied: token", which names the action's input rather than the secret that was unset — the repo had only REGISTRY_USERNAME and REGISTRY_PASSWORD, so every deploy secret was missing on both workflows. That is also why the endpoint_id / pull_image input-name bug had gone unnoticed: neither workflow had ever got far enough to use them. Both workflows now check their required secrets up front and fail listing the ones that are empty. The scope=full-only secrets are only required when the dispatch is actually scope=full. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
4ee7ec71f0 |
feat(deploy): prisma migration history, /version, galactus standalone deploy
Closes the gap between "what tag did I deploy" and "what is actually running", and gives the schema a history that can be reasoned about across releases. Migrations - Baseline the existing schema as 0000_init (migrate diff --from-empty). The schema had only ever been applied with `prisma db push`, so no history existed and schema state was disconnected from app version. Existing databases must be baselined once with `migrate resolve --applied 0000_init`; the workflows print this remedy on P3005. - Run `prisma migrate deploy` as a deploy STEP, not the container CMD — as a CMD, N replicas would race each other applying the same migration. Version reporting - GET /version on the API reports the APP_VERSION / GIT_SHA / BUILD_DATE that build.yml already baked into both images but nothing ever read. - The web footer shows the web build and flags an api/web mismatch. The two cannot drift at build time (one matrix run) but can at deploy time. - Both deploy workflows now fail if the running API does not report the tag that was dispatched — a stack naming a tag is not proof of what is running. - scripts/set-version.mjs stamps every package.json, which had all sat at 0.1.0 while real releases shipped as v1.x. Pre-migrate backup - deploy/scripts/pre-migrate-backup.mjs dumps the database from INSIDE the still-running old API container over Portainer's Docker API, so the file lands in the volume the Operaciones restore screen reads. A dump taken on the CI runner would be unreachable by the only restore path we have. Verifies the artefact with `gzip -t` before letting the migration proceed. galactus - deploy/galactus/*.compose.yml: standalone-Docker ports of the Swarm stacks. Plain compose silently ignores `deploy:`, so restart_policy becomes `restart: unless-stopped` — without it nothing returns after a host reboot. - .gitea/workflows/deploy-galactus.yml drives endpoint 3 with its own secrets. Fixes - deploy.yml passed `endpoint_id` and `pull_image` to cssnr/portainer-stack-deploy-action, which has no such inputs (they are `endpoint` and `pull`). The endpoint was silently never set. docs/DEPLOY_AND_MIGRATIONS.md documents expand/contract as the rule for schema changes: Prisma has no down-migrations, so a code rollback never rolls the schema back, and restoring the replication master from a dump diverges every replica. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
9ba5d2d09a |
feat(bank): multi-bank chequera — required bankAccountId, per-account scoping
The office keeps more than one operating account (Utilities banks in MXN, Seguros in USD), but bank_transactions was a single implicit MXN register by design. Adds Bank/BankAccount and makes every read and write in the module scoped to exactly one account. Schema: - Bank / BankAccount. Currency is fixed per account and BankTransaction has no currency column of its own — a movement inherits its account's, the way a real bank account doesn't mix currencies. - BankTransaction.bankAccountId, required. A movement with no known account isn't reconcilable against a statement. - @@index([bankAccountId, transactionDate]): every read now filters by account and orders/groups by date. Migration: - backfill_bank_accounts.py seeds Scotiabank + "Utilities — Scotiabank (MXN)" and backfills all 22,669 existing rows onto it, then promotes the column to NOT NULL and attaches the FK. Standalone because prisma db push cannot add a required column to a populated table. Idempotent; re-running once a second account exists does not re-point rows. - run_all.py runs it (both modes) before transform_bank.py, which now resolves the account by label and fails fast if it is missing. API: - ?bankAccountId= required on list/stats/facets/summary — not optional with an "all accounts" default, since summing an MXN and a USD register repeats the currency-collapsing mistake the billing module exists to prevent. Missing is 400, unknown is 404. - facets() had no account clause at all and summary() has two raw-SQL rollups; all three are now parameterised. Scoping only one of summary's queries would leave the year list and its drill-down describing different books. - New bank/accounts + bank/banks sub-resource under a MANAGER bank:manage-accounts ability. currency is absent from the update DTO: booked movements are denominated in it, so editing would re-denominate history. Capture into a closed account is rejected. Web: - /banco gains an account picker (remembered per browser) and reads every figure in the selected account's currency; the "single currency (MXN)" doc-comment and the hardcoded MXN formatting are gone. - New /banco/cuentas for banks and accounts. Accounts are closed, never deleted — the FK is required, so deleting one would destroy its register. - /inicio's chequera card names the account it is reading instead of implying a single register. Verified against dev + browser: a second USD account showed full read/write isolation from the MXN register, whose totals were unchanged (22,669 movements, net 1,014,266.97). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
c100dfa224 |
feat(web,api): scale spacing with text size, persist preference per account
Two follow-ups to the text-size control. Spacing now scales with the text. All padding, margin, gap and min-height declarations in globals.css move from px to rem (263 declarations, converted mechanically), so --ui-scale drives the whole layout rather than just the glyphs. Deliberately left in px: border widths, which must stay hairlines; box-shadow offsets; border-radius, which reads as bloated when scaled on large cards; --shell-max, a container cap that must not outgrow the viewport; and media-query breakpoints, which are conditions rather than declarations. With spacing following along, the presets gain a 1.5 "Máximo" step and MAX_UI_SCALE rises from 1.4. The preference now lives on the account instead of only in one browser. User.uiScale (Float, default 1) is added to the schema and to the safe select, so it rides along on /auth/login and /auth/me. PATCH /auth/preferences writes it, guarded by AuthenticatedGuard only — every role including VIEWER may set their own, and the target is always the session's user id, never a body parameter, so this cannot be used to touch another account. The global ValidationPipe's whitelist rejects any extra field, so role cannot ride in alongside uiScale. localStorage stays, demoted to a pre-paint cache for the layout.tsx script; AppShell reconciles it against the account once /auth/me answers, with the account winning. FontScaleControl becomes a controlled component since the same value is now edited from the appbar and the drawer. Verified against the dev API: PATCH persists and is reflected by a subsequent /auth/me, out-of-range values are rejected 400, and an extra "role" field in the body is rejected 400. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
0bf97e6d2c |
feat(web): group top nav, add mobile drawer and app-wide text size control
The appbar had grown to 11 flat links with no responsive behaviour, and overflowed below ~1100px. Nav is now 7 top-level entries: Inicio, Clientes, Pólizas, Propiedades, Reportes stay one click away, while the movement screens (Captura, Estado de cuenta, Chequera) and the admin screens (Catálogos, Usuarios, Operaciones) collapse into "Cobranza" and "Admin" dropdowns. Groups are ability-filtered and disappear entirely when the user can see none of their items, so VIEWER never renders an empty Admin menu. activeHref now scans the flattened link list, and a group trigger highlights while one of its children is current. Below 980px the nav collapses to a burger drawer that lists every group expanded, closing on navigation and on Escape. Text size is user-adjustable app-wide. Every font-size in globals.css is converted from px to rem (mechanically, 133 declarations) and the root size becomes calc(100% * var(--ui-scale)), so one variable on <html> rescales the whole UI. The preference persists in localStorage and is applied by a pre-hydration script in layout.tsx to avoid a flash at the default size; the Aa control lives in the appbar and, as a segmented row, in the drawer. Spacing stays in px by design, which is why 1.3 is the largest preset. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
7df928c3ab |
feat(billing): receipt capture — outstanding workflow, batch by check, reconciliation
Implements docs/RECEIPT_CAPTURE_SPEC.md §1, the legacy "Editor" replacement, on top of the single-movement capture from plan step 6. No new abilities: batching and resolving are both capturing. - outstanding (legacy NOPAGO): capture flag, ?outstanding= filter, and POST /billing/:id/resolve-outstanding (gated ledger:create, not ledger:void — resolving completes a capture rather than reversing one). Outstanding rows are excluded from every balance aggregate, matching the legacy SALDOS ULTIMO 0 query's HAVING NOPAGO = 0, but still count in the movement browser's filtered totals. - POST /billing/batch: many customers' receipts against one check, in one $transaction. Deliberately not a persisted batch entity — checkNumber is already a column and grouping by it answers every legacy by-check query. - GET /billing/by-check + a cheque-count report, replacing REPORTE CHEQUE COUNT / REPORTE POR CHEQUE / EDITA CHEQUE ALF|COUNT|NUM. Print, PDF, CSV and XLSX come free from the existing /reportes/:slug machinery. - Web: /estado-cuenta/lote (the Editor screen, with live reconciliation against the physical check amount), an "Estado de pago" filter, a "sin fondos" row tag and a Resolver dialog, plus a top-level "Captura" nav entry. Integration seam for the OCR auto-capture module (spec §2), which is required to post through createBatch rather than writing Transaction rows itself: items[i] maps to lines[i] so postedTransactionId can be zipped back on; opts.refs[i] stamps captureRef with a duplicate-post guard that a voided row deliberately does not block; opts.source is service-level only, so an HTTP client cannot label hand-keyed rows as machine-captured. captureSource/captureRef are nullable so the 40,136 migrated rows stay NULL rather than being mislabelled. Fixes two pre-existing bugs found while building this: - statement() filtered legacySourceTable with `notIn`, which compiles to SQL NOT IN — and `NULL NOT IN (...)` is NULL, so every app-captured movement was invisible on the customer statement (438 rows in the movement browser vs 392 on the statement) while showing everywhere else. This would have made the whole capture feature look broken. - The balances count query omitted the void filter its own page query applied, so the total disagreed with the rows. Nav highlighting now resolves by longest match; the previous first-startsWith logic lit up both the parent and any nested entry. Verified end-to-end against the dev DB, API and browser; all test rows removed afterwards. Also corrects RESUME.md, which documented the dev ports as :3001/:3000 — they are :4501/:4500, from the env files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
26a4faa33e |
docs(insurance): spec renewal emails, liquidación batch, certificate, carrier APIs
Companion to docs/RECEIPT_CAPTURE_SPEC.md — the insurance half of the 2026-07-25/26 meeting. Documentation only; no application code. Verified against the code and a live query of the dev DB rather than designed from the meeting notes alone, which changed several conclusions: - Renewal emails and the liquidación batch are much smaller than they look. RenewalNotice + its @@unique([policyId, generation]) idempotency key and the aviso-renovacion letter body already exist; the per-policy liquidation fields are wired end to end. What's missing is a scheduler, a mail client, and the batch layer. - Carrier research: ANA and GMX are one company (Grupo Valore). ANA exposes a live SOAP service with a published operation list; GMX publishes no machine interface at all. Every ANA operation serves new-business quoting/issuance, not "list my book" — so the direction question decides whether the feature is buildable. - UTILSEG is unusable for Utilities↔Seguros reconciliation and the spec closes that long-standing open question: DATGRAL.[NUM UTIL] is authoritative (name match 298/563 vs 58/1024), and where the two sources overlap they contradict on 170 of 218 shared ids. Also records two live defects found while verifying: policy_types is missing its INCENDIO and M_EMPR rows (the FK is ON DELETE SET NULL, so 5 m_empr policies silently lost their ramo), and the legacy settlement slots don't match the target model (MULT/INCENDIO carry two, M EMPR carries four, Policy collapses to one). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Sonnet 5
|
159dcc4963 |
docs(plan): add step 11 for receipt-capture + net-new ops features
Points PLAN.md at docs/RECEIPT_CAPTURE_SPEC.md and surfaces its open design questions (OCR provider, Seguros bank details, clave catastral vs. predial, recycling triggers) separately from the existing ops-only open items list. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Sonnet 5
|
9b9ee201c9 |
docs: add receipt capture, OCR, multi-bank & customer-recycling spec
Forward implementation spec covering the legacy "Editor" receipt-capture workflow plus three net-new requests from the 2026-07-25/26 meeting with Jorge: PDF/OCR auto-capture, multi-bank chequera support, and customer-number recycling. Matching logic and data-model gaps for each were verified against the actual migration scripts and API code, not just the schema comments. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
6dbd4a319b |
fix(reports): render renewal-letter premium lines as booleans
`r.netPremium` / `r.total` are strings, so an empty-string value leaked ""` into the JSX instead of rendering nothing. Wrap in Boolean() so the guard is a real conditional. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Sonnet 5
|
f7ae0d5342 |
feat(reports): parameterized renewal-notice report + legacy report reference
Replaces ~40 legacy Access renewal-notice report clones (one per carrier per coverage tier, e.g. AMPL/RC/LIC RENEW X MES/VENCE ATLAS 13/2013) with one parameterized aviso-renovacion report driven by real Policy/Vehicle/ coveragesJson data instead of hand-typed label text per clone. - schema.prisma: add RenewalNotice, replacing the legacy CONTROL <ramo> RENEW[2/3] X MES paper log of which notice generation was sent - reports: new "letter" ReportFormat + aviso-renovacion registry entry + LetterLayout renderer in ReportRunner.tsx - docs/RENEWAL_NOTICES.md + migration/legacy_report_defs/: extracted (via Application.SaveAsText, since the VBA project wouldn't load) and documented the legacy report/query chain this replaces Coveragesjson key names and a mark-as-sent mutation are still unverified/ unbuilt — see caveats in docs/RENEWAL_NOTICES.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 4.8
|
1b79b43a54 |
fix(migration): make Phase B additive sync actually work + verify end-to-end
The --sync path had never been run and was broken in several ways. Fixed and verified against the dev DB (two consecutive syncs, both exit 0, 32/32 assertions: stable PKs, manual-row preservation, changed-row updates, legacy-delete, no child duplication, zero FK orphans; idempotent). - policies/properties: reuse each legacy row's existing id (by provenance) BEFORE building child rows, so children no longer point at a discarded fresh uuid; rebuild legacy-owned children via scoped delete + reinsert. - customers: replace zip(customers, refs) (mispaired almost every row) with a ref-grouped id remap; names now restore and no spurious customers appear. - drop the invalid Vehicle @@unique(legacySourceTable, legacyId) — one legacy policy row carries up to 3 vehicles sharing a legacyId; handle via delete+reinsert. - upsert lookup tables (policy_types, insurance_providers, type_transactions, adjusters) by natural name and remap child FKs instead of inserting fresh uuids that nothing points at. - transactions: drop updatedAt=NOW() (no such column); guard report formatting on NULL legacySourceTable (manual rows). Same report guard in bank. - add manual-safe prune (prune_empty_customers.py --sync, in SYNC_STEPS): prune only legacy-owned empties, never manually-added customers. web: customer-detail mini tx list now strikes voided rows with an "(anulado)" tag (was the last void-UI rendering gap; /estado-cuenta already handled it). docs: RESUME.md updated — Phase B sync marked verified end-to-end, void-UI browser pass recorded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinas
|
8802f08d4f |
feat(reports): reports module + /inicio + edo-cuenta-datos prefill
- New reports backend (registry, service, controller, outputs, types) with catalog endpoint + slug/CSV/XLSX/PDF/print outputs. - /reportes catalog + /reportes/[slug] runner; ReportRunner + ContextReports components wire pre-filtered links from domain pages. - Fix: /reportes/[slug] now reads searchParams and forwards initialParams to ReportRunner so /reportes/edo-cuenta-datos?customerId=... auto-runs instead of dropping the id and forcing a manual customer search. - /inicio landing page; root + login redirect to /inicio. - Company header env vars + logo asset for PDF/print rendering. - exceljs + pdfkit deps. |
||
|
rmancinas
|
921a47cbaa |
feat(web): use company logo PNG in brand mark
Replace the JC text mark in AppShell and login with the real company_logo.png. Restyle .brand-mark to host the image (white rounded bg, object-fit contain). Appbar 38px, login panel 46px. |
||
|
rmancinas
|
27b3bd9efc | feat: expand admin and data sync workflows | ||
|
rmancinasandClaude Opus 4.8
|
70911e7e62 |
feat(deploy): app stack + manual Portainer deploy workflow
Add the missing api/web deployment path on top of the existing image build CI. - deploy/jorgecuadros-app.stack.yml: PROD app stack (api + web) pulling the git.mancinas.io registry images. Does not ship mysql/minio (separate stacks); API reaches them via DATABASE_URL / S3_ENDPOINT. API pinned to the jorgecuadros_db node for stable ingest/backup volumes; web is stateless. - deploy/jorgecuadros-app.env.example: documented stack env template. - .gitea/workflows/deploy.yml: manual (workflow_dispatch) deploy to Portainer via cssnr/portainer-stack-deploy-action. Inputs: image tag + scope (app = web+api, full = db+minio+app, applied db->minio->app). Make the web API origin runtime-configurable instead of build-baked: the root layout injects window.__API_ORIGIN__ from the API_ORIGIN env (force-dynamic) and lib/api.ts resolves it at runtime, so one built image serves any deployment. Also: dev.sh to run both dev servers (frees stale ports first) and move local dev to ports web 4500 / api 4501. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 4.8
|
afe2411c86 |
feat(storage): wire MinIO/S3 document upload & download into the API + web
The schema has carried `storageKey` pointers and the migration has written blobs to MinIO since day one, but the API had no S3 client — documents could only be deleted, never uploaded or retrieved. This adds the missing wiring. API - StorageModule/StorageService (@aws-sdk/client-s3, path-style for MinIO): put/getStream/delete, best-effort bucket ensure on boot, gracefully disabled when S3 env is absent (ServiceUnavailable on use). - Reads S3_ENDPOINT/S3_BUCKET + S3_ACCESS_KEY/S3_SECRET_KEY, falling back to MINIO_ROOT_USER/MINIO_ROOT_PASSWORD so one credential set drives both the migration and the API. - Property service documents: POST :id/documents (multipart), GET :id/documents/:childId/download (streamed), delete now also drops the blob. - Policy documents: same upload/download/delete (previously had none). - Keys stay under the service/<id>/… and policy/<id>/… prefixes the migration established. Web - api.ts: shared uploadFile() helper (uploadIngest refactored onto it), upload/download/remove helpers for property & policy documents. - Servicios, polizas, clientes detail pages: real Descargar links and an upload control (gated by policy:update / property:update) replacing the "storage pending" notes. Infra - docker-compose: minio service (9000/9001, healthcheck, named volume) + S3 env wired into the api service. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinas
|
45afb824ef | Merge feat/crud-rbac: CRUD/RBAC + ops panel + Docker CI/versioning | ||
|
rmancinasandClaude Opus 4.8
|
f1ef1c70b3 |
wip: ops admin panel + migration sync + crud/rbac phase-5 snapshot
Working-tree checkpoint of in-progress work carried across prior sessions on the feat/crud-rbac branch, committed so it lands on the remote alongside the CI changes. - Operaciones admin panel: apps/api/src/ops (ingest upload, backup / restore / re-import jobs) wired into app.module + RBAC abilities, and the apps/web/src/app/operaciones page. docker-compose gets INGEST_DIR / BACKUP_DIR volumes; .gitignore excludes migration/ingest + backups. - migration/sync.py plus transform_*.py / run_all / config / dbenv / blob_extract adjustments for the additive sync path. - crud/rbac phase-5 web bits: AppShell, api/labels/types libs, globals. - schema.prisma + PLAN/RESUME doc updates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 4.8
|
6ad0993a71 |
ci(docker): versioned image builds + Gitea build/push workflow
Add comprehensive Docker image versioning and a Gitea Actions workflow that builds and pushes both the API and web images to the git.mancinas.io registry. Versioning: both Dockerfiles take APP_VERSION / GIT_SHA / BUILD_DATE build-args, surfaced as runtime ENV + OCI labels, so a running container self-reports the exact commit it was built from. metadata-action emits a tag set per build: semver (from vX.Y.Z git tags), branch ref, sha-<short>, and latest (default branch only). Also fix the Dockerfiles for the pnpm workspace: the old npm install could not resolve the "@jorgecuadros/database": "workspace:*" protocol dep and would abort the API build. Now pin pnpm 9.15.9 via corepack, install --frozen-lockfile with node-linker=hoisted (flat tree so the runtime stage copies a single node_modules), and build via --filter. The API build stage gets python3/make/g++ for argon2's musl source compile. Add .dockerignore to keep the build context lean and deterministic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 4.8
|
9dc7f26e02 |
docs: add comprehensive README with run instructions
Covers stack, repo layout, local dev (pnpm install, env, MySQL via docker, prisma db push, seed admin, run api+web, login), full-stack Docker path, common commands, auth/roles, legacy migration, and prod notes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 4.8
|
0260b8110d |
fix(catalogos): render child editor in-place of edited row
The add/edit form was appended after the whole table, so on long lists (e.g. /catalogos aseguradoras, 16+ rows) clicking Editar on a top row opened the form far below the fold — appearing to do nothing. Render the edit form in place of its row, and the add form as the first table row. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 4.8
|
7d9f59e51b |
feat(billing,bank): capture + void web UI (plan phase 5 web)
Completes phase 5 — the ledger and chequera pages get the append+void UI on top of the phase-5 API. Web: - Shared MovementForm (customer picker + línea + cargo/abono sign + amount + moneda + concepto facet + periodo/referencia/cheque/mensaje). Used by both /estado-cuenta (cross-customer, picker) and /estado-cuenta/[id] (customer prefilled). - /estado-cuenta and /estado-cuenta/[id]: "Capturar movimiento" toggle gated ledger:create; per-row "Anular" gated ledger:void; voided rows struck-through. Save/void refresh the list + stats. - /banco: inline BankCaptureForm (ingreso/egreso sign, cheque, operado, transferencia, monto en letras) gated bank:create; per-row "Anular" gated bank:void; voided rows struck-through. - api.ts: createMovement/voidMovement, createBankMovement/voidBankMovement; CreateMovementInput/CreateBankMovementInput types; `voided` on the movement/statement/bank list items. Also: lookups.controller.ts now audit-logs provider/policy-type/adjuster create/update/delete (parity with the other write controllers). API + web compile clean. This is the last piece of the feat/crud-rbac branch — all five sections plus users are now full CRUD with role gating. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 4.8
|
548eeb5798 |
feat(ledger,bank): append + void write API, voided excluded from totals (plan phase 5 API)
Transactions and the bank register become append-only with a void (reversal) action — never edited or hard-deleted. This is the API half of phase 5; the capture/void web UI is the remaining piece. Schema: - Transaction and BankTransaction gain voidedAt + voidedById. A non-null voidedAt reverses the row. Pushed to dev. Correctness (the high-stakes part): - Every aggregate excludes voided rows: billing movements totals, the raw balances SQL, stats (groupBy + the sides/crossLine raw subqueries + first/last), facets (types/sources/years); the statement's running balance freezes on a voided row and its per-currency/per-domain/per-type summaries skip them; customers.detail and property owner-ledger groupBy; and every bank total (totalsFor, stats counts/bounds, facets + summary raw SQL). List views still return voided rows with a `voided` flag so the UI can strike them through. - Bank's legacy zero-amount "void" cheques are unchanged and distinct from app voids (voidedAt). API: - POST /billing + POST /billing/:id/void (ledger:create / ledger:void); POST /bank + POST /bank/:id/void (bank:create / bank:void). Create needs STAFF+, void needs MANAGER+. Double-void -> 400, unknown id -> 404, bad date -> 400. Mutations audited. DTOs added. Verified against dev end-to-end: a -500 MXN charge moved a customer balance 31082.08 -> 30582.08, and voiding it returned it to 31082.08 to the cent; a +1234.56 bank ingreso moved net 899375.77 -> 900610.33 and voiding returned it to 899375.77. VIEWER create/void both 403, double-void 400. API compiles clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 4.8
|
506f8ce684 |
feat(properties): CRUD + service/trust/document editors (plan phase 4)
Utilities section becomes create/edit/archive-able, with its child data. API: - Property gains archivedAt (soft-delete); list/browser default to archivedAt=null with ?includeArchived opt-in. - PropertiesService: header create/update/archive/restore (customer FK validated); PropertyService add/update/remove scoped to the property; TrustAccount upsert (1:1) + remove; ServiceDocument pointer delete. - Controller write routes: create needs STAFF+ (property:create), archive MANAGER+ (property:delete), every service/trust/document route property:update. Mutations audited. DTOs added. - Document *upload* deliberately deferred: it needs the object-storage client wired into the API (today only the migration writes to MinIO); removing an existing pointer row is supported and the UI says so. Web: - PropertyForm (header) with CustomerPicker; /servicios/nuevo (accepts ?customerId prefill) and /servicios/[id]/editar. - Property detail: gated action bar (Editar/Archivar) + "Administrar propiedad" — services via the shared ChildCollection editor, an inline 1:1 TrustEditor (create/update/clear), and document-row delete. - "Nueva propiedad" buttons on the list and customer detail (prefilled). api.ts + types for all of it. Verified against dev: property create (archivedAt null), service add/update, VIEWER service-add 403, trust upsert (create then update the same row), trust/service remove, cross-property child guard 404, archive drops from the default list and includeArchived surfaces it. Both apps compile clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |