Mirrors the utility statement intake on the insurance side: a policy_ocr
batch/document pair of tables, a GMX parser, a matcher keyed on
Policy.policyNumber, and a "Captura" screen under /polizas that proposes
policy -> customer for staff to confirm.
Lifts the OCR seam out of StatementsModule into its own OcrModule so
PolicyOcrModule can inject OCR_PROVIDER without taking on the rest of
the statement pipeline; StatementsModule now imports it and binds
nothing itself.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the ZONA FEDERAL TIJUANA parser to the statement intake, measured
against 8 pages of real "Zona Federal Marítimo Terrestre" receipts — the
federal maritime-zone occupancy fee the municipality bills on beachfront
lots. Provider read on 8/8, amount on 8/8 (each verified against the
paper), concession clave on 6/8, period on 8/8, deadline on 2/8.
Four things the corpus forced:
- Tijuana bills predial and zona federal from the same treasury: same
header, same Paseo del Centenario address, same ATB-541201 RFC. Every
predial discriminator matches a zona federal page too, so whichever
rule is asked first wins it. The only words exclusive to this layout
are "Marítimo Terrestre", so its brand rule is asked ahead of all
three predial ones — and its structural rule, anchored on the stub's
"Derechos de ocupación", ahead of theirs.
- FEDERAL_ZONE.accountNumber is an amount, not a reference. It holds
DATMEX.zfed, whose 77 values include 246.06, 2369.09, 22653.94 and a
negative -1679, while the concession claves these receipts are keyed
by appear nowhere in the database. Matching on that column could never
hit — and because every row already has a value, the `[field]: null`
guards on learnAccountRefs and on the review blank-service fill would
never fire either, so every page would return to the queue every
bimester forever. The clave moves to meterNumber, joining gas and
Tijuana predial, and the first confirm teaches the match.
- The payable figure is not the printed subtotal. The municipality
rounds to whole pesos and prints the difference on its own "Ajuste Ley
Hacienda Mpal" line (-$0.05 against a 591.05 subtotal, $0.21 against
2,872.79). The "Total a pagar" box carrying the rounded figure sits on
a grey fill and OCR'd on 1 of 8 pages; the SubTotal row read on 8 of
8. So the amount is the rounded subtotal, cross-checked against the
printed box wherever it survives — where it did, it agreed.
- The clave is 2 digits, a letter and 3 digits (12-T -012), not the
cadastral shape, and the letter is kept as printed: toDigits maps D to
0, which turns a real 14-D -014 into 140014. It is printed twice,
which rescued a page whose heading was struck through by the office's
own highlighter — the failure mode behind both missing claves.
Deriving the deadline from the bimester is deliberately not attempted:
it is the 17th of the month after the bimester closes on a current bill,
but four of these eight are late (a $1,000 Multa) and print a
recalculated date, so a derived date would be wrong on exactly the pages
a human most wants to see.
Re-ran the earlier corpora (25 pages: predial Tijuana/Rosarito/Ensenada,
CFE, CESPT, Telnor) through detection to confirm the new rules steal
nothing — all 25 still read as their original provider, including the
five Tijuana predial pages that share the RFC.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds four parsers to the statement intake — GAS TIJUANA plus one per
municipality, because Tijuana, Rosarito and Ensenada issue three
completely different predial documents — and a text-layer fast path for
the born-digital invoices the gas company sends.
Measured against a new corpus of 14 documents / 29 pages: provider read
on 29/29, amount on 26/29, and 21/29 auto-matched against the dev
database (22/29 identified). The eight review cases are all legitimate.
Five things the corpus forced:
- Not every statement is a scan. The gas invoices are born-digital CFDIs
whose text layer is exact; rasterising them only loses information (one
sample turned `MEDIDOR: VM01014426` into `ar (LTR): 014420`). The new
`OcrProvider.textPages` reads the embedded layer via `pdftotext
-bbox-layout` — same poppler package as `pdftoppm`, so no new
dependency — and OCR stays the fallback for real scans. Poppler's own
`<line>` grouping follows text flow rather than the page, so words are
regrouped by vertical position; without that, a two-column header
leaves every label separated from the value printed beside it.
- The clave catastral is not two letters and six digits. Position three
is a letter in 15 of the 932 stored claves, and digitising the whole
tail mapped a real `MMB01041` to a nonexistent `MM801041`.
- Tijuana predial prints no clave at all. Its only identifier is an
8-digit municipal account carried in a 32-digit payment barcode, which
the legacy database never held, so it goes in `meterNumber` alongside
gas — `accountNumber` holds `DATMEX.predial`, which is not a
per-property key and must not be overwritten. Those pages start cold
and are taught by the first confirm.
- On Rosarito and Ensenada the clave is the primary key, not a fallback:
those receipts print nothing else, so a unique hit auto-matches. On a
utility bill that merely happens to print one it stays a review hint.
- A misread `$` is the dangerous failure. An Ensenada receipt for
$2,203.00 OCR'd as `82,203.00`, which would post a charge 37x too large
and look ordinary in the ledger. Predial amounts now require a literal
`$` and a page that cannot produce one goes to review.
The scoped match field is now one exported function rather than three
copies of `kind === "GAS" ? ... : ...`, since the lookup, the
blank-service fill and the confirm write-back have to agree or a
reference gets learned into a column nothing searches.
First tests in this package: 23 specs over the parsers and the text-layer
reader, every fixture a verbatim OCR excerpt from a real receipt. Adds
the jest config they need and a build tsconfig so they stay out of dist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Backend already returns per-status counts via byStatus; render a real
progress bar (X% / N de M / en cola) while PENDING_OCR pages remain,
using the existing progress-track CSS. Falls back to indeterminate
when no docs have been reported yet.
release.yml pushes the release commit and its tag in a single `git push`,
so Gitea created two build.yml runs for the same commit. Only the tag run
matters: it emits the X.Y.Z and X.Y image tags, and since it is the same
commit it publishes `latest` and `sha-<short>` as well. The master run was
pure duplicate work that had to be waited out or cancelled by hand.
Guard the build job with an `if` that skips a branch push whose head commit
message starts with `chore(release):`. Ordinary pushes to master are
unaffected, and tag pushes and manual dispatches always build.
The skipped master run keeps the release commit's sha, which would have let
release.yml's "Verify build.yml started" check go green on it alone even if
the tag run were never created — the exact failure that check exists to
catch. It now also requires the run's ref to be the tag, falling back to the
sha match only when the API reports no ref.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every transform resolves its target through dbenv.database_url(), which lets a
DATABASE_URL in the process environment win — that is how the API container
drives a re-import against its own database with no deploy/ directory present.
blob_extract.py was the one step that bypassed it and called load_env()
directly for the MinIO credentials, so the "Operaciones" re-import loaded all
the data and then exited 1 on:
missing /repo/deploy/.env.prod — deploy the 'prod' DB stack and write its
.env first
Give the S3 settings the same resolution as the DB URL: load_env() now returns
{} for an absent file, and setting()/require() layer the process environment on
top of it. blob_extract reads S3_ENDPOINT / S3_BUCKET and accepts either
S3_ACCESS_KEY/S3_SECRET_KEY or MINIO_ROOT_USER/MINIO_ROOT_PASSWORD, matching
the fallback order in storage.service.ts and the vars the api service already
sets in deploy/galactus/jorgecuadros-app.compose.yml. A genuinely missing
setting still fails fast, now naming the variable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The API image installed Alpine's `mdbtools` package, which ships only the
shared library. The command-line tools that migration/extract.py actually
shells out to -- `mdb-tables` and `mdb-export` -- are in the separate
`mdbtools-utils` subpackage, so the build succeeded and the re-import in the
"Operaciones" admin panel failed at run time with:
RuntimeError: mdbtools not found on PATH (need mdb-tables and mdb-export)
Install `mdbtools-utils` instead; it pulls the library in as a dependency.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gitea's workflow dispatch API 404s on a bare `v1.0.3` and accepts only
`refs/tags/v1.0.3`, so the fallback added in fdbe9fd would have failed
the release instead of rescuing it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gitea creates workflow runs from the post-receive hook. When that hook
errors the refs still land, git prints `remote: error: Internal Server
Error` and exits 0 — a post-receive failure does not fail a push. v1.0.3
was cut exactly that way: tag pushed, no build run created, no images
published, and the release step green. It surfaced two steps later as a
404 when the deploy tried to pull 1.0.3.
Capture the push output and warn on `remote: error`, then verify a
build.yml run actually exists for the new commit, dispatching it against
the tag if not. Fail the release if that does not take either, so a
release that publishes nothing is red instead of green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every backup on galactus died with:
mysqldump: unknown variable 'set-gtid-purged=OFF'
respaldo incompleto eliminado
Alpine's mysql-client is MariaDB's, so `mysqldump` inside the API
container is a shim over `mariadb-dump`, which has no --set-gtid-purged.
That took out BACKUP and, because they take a safety dump first, SYNC
and REIMPORT too.
Probe `mysqldump --help` and pass the flag only when it is advertised,
calling `mariadb-dump` directly otherwise — MariaDB writes no GTID state
unless asked with --gtid, so there is nothing to suppress. Testing
whether mariadb-dump merely exists would be wrong: on a host carrying
both clients it would shadow a perfectly good MySQL mysqldump.
The probe uses a command substitution rather than `--help | grep -q`
because PIPEFAIL is in effect for these commands and grep closing the
pipe early would report a supported flag as unsupported.
pre-migrate-backup.mjs is unaffected — it dumps from a real mysql:8.4
image, not from the API container.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The ingest upload used fetch(), which cannot report request-body
progress, so the only feedback was a static "Cargando…" label — no way
to tell a stalled 2 GB upload from a working one.
Switch uploadFile() to XMLHttpRequest and expose an optional onProgress
callback reporting loaded/total bytes, a smoothed transfer rate and a
remaining-time estimate. The Operaciones ingest table renders a progress
bar row under the file being uploaded. Once the bytes are all sent the
server still has to write the file, so that tail reads "Procesando en el
servidor…" rather than parking at 100%.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stamps every package.json, commits chore(release): vX.Y.Z, tags and pushes
both refs in one dispatch — patch/minor/major, or an explicit number. Cutting
a release from a laptop is how a manifest bump gets forgotten or a tag lands
on an unpushed commit; the only input here is the number.
Guards: refuses a version that already exists as a tag (releases are
immutable), a no-op bump, a leading `v`, and a malformed number. Checkout is
full-depth because the duplicate-tag check is meaningless against a shallow
clone.
Pushes with a RELEASE_TOKEN PAT rather than the built-in Actions token —
whether a push made with that token re-triggers build.yml depends on the Gitea
version, and a release that quietly publishes no images is worse than one that
fails outright.
Builds and deploys stay separate: the tag push triggers build.yml, and
deploying remains a deliberate dispatch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scanning a stack of bills and keying them in are the same daily job, ending
in the same ledger path, so OCR intake becomes a mode of the capture screen
instead of a second menu entry:
- components/Captura.tsx holds the mode switch; the manual check form moves
verbatim to components/ManualCheckCapture.tsx and the OCR intake to
components/StatementIntake.tsx.
- /estado-cuenta/lote opens on manual, /recibos on automatic — both render
Captura, so batch-review links and old bookmarks still land right.
- Nav drops "Recibos (OCR)"; "Captura" covers both, with a NavLink.aliases
field so /recibos still highlights it.
Also fixes the "El almacenamiento de documentos no está configurado" failure
staff hit on upload. Uploading with no object storage configured used to
succeed, then die on the first put minutes later, leaving a FAILED batch
whose only explanation was that string. createBatch now refuses up front,
GET /statements/status reports storageAvailable alongside ocrAvailable, and
the intake tab explains the situation instead of offering an upload that
cannot work. S3_* documented in .env.example (deploy stacks already set it).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Staff key 300+ utility statements per company per month by hand. This adds
the ingest -> split -> OCR -> match -> review pipeline that proposes customer
and amount per page instead (RECEIPT_CAPTURE_SPEC §2), posting through the
existing BillingService.createBatch seam with source=OCR and a per-document
captureRef so machine and hand capture share one write path and audit trail.
Everything was designed against 10 real scanned statements (46 pages of CFE,
CESPT and Telnor bills) rather than from the sample-free spec. The scans have
no text layer at all — they are camera images — so OCR is mandatory, and they
arrive bundled one customer per page. Measured on those pages the parser
identifies the provider 46/46 and reads an account reference 43/46; against
the dev database that is 39/46 (85%) exact auto-match, 40/46 identified, with
the rest genuine review cases. That closes the OCR-provider question in favour
of self-hosted Tesseract: it clears the bar for a queue where a human confirms
every row, and OcrProvider keeps a managed API a one-line swap.
The samples corrected three things the spec had wrong or unknown:
- Clave catastral is NOT predial. DATMEX.clave (934 rows) is what CESPT and
predial bills print; DATMEX.predial, which PROPERTY_TAX.accountNumber holds,
has 663 distinct values across 1135 rows and appears on no statement. The
clave now lives on Property.cadastralKey as the matcher's secondary key;
predial is left untouched. This had been blocking predial matching.
- Gas was recoverable: 160 of 334 DATMEX.gas values are real account numbers
(the rest are ESTACIONARIO/CILINDRO descriptors), now in GAS.meterNumber.
- Phone is one billed line per property (534/18/1 across phone1/2/3), so the
new TELEPHONE ServiceKind backfills from phone1 only, not three rows.
Matching is scoped to one column per service kind and never reads the customer
name — a CESPT receipt prints ARNAIZ ROSAS ELSA AURORA for an account this
office holds under CATT, RANDY, because the printed name is the registrant,
not the current owner. Where a provider prints a payment barcode it beats the
printed label (one CFE label OCR'd a digit too many while its barcode was
correct) and the two cross-check, with disagreement forcing review.
Confirming a document whose service had no reference writes it back, so gas
and any other cold start is a one-time cost rather than a permanent queue.
Verified end to end against the live dev API and MinIO: real scans uploaded
over HTTP, matched, confirmed against a check, and the resulting rows checked
in MySQL (negative amounts, captureSource=OCR, concept derived from the batch
kind, captureRef linking back to each page). Re-confirming a posted batch is
refused. Test data was removed afterwards.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
v1.0.0's images were built from db2bd54, which predates the full-hash
footer. Deploying 1.0.0 would ship the abbreviated footer, so the version
that actually goes to galactus is 1.0.1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The footer abbreviated to 7 characters, so the line read
"v master · 19f0319". That line exists to be pasted into `git show` or
compared against a registry tag, and an abbreviation makes both a manual
step — while the full 40-char value was already baked into the image
(build.yml passes `github.sha` whole, and /version returns it untouched).
`shortSha` had no other caller, so it goes with it.
The span gets `overflow-wrap: anywhere` and `min-width: 0`: hex offers no
break opportunity, and the api/web mismatch branch renders two of these
hashes side by side, which would otherwise push a phone into horizontal
scroll. Measured at a simulated 360px with both hashes present — the span
wraps, and documentElement.scrollWidth stays equal to clientWidth.
Verified in the browser against the dev database: footer renders
"v1.0.0 · db2bd54c0ffee1234567890abcdef0123456789a", hash length 40.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every manifest still read 0.1.0 while the deployed images were addressed by
the moving tag `latest`. That combination is what hid the stale-image bug:
a checkout could not be placed against a running container, and `latest`
silently kept serving two-commit-old web code through a green deploy.
Tagging v1.0.0 makes docker/metadata-action publish immutable `1.0.0` and
`1.0` image tags, so deploys can name a version instead of a moving target.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Operaciones panel (backup, restore, sync, re-import) shelled out to
mysqldump as the application user, parsed straight out of DATABASE_URL.
`--single-transaction` issues FLUSH TABLES, which needs the global RELOAD
privilege, and the app user is granted only ALL ON jorgecuadros.* plus
USAGE ON *.*. BACKUP failed outright; SYNC and REIMPORT failed with it,
since both take a safety backup first.
An admin credential is now supplied out of band via OPS_DB_ADMIN_USER /
OPS_DB_ADMIN_PASSWORD, mirroring what deploy/scripts/pre-migrate-backup.mjs
already does, rather than permanently elevating the user the API serves
requests as. Host, port and database still come from DATABASE_URL, so the
override can only change who logs in, never which server. Unset, it falls
back to the DATABASE_URL credentials and warns — local development is
unaffected.
Two defects in the dumps themselves, both shared with the deploy backup
before it was rewritten:
- No --set-gtid-purged=OFF. The production server is the replication source
with GTID on, so every dump embedded SET @@GLOBAL.GTID_PURGED and was
unrestorable onto the server it came from — the one thing the restore
screen is for.
- The pipeline's exit status was gzip's, and gzip succeeded. A mysqldump
that died on its first statement left a small, perfectly valid archive
that the job recorded as SUCCESS and the restore screen listed as an
ordinary restore point. Dumps now run under `set -o pipefail`, assert a
CREATE TABLE count, and delete their own output on failure. Verified with
a stubbed mysqldump: a failing dump exits 1, surfaces the real error,
removes the partial file, and — critically — stops SYNC/REIMPORT before
the ETL touches anything.
Restores gained pipefail too: a corrupt archive made gunzip fail while
mysql, fed a truncated stream, could still exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pre-migrate backup ran INSIDE the API container, which made it depend on
that image's toolchain — and deadlocked: the running image shipped a MySQL
client that could not authenticate, so the backup failed, which blocked the
very deploy that would have replaced the broken image. A backup must not depend
on the thing being deployed.
The dump now runs in a throwaway container built from mysql:8.4 with the API's
backup volume mounted. The volume name is discovered from the API container's
mounts, so the file still lands where the Operaciones restore screen looks. As
a container rather than an exec, its logs can simply be read — no more failures
reported as a bare exit code. The image is pulled if the host lacks it, since a
scope:app deploy never touches the db stack.
Three further defects found while verifying, none of which would have surfaced
without dumping against the real database:
- The dump now runs as root. mysqldump --single-transaction issues FLUSH
TABLES, needing the global RELOAD privilege; the MySQL image grants the
application user only ALL ON `<db>`.*, and --skip-lock-tables does not avoid
it. Elevating the app's own runtime user would have been the worse trade.
- --set-gtid-purged=OFF. galactus is the replication SOURCE with GTID on, so a
default dump embeds SET @@GLOBAL.GTID_PURGED and is unrestorable onto the
server it came from. Verified: 0 GTID_PURGED lines in the output.
- Verification was too weak to be worth having. `test -s` plus `gzip -t` passes
on a 372-byte gzip containing no tables, which is exactly what a dump that
died on its first statement produces. It now asserts a CREATE TABLE count and
logs it. A failed attempt also deletes its own output, so a truncated file
never appears in the restore list.
Verified against live prod, both paths: success writes a 31-table dump the API
container can see; a wrong password fails with mysqldump's own error quoted and
leaves the volume empty.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pre-migrate backup failed with "mysqldump exited 2" and nothing else.
Reproduced on the host with stderr captured:
ERROR 1045: Plugin caching_sha2_password could not be loaded:
/usr/lib/mariadb/plugin/caching_sha2_password.so: No such file or directory
Alpine's `mysql-client` is MariaDB's client and ships an EMPTY plugin
directory, so it cannot perform caching_sha2_password — MySQL 8.4's default and
effectively only auth method. `mariadb-connector-c` provides the plugin.
This was never about the deploy backup alone. Every mysqldump/mysql call from
the API container was broken, which means the whole Operaciones panel — backup,
restore, sync, re-import — could not work in a container. It went unnoticed
because that feature had only ever been run with the API on a developer
machine, where the Oracle client is installed. Verified after the fix: dump
exits 0, gzip valid, 31 CREATE TABLEs.
Also fixed, both found while chasing the above:
- The backup script reported an exit code and nothing else, because a detached
exec captures no output — which is precisely why this needed a manual
reproduction. mysqldump's stderr is now redirected to a file and read back
through a short attached exec on failure, so the deploy log states the cause.
Verified against live prod: the log now carries the 1045 line itself.
- Listing ONLY 100.100.100.100 as the containers' resolver costs them public
DNS, since MagicDNS does not forward upstream unless the tailnet defines
global nameservers. Nothing at runtime needed it, but `apk` inside the
container stopped resolving, and anything outbound would have too. A public
fallback resolver is now listed after MagicDNS.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Prod came up with nobody able to log in, in two separate ways.
1. No sign-in account exists. `prisma migrate deploy` creates tables, never
rows, and nothing in the deploy path seeds one — deliberately, since making
an administrator should not be a side effect of shipping code. But
apps/api/scripts was not in the runtime image either, so the only way to
create the first account was to run the script from a developer machine
against a production DATABASE_URL. Ship scripts/ in the image so it can be
run on the host with docker exec. Still never run automatically.
2. Login could not establish a session at all. cookie.secure followed NODE_ENV,
the image sets NODE_ENV=production, and the app is served over plain HTTP —
express-session then silently emits NO Set-Cookie header. POST /auth/login
still answered 200 with the full user object, no session was created, every
later request 403'd, and the UI would have looped back to /login. It reads
as an auth bug and is really a transport mismatch.
The flag is now driven by SESSION_COOKIE_SECURE, still defaulting to
NODE_ENV. An EMPTY value counts as unset rather than false, because compose
turns an absent `${SESSION_COOKIE_SECURE:-}` into the empty string and the
naive check would have quietly dropped Secure on any deployment that merely
passed the variable through.
galactus sets it to "false". That is acceptable ONLY because the host is
reachable exclusively over Tailscale, so WireGuard already encrypts the
wire. It must go back to "true" when the app is served over TLS or exposed
off-tailnet; behind a TLS-terminating proxy, set trust proxy instead.
Verified against live prod: seeded an admin, POST /auth/login returns 200 with
full ADMIN abilities, a wrong password is rejected with 401, and no Set-Cookie
was present before this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first successful galactus deploy came up all-green while the web tier was
running a build from two commits earlier. The registry held web:latest from
3ff56e6; the host still had a web:latest cached from 4ee7ec7; the deploy
reported success and served the old one. The API was only current because it
had been pulled by hand during earlier debugging.
Two independent failures, both fixed here.
1. Images are not pulled. The deploy action's `pull: true` does not reliably
refresh an already-cached moving tag on a standalone endpoint. Added a
Pull images step (deploy/scripts/pull-images.mjs) that pulls each image
through Portainer's Docker API with registry credentials and fails the
deploy if a pull fails — note the endpoint answers 200 even when the pull
errored, so the stream body has to be inspected, not just the status.
2. The drift check could not see it. Both the verify step and the web footer
compared APP_VERSION, but on a branch build BOTH tiers report "master", so
equality proved nothing. They now compare gitSha, which is the only field
that differs between two builds of the same branch. api and web come from
one matrix run, so a difference can only mean an image was not replaced.
This needed a /version on the web tier too — previously its build identity
was only readable by scraping window.__APP_BUILD__ out of the HTML.
pull-images.mjs builds the X-Registry-Auth header as URL-safe base64 WITH
padding: Node's "base64url" omits the padding and Portainer's Go decoder
rejects it with "Illegal base64 data at input byte N".
Verified against galactus: pulls both images, and exits non-zero on a
nonexistent tag.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With the image fixed, the API got as far as connecting and then died with
Prisma P1001 "can't reach database server". The cause is DNS, not routing.
galactus runs systemd-resolved, whose 127.0.0.53 stub is unreachable from
inside a container, so Docker falls back to the upstream resolver in
/run/systemd/resolve/resolv.conf — the LAN router, which knows nothing about
the tailnet. Verified from a probe container on galactus: resolving
galactus.tail01aa2.ts.net fails outright, while `nc 100.103.77.46 3306` is
OPEN. Only the lookup was broken.
Pin the api and web services to Tailscale's own resolver (100.100.100.100,
the same anycast address on every tailnet) with this tailnet's search suffix.
Both are overridable via TAILSCALE_DNS / TAILNET_SUFFIX. db and minio need
nothing — they make no outbound calls.
Verified end to end: the published image, unmodified, with only these DNS
settings, boots on galactus against the real database and serves
/health {"status":"ok"}
/version {"service":"api","version":"master","gitSha":"3ff56e6b..."}
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two independent defects in docker/api.Dockerfile, both found by booting the
published image on galactus rather than by reading it. Neither had ever been
observed because no deploy had previously got far enough to start the API.
1. "Cannot find module '@jorgecuadros/database'".
node-linker=hoisted flattens EXTERNAL dependencies into /repo/node_modules,
but the workspace dependency stays linked per-package at
apps/api/node_modules/@jorgecuadros/database -> ../../../../packages/database.
The runtime stage copied only /repo/node_modules, so the link was dropped.
Copy the @jorgecuadros scope dir as well — not the whole directory, whose
only other contents are devDependencies.
2. "Prisma Client could not locate the Query Engine for runtime
linux-musl-openssl-3.0.x ... generated for linux-musl".
Prisma picks its engine by sniffing the build environment. The build stage
had no openssl so it generated for plain "linux-musl", while the runtime
stage demanded the openssl-3.0.x variant and refused to start. Fixed at both
ends: binaryTargets now names the musl target explicitly in schema.prisma,
so the shipped engine no longer depends on what happens to be installed at
build time, and openssl is installed in the deps stage (generate) and the
runtime stage (Prisma needs it regardless).
Verified by running the published image on galactus with each fix patched in
by hand, against the real database, until it got past both failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first deploy attempt (run 705) died on "Input required and not supplied:
token", which names the action's input rather than the secret that was unset —
the repo had only REGISTRY_USERNAME and REGISTRY_PASSWORD, so every deploy
secret was missing on both workflows. That is also why the endpoint_id /
pull_image input-name bug had gone unnoticed: neither workflow had ever got
far enough to use them.
Both workflows now check their required secrets up front and fail listing the
ones that are empty. The scope=full-only secrets are only required when the
dispatch is actually scope=full.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the gap between "what tag did I deploy" and "what is actually running",
and gives the schema a history that can be reasoned about across releases.
Migrations
- Baseline the existing schema as 0000_init (migrate diff --from-empty). The
schema had only ever been applied with `prisma db push`, so no history
existed and schema state was disconnected from app version. Existing
databases must be baselined once with `migrate resolve --applied 0000_init`;
the workflows print this remedy on P3005.
- Run `prisma migrate deploy` as a deploy STEP, not the container CMD — as a
CMD, N replicas would race each other applying the same migration.
Version reporting
- GET /version on the API reports the APP_VERSION / GIT_SHA / BUILD_DATE that
build.yml already baked into both images but nothing ever read.
- The web footer shows the web build and flags an api/web mismatch. The two
cannot drift at build time (one matrix run) but can at deploy time.
- Both deploy workflows now fail if the running API does not report the tag
that was dispatched — a stack naming a tag is not proof of what is running.
- scripts/set-version.mjs stamps every package.json, which had all sat at
0.1.0 while real releases shipped as v1.x.
Pre-migrate backup
- deploy/scripts/pre-migrate-backup.mjs dumps the database from INSIDE the
still-running old API container over Portainer's Docker API, so the file
lands in the volume the Operaciones restore screen reads. A dump taken on
the CI runner would be unreachable by the only restore path we have.
Verifies the artefact with `gzip -t` before letting the migration proceed.
galactus
- deploy/galactus/*.compose.yml: standalone-Docker ports of the Swarm stacks.
Plain compose silently ignores `deploy:`, so restart_policy becomes
`restart: unless-stopped` — without it nothing returns after a host reboot.
- .gitea/workflows/deploy-galactus.yml drives endpoint 3 with its own secrets.
Fixes
- deploy.yml passed `endpoint_id` and `pull_image` to
cssnr/portainer-stack-deploy-action, which has no such inputs (they are
`endpoint` and `pull`). The endpoint was silently never set.
docs/DEPLOY_AND_MIGRATIONS.md documents expand/contract as the rule for schema
changes: Prisma has no down-migrations, so a code rollback never rolls the
schema back, and restoring the replication master from a dump diverges every
replica.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The office keeps more than one operating account (Utilities banks in MXN,
Seguros in USD), but bank_transactions was a single implicit MXN register by
design. Adds Bank/BankAccount and makes every read and write in the module
scoped to exactly one account.
Schema:
- Bank / BankAccount. Currency is fixed per account and BankTransaction has
no currency column of its own — a movement inherits its account's, the way
a real bank account doesn't mix currencies.
- BankTransaction.bankAccountId, required. A movement with no known account
isn't reconcilable against a statement.
- @@index([bankAccountId, transactionDate]): every read now filters by
account and orders/groups by date.
Migration:
- backfill_bank_accounts.py seeds Scotiabank + "Utilities — Scotiabank (MXN)"
and backfills all 22,669 existing rows onto it, then promotes the column to
NOT NULL and attaches the FK. Standalone because prisma db push cannot add
a required column to a populated table. Idempotent; re-running once a second
account exists does not re-point rows.
- run_all.py runs it (both modes) before transform_bank.py, which now resolves
the account by label and fails fast if it is missing.
API:
- ?bankAccountId= required on list/stats/facets/summary — not optional with an
"all accounts" default, since summing an MXN and a USD register repeats the
currency-collapsing mistake the billing module exists to prevent. Missing is
400, unknown is 404.
- facets() had no account clause at all and summary() has two raw-SQL rollups;
all three are now parameterised. Scoping only one of summary's queries would
leave the year list and its drill-down describing different books.
- New bank/accounts + bank/banks sub-resource under a MANAGER
bank:manage-accounts ability. currency is absent from the update DTO: booked
movements are denominated in it, so editing would re-denominate history.
Capture into a closed account is rejected.
Web:
- /banco gains an account picker (remembered per browser) and reads every
figure in the selected account's currency; the "single currency (MXN)"
doc-comment and the hardcoded MXN formatting are gone.
- New /banco/cuentas for banks and accounts. Accounts are closed, never
deleted — the FK is required, so deleting one would destroy its register.
- /inicio's chequera card names the account it is reading instead of implying
a single register.
Verified against dev + browser: a second USD account showed full read/write
isolation from the MXN register, whose totals were unchanged (22,669
movements, net 1,014,266.97).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two follow-ups to the text-size control.
Spacing now scales with the text. All padding, margin, gap and min-height
declarations in globals.css move from px to rem (263 declarations, converted
mechanically), so --ui-scale drives the whole layout rather than just the
glyphs. Deliberately left in px: border widths, which must stay hairlines;
box-shadow offsets; border-radius, which reads as bloated when scaled on large
cards; --shell-max, a container cap that must not outgrow the viewport; and
media-query breakpoints, which are conditions rather than declarations. With
spacing following along, the presets gain a 1.5 "Máximo" step and MAX_UI_SCALE
rises from 1.4.
The preference now lives on the account instead of only in one browser.
User.uiScale (Float, default 1) is added to the schema and to the safe select,
so it rides along on /auth/login and /auth/me. PATCH /auth/preferences writes
it, guarded by AuthenticatedGuard only — every role including VIEWER may set
their own, and the target is always the session's user id, never a body
parameter, so this cannot be used to touch another account. The global
ValidationPipe's whitelist rejects any extra field, so role cannot ride in
alongside uiScale.
localStorage stays, demoted to a pre-paint cache for the layout.tsx script;
AppShell reconciles it against the account once /auth/me answers, with the
account winning. FontScaleControl becomes a controlled component since the
same value is now edited from the appbar and the drawer.
Verified against the dev API: PATCH persists and is reflected by a subsequent
/auth/me, out-of-range values are rejected 400, and an extra "role" field in
the body is rejected 400.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The appbar had grown to 11 flat links with no responsive behaviour, and
overflowed below ~1100px.
Nav is now 7 top-level entries: Inicio, Clientes, Pólizas, Propiedades,
Reportes stay one click away, while the movement screens (Captura, Estado de
cuenta, Chequera) and the admin screens (Catálogos, Usuarios, Operaciones)
collapse into "Cobranza" and "Admin" dropdowns. Groups are ability-filtered
and disappear entirely when the user can see none of their items, so VIEWER
never renders an empty Admin menu. activeHref now scans the flattened link
list, and a group trigger highlights while one of its children is current.
Below 980px the nav collapses to a burger drawer that lists every group
expanded, closing on navigation and on Escape.
Text size is user-adjustable app-wide. Every font-size in globals.css is
converted from px to rem (mechanically, 133 declarations) and the root size
becomes calc(100% * var(--ui-scale)), so one variable on <html> rescales the
whole UI. The preference persists in localStorage and is applied by a
pre-hydration script in layout.tsx to avoid a flash at the default size; the
Aa control lives in the appbar and, as a segmented row, in the drawer.
Spacing stays in px by design, which is why 1.3 is the largest preset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Implements docs/RECEIPT_CAPTURE_SPEC.md §1, the legacy "Editor"
replacement, on top of the single-movement capture from plan step 6.
No new abilities: batching and resolving are both capturing.
- outstanding (legacy NOPAGO): capture flag, ?outstanding= filter, and
POST /billing/:id/resolve-outstanding (gated ledger:create, not
ledger:void — resolving completes a capture rather than reversing
one). Outstanding rows are excluded from every balance aggregate,
matching the legacy SALDOS ULTIMO 0 query's HAVING NOPAGO = 0, but
still count in the movement browser's filtered totals.
- POST /billing/batch: many customers' receipts against one check, in
one $transaction. Deliberately not a persisted batch entity —
checkNumber is already a column and grouping by it answers every
legacy by-check query.
- GET /billing/by-check + a cheque-count report, replacing REPORTE
CHEQUE COUNT / REPORTE POR CHEQUE / EDITA CHEQUE ALF|COUNT|NUM. Print,
PDF, CSV and XLSX come free from the existing /reportes/:slug machinery.
- Web: /estado-cuenta/lote (the Editor screen, with live reconciliation
against the physical check amount), an "Estado de pago" filter, a
"sin fondos" row tag and a Resolver dialog, plus a top-level "Captura"
nav entry.
Integration seam for the OCR auto-capture module (spec §2), which is
required to post through createBatch rather than writing Transaction
rows itself: items[i] maps to lines[i] so postedTransactionId can be
zipped back on; opts.refs[i] stamps captureRef with a duplicate-post
guard that a voided row deliberately does not block; opts.source is
service-level only, so an HTTP client cannot label hand-keyed rows as
machine-captured. captureSource/captureRef are nullable so the 40,136
migrated rows stay NULL rather than being mislabelled.
Fixes two pre-existing bugs found while building this:
- statement() filtered legacySourceTable with `notIn`, which compiles to
SQL NOT IN — and `NULL NOT IN (...)` is NULL, so every app-captured
movement was invisible on the customer statement (438 rows in the
movement browser vs 392 on the statement) while showing everywhere
else. This would have made the whole capture feature look broken.
- The balances count query omitted the void filter its own page query
applied, so the total disagreed with the rows.
Nav highlighting now resolves by longest match; the previous
first-startsWith logic lit up both the parent and any nested entry.
Verified end-to-end against the dev DB, API and browser; all test rows
removed afterwards. Also corrects RESUME.md, which documented the dev
ports as :3001/:3000 — they are :4501/:4500, from the env files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Companion to docs/RECEIPT_CAPTURE_SPEC.md — the insurance half of the
2026-07-25/26 meeting. Documentation only; no application code.
Verified against the code and a live query of the dev DB rather than
designed from the meeting notes alone, which changed several conclusions:
- Renewal emails and the liquidación batch are much smaller than they
look. RenewalNotice + its @@unique([policyId, generation]) idempotency
key and the aviso-renovacion letter body already exist; the per-policy
liquidation fields are wired end to end. What's missing is a scheduler,
a mail client, and the batch layer.
- Carrier research: ANA and GMX are one company (Grupo Valore). ANA
exposes a live SOAP service with a published operation list; GMX
publishes no machine interface at all. Every ANA operation serves
new-business quoting/issuance, not "list my book" — so the direction
question decides whether the feature is buildable.
- UTILSEG is unusable for Utilities↔Seguros reconciliation and the spec
closes that long-standing open question: DATGRAL.[NUM UTIL] is
authoritative (name match 298/563 vs 58/1024), and where the two
sources overlap they contradict on 170 of 218 shared ids.
Also records two live defects found while verifying: policy_types is
missing its INCENDIO and M_EMPR rows (the FK is ON DELETE SET NULL, so 5
m_empr policies silently lost their ramo), and the legacy settlement
slots don't match the target model (MULT/INCENDIO carry two, M EMPR
carries four, Policy collapses to one).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Points PLAN.md at docs/RECEIPT_CAPTURE_SPEC.md and surfaces its open
design questions (OCR provider, Seguros bank details, clave catastral
vs. predial, recycling triggers) separately from the existing ops-only
open items list.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Forward implementation spec covering the legacy "Editor" receipt-capture
workflow plus three net-new requests from the 2026-07-25/26 meeting with
Jorge: PDF/OCR auto-capture, multi-bank chequera support, and
customer-number recycling. Matching logic and data-model gaps for each
were verified against the actual migration scripts and API code, not
just the schema comments.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`r.netPremium` / `r.total` are strings, so an empty-string value leaked
""` into the JSX instead of rendering nothing. Wrap in Boolean() so the
guard is a real conditional.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces ~40 legacy Access renewal-notice report clones (one per carrier
per coverage tier, e.g. AMPL/RC/LIC RENEW X MES/VENCE ATLAS 13/2013) with
one parameterized aviso-renovacion report driven by real Policy/Vehicle/
coveragesJson data instead of hand-typed label text per clone.
- schema.prisma: add RenewalNotice, replacing the legacy CONTROL <ramo>
RENEW[2/3] X MES paper log of which notice generation was sent
- reports: new "letter" ReportFormat + aviso-renovacion registry entry +
LetterLayout renderer in ReportRunner.tsx
- docs/RENEWAL_NOTICES.md + migration/legacy_report_defs/: extracted (via
Application.SaveAsText, since the VBA project wouldn't load) and
documented the legacy report/query chain this replaces
Coveragesjson key names and a mark-as-sent mutation are still unverified/
unbuilt — see caveats in docs/RENEWAL_NOTICES.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The --sync path had never been run and was broken in several ways. Fixed and
verified against the dev DB (two consecutive syncs, both exit 0, 32/32
assertions: stable PKs, manual-row preservation, changed-row updates,
legacy-delete, no child duplication, zero FK orphans; idempotent).
- policies/properties: reuse each legacy row's existing id (by provenance)
BEFORE building child rows, so children no longer point at a discarded fresh
uuid; rebuild legacy-owned children via scoped delete + reinsert.
- customers: replace zip(customers, refs) (mispaired almost every row) with a
ref-grouped id remap; names now restore and no spurious customers appear.
- drop the invalid Vehicle @@unique(legacySourceTable, legacyId) — one legacy
policy row carries up to 3 vehicles sharing a legacyId; handle via delete+reinsert.
- upsert lookup tables (policy_types, insurance_providers, type_transactions,
adjusters) by natural name and remap child FKs instead of inserting fresh
uuids that nothing points at.
- transactions: drop updatedAt=NOW() (no such column); guard report formatting
on NULL legacySourceTable (manual rows). Same report guard in bank.
- add manual-safe prune (prune_empty_customers.py --sync, in SYNC_STEPS): prune
only legacy-owned empties, never manually-added customers.
web: customer-detail mini tx list now strikes voided rows with an "(anulado)"
tag (was the last void-UI rendering gap; /estado-cuenta already handled it).
docs: RESUME.md updated — Phase B sync marked verified end-to-end, void-UI
browser pass recorded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- New reports backend (registry, service, controller, outputs, types)
with catalog endpoint + slug/CSV/XLSX/PDF/print outputs.
- /reportes catalog + /reportes/[slug] runner; ReportRunner + ContextReports
components wire pre-filtered links from domain pages.
- Fix: /reportes/[slug] now reads searchParams and forwards initialParams to
ReportRunner so /reportes/edo-cuenta-datos?customerId=... auto-runs
instead of dropping the id and forcing a manual customer search.
- /inicio landing page; root + login redirect to /inicio.
- Company header env vars + logo asset for PDF/print rendering.
- exceljs + pdfkit deps.
Replace the JC text mark in AppShell and login with the real
company_logo.png. Restyle .brand-mark to host the image (white
rounded bg, object-fit contain). Appbar 38px, login panel 46px.
Add the missing api/web deployment path on top of the existing image build CI.
- deploy/jorgecuadros-app.stack.yml: PROD app stack (api + web) pulling the
git.mancinas.io registry images. Does not ship mysql/minio (separate stacks);
API reaches them via DATABASE_URL / S3_ENDPOINT. API pinned to the
jorgecuadros_db node for stable ingest/backup volumes; web is stateless.
- deploy/jorgecuadros-app.env.example: documented stack env template.
- .gitea/workflows/deploy.yml: manual (workflow_dispatch) deploy to Portainer
via cssnr/portainer-stack-deploy-action. Inputs: image tag + scope
(app = web+api, full = db+minio+app, applied db->minio->app).
Make the web API origin runtime-configurable instead of build-baked: the root
layout injects window.__API_ORIGIN__ from the API_ORIGIN env (force-dynamic) and
lib/api.ts resolves it at runtime, so one built image serves any deployment.
Also: dev.sh to run both dev servers (frees stale ports first) and move local
dev to ports web 4500 / api 4501.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The schema has carried `storageKey` pointers and the migration has written
blobs to MinIO since day one, but the API had no S3 client — documents could
only be deleted, never uploaded or retrieved. This adds the missing wiring.
API
- StorageModule/StorageService (@aws-sdk/client-s3, path-style for MinIO):
put/getStream/delete, best-effort bucket ensure on boot, gracefully disabled
when S3 env is absent (ServiceUnavailable on use).
- Reads S3_ENDPOINT/S3_BUCKET + S3_ACCESS_KEY/S3_SECRET_KEY, falling back to
MINIO_ROOT_USER/MINIO_ROOT_PASSWORD so one credential set drives both the
migration and the API.
- Property service documents: POST :id/documents (multipart), GET
:id/documents/:childId/download (streamed), delete now also drops the blob.
- Policy documents: same upload/download/delete (previously had none).
- Keys stay under the service/<id>/… and policy/<id>/… prefixes the migration
established.
Web
- api.ts: shared uploadFile() helper (uploadIngest refactored onto it),
upload/download/remove helpers for property & policy documents.
- Servicios, polizas, clientes detail pages: real Descargar links and an
upload control (gated by policy:update / property:update) replacing the
"storage pending" notes.
Infra
- docker-compose: minio service (9000/9001, healthcheck, named volume) + S3
env wired into the api service.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Working-tree checkpoint of in-progress work carried across prior
sessions on the feat/crud-rbac branch, committed so it lands on the
remote alongside the CI changes.
- Operaciones admin panel: apps/api/src/ops (ingest upload, backup /
restore / re-import jobs) wired into app.module + RBAC abilities, and
the apps/web/src/app/operaciones page. docker-compose gets INGEST_DIR
/ BACKUP_DIR volumes; .gitignore excludes migration/ingest + backups.
- migration/sync.py plus transform_*.py / run_all / config / dbenv /
blob_extract adjustments for the additive sync path.
- crud/rbac phase-5 web bits: AppShell, api/labels/types libs, globals.
- schema.prisma + PLAN/RESUME doc updates.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add comprehensive Docker image versioning and a Gitea Actions workflow
that builds and pushes both the API and web images to the
git.mancinas.io registry.
Versioning: both Dockerfiles take APP_VERSION / GIT_SHA / BUILD_DATE
build-args, surfaced as runtime ENV + OCI labels, so a running container
self-reports the exact commit it was built from. metadata-action emits a
tag set per build: semver (from vX.Y.Z git tags), branch ref,
sha-<short>, and latest (default branch only).
Also fix the Dockerfiles for the pnpm workspace: the old npm install
could not resolve the "@jorgecuadros/database": "workspace:*" protocol
dep and would abort the API build. Now pin pnpm 9.15.9 via corepack,
install --frozen-lockfile with node-linker=hoisted (flat tree so the
runtime stage copies a single node_modules), and build via --filter. The
API build stage gets python3/make/g++ for argon2's musl source compile.
Add .dockerignore to keep the build context lean and deterministic.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Covers stack, repo layout, local dev (pnpm install, env, MySQL via
docker, prisma db push, seed admin, run api+web, login), full-stack
Docker path, common commands, auth/roles, legacy migration, and prod
notes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gitea-actions
rmancinasandClaude Opus 5