b59abda895a822edb4241e5e798649b99187d95f
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  rmancinasandClaude Opus 5
|
b59abda895 |
feat(captura): fold recibo OCR into Captura as an automatic mode
Scanning a stack of bills and keying them in are the same daily job, ending in the same ledger path, so OCR intake becomes a mode of the capture screen instead of a second menu entry: - components/Captura.tsx holds the mode switch; the manual check form moves verbatim to components/ManualCheckCapture.tsx and the OCR intake to components/StatementIntake.tsx. - /estado-cuenta/lote opens on manual, /recibos on automatic — both render Captura, so batch-review links and old bookmarks still land right. - Nav drops "Recibos (OCR)"; "Captura" covers both, with a NavLink.aliases field so /recibos still highlights it. Also fixes the "El almacenamiento de documentos no está configurado" failure staff hit on upload. Uploading with no object storage configured used to succeed, then die on the first put minutes later, leaving a FAILED batch whose only explanation was that string. createBatch now refuses up front, GET /statements/status reports storageAvailable alongside ocrAvailable, and the intake tab explains the situation instead of offering an upload that cannot work. S3_* documented in .env.example (deploy stacks already set it). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
4d5008b545 |
feat(statements): OCR intake for scanned utility bills
Staff key 300+ utility statements per company per month by hand. This adds the ingest -> split -> OCR -> match -> review pipeline that proposes customer and amount per page instead (RECEIPT_CAPTURE_SPEC §2), posting through the existing BillingService.createBatch seam with source=OCR and a per-document captureRef so machine and hand capture share one write path and audit trail. Everything was designed against 10 real scanned statements (46 pages of CFE, CESPT and Telnor bills) rather than from the sample-free spec. The scans have no text layer at all — they are camera images — so OCR is mandatory, and they arrive bundled one customer per page. Measured on those pages the parser identifies the provider 46/46 and reads an account reference 43/46; against the dev database that is 39/46 (85%) exact auto-match, 40/46 identified, with the rest genuine review cases. That closes the OCR-provider question in favour of self-hosted Tesseract: it clears the bar for a queue where a human confirms every row, and OcrProvider keeps a managed API a one-line swap. The samples corrected three things the spec had wrong or unknown: - Clave catastral is NOT predial. DATMEX.clave (934 rows) is what CESPT and predial bills print; DATMEX.predial, which PROPERTY_TAX.accountNumber holds, has 663 distinct values across 1135 rows and appears on no statement. The clave now lives on Property.cadastralKey as the matcher's secondary key; predial is left untouched. This had been blocking predial matching. - Gas was recoverable: 160 of 334 DATMEX.gas values are real account numbers (the rest are ESTACIONARIO/CILINDRO descriptors), now in GAS.meterNumber. - Phone is one billed line per property (534/18/1 across phone1/2/3), so the new TELEPHONE ServiceKind backfills from phone1 only, not three rows. Matching is scoped to one column per service kind and never reads the customer name — a CESPT receipt prints ARNAIZ ROSAS ELSA AURORA for an account this office holds under CATT, RANDY, because the printed name is the registrant, not the current owner. Where a provider prints a payment barcode it beats the printed label (one CFE label OCR'd a digit too many while its barcode was correct) and the two cross-check, with disagreement forcing review. Confirming a document whose service had no reference writes it back, so gas and any other cold start is a one-time cost rather than a permanent queue. Verified end to end against the live dev API and MinIO: real scans uploaded over HTTP, matched, confirmed against a check, and the resulting rows checked in MySQL (negative amounts, captureSource=OCR, concept derived from the batch kind, captureRef linking back to each page). Re-confirming a posted batch is refused. Test data was removed afterwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
30dfc7dc3e |
fix(ops): run backups as an admin login, and stop recording failed dumps as good
The Operaciones panel (backup, restore, sync, re-import) shelled out to mysqldump as the application user, parsed straight out of DATABASE_URL. `--single-transaction` issues FLUSH TABLES, which needs the global RELOAD privilege, and the app user is granted only ALL ON jorgecuadros.* plus USAGE ON *.*. BACKUP failed outright; SYNC and REIMPORT failed with it, since both take a safety backup first. An admin credential is now supplied out of band via OPS_DB_ADMIN_USER / OPS_DB_ADMIN_PASSWORD, mirroring what deploy/scripts/pre-migrate-backup.mjs already does, rather than permanently elevating the user the API serves requests as. Host, port and database still come from DATABASE_URL, so the override can only change who logs in, never which server. Unset, it falls back to the DATABASE_URL credentials and warns — local development is unaffected. Two defects in the dumps themselves, both shared with the deploy backup before it was rewritten: - No --set-gtid-purged=OFF. The production server is the replication source with GTID on, so every dump embedded SET @@GLOBAL.GTID_PURGED and was unrestorable onto the server it came from — the one thing the restore screen is for. - The pipeline's exit status was gzip's, and gzip succeeded. A mysqldump that died on its first statement left a small, perfectly valid archive that the job recorded as SUCCESS and the restore screen listed as an ordinary restore point. Dumps now run under `set -o pipefail`, assert a CREATE TABLE count, and delete their own output on failure. Verified with a stubbed mysqldump: a failing dump exits 1, surfaces the real error, removes the partial file, and — critically — stops SYNC/REIMPORT before the ETL touches anything. Restores gained pipefail too: a corrupt archive made gunzip fail while mysql, fed a truncated stream, could still exit 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
19f03198d6 |
fix(docker): install the MySQL 8.4 auth plugin; report why a dump fails
The pre-migrate backup failed with "mysqldump exited 2" and nothing else.
Reproduced on the host with stderr captured:
ERROR 1045: Plugin caching_sha2_password could not be loaded:
/usr/lib/mariadb/plugin/caching_sha2_password.so: No such file or directory
Alpine's `mysql-client` is MariaDB's client and ships an EMPTY plugin
directory, so it cannot perform caching_sha2_password — MySQL 8.4's default and
effectively only auth method. `mariadb-connector-c` provides the plugin.
This was never about the deploy backup alone. Every mysqldump/mysql call from
the API container was broken, which means the whole Operaciones panel — backup,
restore, sync, re-import — could not work in a container. It went unnoticed
because that feature had only ever been run with the API on a developer
machine, where the Oracle client is installed. Verified after the fix: dump
exits 0, gzip valid, 31 CREATE TABLEs.
Also fixed, both found while chasing the above:
- The backup script reported an exit code and nothing else, because a detached
exec captures no output — which is precisely why this needed a manual
reproduction. mysqldump's stderr is now redirected to a file and read back
through a short attached exec on failure, so the deploy log states the cause.
Verified against live prod: the log now carries the 1045 line itself.
- Listing ONLY 100.100.100.100 as the containers' resolver costs them public
DNS, since MagicDNS does not forward upstream unless the tailnet defines
global nameservers. Nothing at runtime needed it, but `apk` inside the
container stopped resolving, and anything outbound would have too. A public
fallback resolver is now listed after MagicDNS.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
b2cdcbe2cd |
fix(api): session cookie never issued over HTTP; ship the seed script
Prod came up with nobody able to log in, in two separate ways.
1. No sign-in account exists. `prisma migrate deploy` creates tables, never
rows, and nothing in the deploy path seeds one — deliberately, since making
an administrator should not be a side effect of shipping code. But
apps/api/scripts was not in the runtime image either, so the only way to
create the first account was to run the script from a developer machine
against a production DATABASE_URL. Ship scripts/ in the image so it can be
run on the host with docker exec. Still never run automatically.
2. Login could not establish a session at all. cookie.secure followed NODE_ENV,
the image sets NODE_ENV=production, and the app is served over plain HTTP —
express-session then silently emits NO Set-Cookie header. POST /auth/login
still answered 200 with the full user object, no session was created, every
later request 403'd, and the UI would have looped back to /login. It reads
as an auth bug and is really a transport mismatch.
The flag is now driven by SESSION_COOKIE_SECURE, still defaulting to
NODE_ENV. An EMPTY value counts as unset rather than false, because compose
turns an absent `${SESSION_COOKIE_SECURE:-}` into the empty string and the
naive check would have quietly dropped Secure on any deployment that merely
passed the variable through.
galactus sets it to "false". That is acceptable ONLY because the host is
reachable exclusively over Tailscale, so WireGuard already encrypts the
wire. It must go back to "true" when the app is served over TLS or exposed
off-tailnet; behind a TLS-terminating proxy, set trust proxy instead.
Verified against live prod: seeded an admin, POST /auth/login returns 200 with
full ADMIN abilities, a wrong password is rejected with 401, and no Set-Cookie
was present before this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
1cba9bfc32 |
fix(galactus): give containers Tailscale's resolver so MagicDNS names resolve
With the image fixed, the API got as far as connecting and then died with
Prisma P1001 "can't reach database server". The cause is DNS, not routing.
galactus runs systemd-resolved, whose 127.0.0.53 stub is unreachable from
inside a container, so Docker falls back to the upstream resolver in
/run/systemd/resolve/resolv.conf — the LAN router, which knows nothing about
the tailnet. Verified from a probe container on galactus: resolving
galactus.tail01aa2.ts.net fails outright, while `nc 100.103.77.46 3306` is
OPEN. Only the lookup was broken.
Pin the api and web services to Tailscale's own resolver (100.100.100.100,
the same anycast address on every tailnet) with this tailnet's search suffix.
Both are overridable via TAILSCALE_DNS / TAILNET_SUFFIX. db and minio need
nothing — they make no outbound calls.
Verified end to end: the published image, unmodified, with only these DNS
settings, boots on galactus against the real database and serves
/health {"status":"ok"}
/version {"service":"api","version":"master","gitSha":"3ff56e6b..."}
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinasandClaude Opus 5
|
4ee7ec71f0 |
feat(deploy): prisma migration history, /version, galactus standalone deploy
Closes the gap between "what tag did I deploy" and "what is actually running", and gives the schema a history that can be reasoned about across releases. Migrations - Baseline the existing schema as 0000_init (migrate diff --from-empty). The schema had only ever been applied with `prisma db push`, so no history existed and schema state was disconnected from app version. Existing databases must be baselined once with `migrate resolve --applied 0000_init`; the workflows print this remedy on P3005. - Run `prisma migrate deploy` as a deploy STEP, not the container CMD — as a CMD, N replicas would race each other applying the same migration. Version reporting - GET /version on the API reports the APP_VERSION / GIT_SHA / BUILD_DATE that build.yml already baked into both images but nothing ever read. - The web footer shows the web build and flags an api/web mismatch. The two cannot drift at build time (one matrix run) but can at deploy time. - Both deploy workflows now fail if the running API does not report the tag that was dispatched — a stack naming a tag is not proof of what is running. - scripts/set-version.mjs stamps every package.json, which had all sat at 0.1.0 while real releases shipped as v1.x. Pre-migrate backup - deploy/scripts/pre-migrate-backup.mjs dumps the database from INSIDE the still-running old API container over Portainer's Docker API, so the file lands in the volume the Operaciones restore screen reads. A dump taken on the CI runner would be unreachable by the only restore path we have. Verifies the artefact with `gzip -t` before letting the migration proceed. galactus - deploy/galactus/*.compose.yml: standalone-Docker ports of the Swarm stacks. Plain compose silently ignores `deploy:`, so restart_policy becomes `restart: unless-stopped` — without it nothing returns after a host reboot. - .gitea/workflows/deploy-galactus.yml drives endpoint 3 with its own secrets. Fixes - deploy.yml passed `endpoint_id` and `pull_image` to cssnr/portainer-stack-deploy-action, which has no such inputs (they are `endpoint` and `pull`). The endpoint was silently never set. docs/DEPLOY_AND_MIGRATIONS.md documents expand/contract as the rule for schema changes: Prisma has no down-migrations, so a code rollback never rolls the schema back, and restoring the replication master from a dump diverges every replica. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
9ba5d2d09a |
feat(bank): multi-bank chequera — required bankAccountId, per-account scoping
The office keeps more than one operating account (Utilities banks in MXN, Seguros in USD), but bank_transactions was a single implicit MXN register by design. Adds Bank/BankAccount and makes every read and write in the module scoped to exactly one account. Schema: - Bank / BankAccount. Currency is fixed per account and BankTransaction has no currency column of its own — a movement inherits its account's, the way a real bank account doesn't mix currencies. - BankTransaction.bankAccountId, required. A movement with no known account isn't reconcilable against a statement. - @@index([bankAccountId, transactionDate]): every read now filters by account and orders/groups by date. Migration: - backfill_bank_accounts.py seeds Scotiabank + "Utilities — Scotiabank (MXN)" and backfills all 22,669 existing rows onto it, then promotes the column to NOT NULL and attaches the FK. Standalone because prisma db push cannot add a required column to a populated table. Idempotent; re-running once a second account exists does not re-point rows. - run_all.py runs it (both modes) before transform_bank.py, which now resolves the account by label and fails fast if it is missing. API: - ?bankAccountId= required on list/stats/facets/summary — not optional with an "all accounts" default, since summing an MXN and a USD register repeats the currency-collapsing mistake the billing module exists to prevent. Missing is 400, unknown is 404. - facets() had no account clause at all and summary() has two raw-SQL rollups; all three are now parameterised. Scoping only one of summary's queries would leave the year list and its drill-down describing different books. - New bank/accounts + bank/banks sub-resource under a MANAGER bank:manage-accounts ability. currency is absent from the update DTO: booked movements are denominated in it, so editing would re-denominate history. Capture into a closed account is rejected. Web: - /banco gains an account picker (remembered per browser) and reads every figure in the selected account's currency; the "single currency (MXN)" doc-comment and the hardcoded MXN formatting are gone. - New /banco/cuentas for banks and accounts. Accounts are closed, never deleted — the FK is required, so deleting one would destroy its register. - /inicio's chequera card names the account it is reading instead of implying a single register. Verified against dev + browser: a second USD account showed full read/write isolation from the MXN register, whose totals were unchanged (22,669 movements, net 1,014,266.97). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
26a4faa33e |
docs(insurance): spec renewal emails, liquidación batch, certificate, carrier APIs
Companion to docs/RECEIPT_CAPTURE_SPEC.md — the insurance half of the 2026-07-25/26 meeting. Documentation only; no application code. Verified against the code and a live query of the dev DB rather than designed from the meeting notes alone, which changed several conclusions: - Renewal emails and the liquidación batch are much smaller than they look. RenewalNotice + its @@unique([policyId, generation]) idempotency key and the aviso-renovacion letter body already exist; the per-policy liquidation fields are wired end to end. What's missing is a scheduler, a mail client, and the batch layer. - Carrier research: ANA and GMX are one company (Grupo Valore). ANA exposes a live SOAP service with a published operation list; GMX publishes no machine interface at all. Every ANA operation serves new-business quoting/issuance, not "list my book" — so the direction question decides whether the feature is buildable. - UTILSEG is unusable for Utilities↔Seguros reconciliation and the spec closes that long-standing open question: DATGRAL.[NUM UTIL] is authoritative (name match 298/563 vs 58/1024), and where the two sources overlap they contradict on 170 of 218 shared ids. Also records two live defects found while verifying: policy_types is missing its INCENDIO and M_EMPR rows (the FK is ON DELETE SET NULL, so 5 m_empr policies silently lost their ramo), and the legacy settlement slots don't match the target model (MULT/INCENDIO carry two, M EMPR carries four, Policy collapses to one). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Sonnet 5
|
9b9ee201c9 |
docs: add receipt capture, OCR, multi-bank & customer-recycling spec
Forward implementation spec covering the legacy "Editor" receipt-capture workflow plus three net-new requests from the 2026-07-25/26 meeting with Jorge: PDF/OCR auto-capture, multi-bank chequera support, and customer-number recycling. Matching logic and data-model gaps for each were verified against the actual migration scripts and API code, not just the schema comments. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Sonnet 5
|
f7ae0d5342 |
feat(reports): parameterized renewal-notice report + legacy report reference
Replaces ~40 legacy Access renewal-notice report clones (one per carrier per coverage tier, e.g. AMPL/RC/LIC RENEW X MES/VENCE ATLAS 13/2013) with one parameterized aviso-renovacion report driven by real Policy/Vehicle/ coveragesJson data instead of hand-typed label text per clone. - schema.prisma: add RenewalNotice, replacing the legacy CONTROL <ramo> RENEW[2/3] X MES paper log of which notice generation was sent - reports: new "letter" ReportFormat + aviso-renovacion registry entry + LetterLayout renderer in ReportRunner.tsx - docs/RENEWAL_NOTICES.md + migration/legacy_report_defs/: extracted (via Application.SaveAsText, since the VBA project wouldn't load) and documented the legacy report/query chain this replaces Coveragesjson key names and a mark-as-sent mutation are still unverified/ unbuilt — see caveats in docs/RENEWAL_NOTICES.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
rmancinas
|
0268ed896c |
Add Queries/Forms/Reports reference (DAO catalog)
ODBC only exposes Tables and non-hidden SELECT queries, so this used DAO COM automation (migration/catalog_objects.py, requires pywin32) to catalog Reports, Forms, and full Query SQL text across all four files instead. Key finding: SEGUROS 16.mdb, previously noted as having zero data tables, turns out to hold all 212 Reports/149 Forms/857 Queries for the insurance line - SEGUROS 16_be.mdb is confirmed pure data storage with zero saved objects. The renewal-notice reports also reveal a RENEW/RENEW2/RENEW3 multi-notice reminder cycle not visible in the table schema. docs/LEGACY_DATABASES_OBJECTS.md documents all of this with the full 751 real queries' SQL text (business logic: billing math, year- rollover batches, duplicate/delinquency detection). Raw output at migration/objects.json. Cross-linked from RESUME.md and the existing table-only LEGACY_DATABASES.md. |
||
|
rmancinas
|
7e65fc81d5 |
Add legacy database structural reference for non-Windows machines
docs/LEGACY_DATABASES.md documents all three source Access databases (every table, column, type, and known data-quality quirk) generated from a live read of the real files, so no Windows/Access driver is needed to understand their structure going forward. New migration/ tooling: catalog_schema.py connects to the real files and walks every table (including excluded scratch tables); render_catalog_md.py renders that into the doc's appendix. Raw output checked in at migration/catalog.json so the doc can be regenerated without touching Access again. |