Files
jorgecuadros-platform/docs/BACKLOG.md
T
rmancinasandClaude Opus 5 17d83291c3
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m51s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m42s
feat(migration): refuse a full re-import that would delete native rows
A full run_all.py pass truncates and rebuilds every table it owns from the
Access extract. That was harmless while the platform was a read-only mirror --
every row came from the extract, so wiping and rebuilding lost nothing. It
stopped being harmless once the platform started minting rows Access has never
heard of: allocated portal NUMids, customers created in the staff UI,
OCR-captured policies, app-booked ledger rows, uploaded documents.

REIMPORT is a button in /operaciones, so that was one click away.

native_guard.py counts what only exists here and exits 3; run_all.py runs it
before the first truncate and stops. Detecting an allocated NUMid needs the
staged Parquet -- the customer holds an ordinary-looking (utilities, DATGRAL,
'1172') ref, so "customer has no refs" cannot see it and only comparing against
the extract can. Missing staging is therefore treated as blocking rather than
as "nothing to protect".

The guard does not teach full mode to preserve anything: --sync already upserts
legacy rows against the existing refs and leaves the rest alone, and rebuilding
that inside full mode would re-implement it. --force-full (checkbox in the
REIMPORT confirm, recorded in the audit log) deletes them deliberately.

Verified against dev: clean before, exit 3 listing utilities/1172 with a
synthetic ref present, clean again after removing it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 21:01:01 -07:00

14 KiB
Raw Blame History

Backlog — what is pending, missing, and not yet built

One place for work that is known-outstanding. Compiled 2026-08-02 from PLAN.md, RESUME.md, the four specs and the two OCR docs, then checked against the code and the dev database rather than trusted — several items in those documents had already been closed, and two defects they describe are still live.

This file is an index, not a replacement. Each item points at the document that carries the reasoning. Close an item there as well as here, or the two drift.

Verified against dev at compile time (re-run before trusting the numbers):

policy_types: AUTO, LICENCIAS, MULT
policies NULL policyTypeId: 5
policies pending liquidación: 226
customers: 1536
last tag: v1.0.6 (2026-08-02 02:06 UTC) — 14 commits, 5 migrations behind HEAD

0. Ship-blocked — read before the next deploy

Everything from the notificaciones arc is unreleased. v1.0.6 predates it. Five migrations are waiting:

20260801120000_ocr_batch_discarded
20260801130000_renewal_email_notifications
20260801200000_mass_email_notifications
20260802120000_renewal_notices_unified_log
20260802140000_app_settings

Plus three backup-pipeline fixes that have never reached prod (860d483, 567b033, 898cf48 — the last prod run went green through the whole chain and died on the final step wanting deploy/.env.prod).

SES_* secrets created in Gitea 2026-08-02 — unblocked, unverified

The variables were wired through both deploy workflows and the app stack but had never been set. They now exist. What that clears: the production image runs NODE_ENV=production, which disables the stdout dev fallback, so a blank config made every send fail and log FAILED — and the pólizas sweep defaults to enabled, 06:00 America/Tijuana, so the failure would have repeated nightly.

Not yet confirmed, and the first deploy is what confirms it:

  1. Names match. The preflight checks SES_REGION, SES_FROM, SES_ACCESS_KEY, SES_SECRET_KEY and warns by name if any is blank — read that warning on the next run. No _GALACTUS suffix on any of them; one SES identity serves every deployment.
  2. SES_FROM is a verified identity in SES_REGION. An unverified sender is rejected per-send, which looks identical to a missing config in the log.
  3. The AWS account is out of the SES sandbox. This is the one that would hurt: in sandbox, SES only delivers to verified recipients, so a renewal sweep across 815 policyholders would fail almost every send while the config looks entirely correct. Check before letting a real sweep run.

Until 2 and 3 are confirmed, run the first sweep with debug on — it diverts every recipient to the override inbox, and on the pólizas side it also leaves the avisos pending, so nothing is consumed by a failed test. See MASS_EMAIL_NOTIFICATIONS.md "Send flags".

Also outstanding on the deploy path: every pre-existing database still needs its one-time prisma migrate resolve --applied 0000_init.


1. Blocked on a decision from Jorge

Nothing here is a build problem. Each one makes the work either impossible or a guess.

# Question Blocks Source
1.1 What "garantías" refers to §2 liquidación's exclusion filter INSURANCE §2
1.2 What "Solicitud Atlas" is — application form or certificate §3 entirely INSURANCE §3
1.3 Carrier API direction: outbound quote/issue (ANA supports today) or inbound portfolio sync (no evidence either carrier offers it) whether §4 is buildable at all INSURANCE §4
1.4 CFE amount: the rounded barcode figure ($268, what is paid at the window) or the exact breakdown total ($268.88) the parser currently takes the barcode STATEMENT_OCR / RECEIPT §2
1.5 The Seguros USD bank's name, currency and details multi-bank is built; that account does not exist yet RECEIPT §3
1.6 Whether recycling ever means a true data purge. The triggers are now settled and built (see §5 "NUMid allocation"); what is still open is whether a recycled id's old rows are ever deleted rather than left attached to the previous customer nothing — the allocator ships without a purge RECEIPT §4
1.7 Notice body in Spanish or English Customer carries no language preference INSURANCE §1
1.8 How to model TRASPASOS PAYPAL — a clearing account, not a customer, carrying 7.03M MXN over 309 movements and therefore topping the adeudo worklist deliberately not special-cased in code RESUME §6
1.9 The 78 policyholders with no email — skip silently or produce a print worklist recommendation is the worklist INSURANCE §1

1.3 also needs the practical half: WSDL + credentials for server.anaseguros.com.mx/ananetws/service.asmx, whether a cartera download exists for an agent's own book, whether GMX daños has any machine interface at all, and whether one Grupo Valore credential spans both carriers. All four go in the same phone call — (55) 5480-4000.


2. Live data defects — open, and confirmed open today

2.1 policy_types is missing INCENDIO and M_EMPR, and 5 policies are orphaned

policyTypeId is String? with a plain relation, so Prisma's default is SetNull. The spec's recommended onDelete: Restrict was never applied. Five m_empr policies lost their ramo; four of them are pending liquidación and are invisible to every ramo-filtered query — including the pending report §2 is supposed to produce.

Fix alongside the liquidación work (3.1), since it distorts that feature's own report. Source: INSURANCE "Two defects found while verifying this spec".

2.2 ≤41 MULT second settlements were dropped in migration

MULT/INCENDIO carry two settlement slots and M EMPR carries four; Policy collapses to one. Spec recommends moving settlement onto PolicyPaymentInstallment rather than adding a second slot. Open sub-question: whether to backfill the lost rows.

2.3 Three dead tables

EmailTemplate, EmailCampaign and EmailLog exist in the schema with zero references anywhere in apps/api/src or apps/web/src. They were scaffolded for plan step 10's "email campaigns"; notificaciones shipped against email_notification_log instead. Either wire them or drop them — a schema that carries tables nothing writes teaches the next reader the wrong thing.


3. Spec'd, not built

# Item State Source
3.1 Liquidación batch workflow ~70% of the fields already wired end to end. 226 policies pending. Needs the ramo-parameterized pending report + POST /policies/liquidate-batch under a new policy:liquidate (MANAGER). Smallest remaining piece of step 12 INSURANCE §2
3.2 Certificate rendering The report half is buildable now off the same format: "letter" machinery as aviso-renovacion. Portal delivery waits on steps 8/9. Whole section waits on 1.2 INSURANCE §3
3.3 Carrier API integration Blocked on 1.3. ANA's SOAP service is real with a known operation list; GMX publishes nothing machine-readable and writes the larger half of this book INSURANCE §4
3.4 Customer-number recycling Not started. Customer.customerNumber does not exist in the schema. Backfill needs care: ~140 utilities rows and all insurance-only customers carry synthetic rownum_N/insrow_N placeholders, not real NUM ids. Last unbuilt piece of step 11 RECEIPT §4

Note 3.3 partly overlaps what POLICY_OCR.md already does — an OCR path that turns a carrier PDF into a Policy row covers some of what the API was wanted for, and unlike the API it is not waiting on a phone call.


4. Roadmap steps with no spec

# Item State
4.1 Step 8 — VPS provisioning Not started. Provider undecided (Hetzner vs DigitalOcean), size, Tailscale + MySQL replica. Pure ops; the design is settled. RESUME calls this the only genuinely blocking item left on the roadmap
4.2 Step 9 — sync worker Not built. Unblocked now that utility_dbo and the portal code are on disk, but depends on 4.1. Portal write points to poll: peticion_gas, PayPal payments, notifications_settings, verification_codes
4.3 Step 10 — reports / campaigns / admin Mostly done by other work. /reportes exists; "email campaigns" landed as /notificaciones against a different table (see 2.3)
4.4 Phase B sync in production Verified 32/32 against dev, never run from the /operaciones UI (the OpsService path) nor against a prod-shaped database

5. Gaps in features that already shipped

Each of these is a known, deliberate stopping point rather than a bug.

NotificacionesMASS_EMAIL_NOTIFICATIONS.md

  • No multi-replica lock on the servicios sweep (pólizas has one via scheduled_job_states). Safe only while the deployment stays single-replica.
  • No per-recipient preview of a sent body in the UI.
  • No SNS bounce/complaint webhook. providerMessageId is captured so one can be added.
  • No SKIPPED_NO_EMAIL worklist (see 1.9).

Policy OCRPOLICY_OCR.md

  • GMX only. The dispatcher is a [provider, pattern] table plus a parser map, so a second carrier is one function and two entries — but no other layout has been seen, and guessing produces a parser nobody can verify.
  • The recibo PDF is unread. The GMX certificate carries no premium at all; reading the separate receipt and pairing it to its certificate is what would let postPremium stop being a manual tick.
  • No versioning. A re-issued policy arrives as a new certificate with the same number and confirm updates the existing row. Nothing records that this is the 2027 issue of that policy.

Statement OCRSTATEMENT_OCR.md

  • CFE / CESPT / Telnor have no unit suite. They predate the gas/predial extension and were verified end to end against the 46-page corpus only. Close this if those parsers are ever touched.
  • No way to re-run a corrected parser over a stored batch, though the source PDFs are kept precisely so it is possible.
  • Handwritten folder numbers are deliberately not an input to matching (Tesseract read 405 as 205).

NUMid allocationPOST /customers/:id/portal-access assigns the portal "Security Number", on a staff action rather than at create time, because an insurance-only customer has no reason to hold a utilities id.

  • Recycling is built but switched off. numid.recycleEmpty in app_settings defaults to false, and that default is a safety property, not a preference: every reusable id still exists in Access DATGRAL, and a --sync migration run upserts refs with ON DUPLICATE KEY UPDATE customerId (transform_customers.py:327), so an id recycled today is handed back to its Access owner on the next sync and the customer given it loses portal access. Flip it on after utilities cuts over, or for ids deleted at the source.
  • A full re-import would destroy every natively allocated id — now guarded. transform_customers.py:246 truncates customers and customer_legacy_refs (and the other transforms truncate everything they own), then rebuild from Access alone. migration/native_guard.py runs before any of it and refuses when the target holds rows Access has never seen; run_all.py --force-full, or the checkbox in the REIMPORT confirm, overrides and deletes them. --sync remains the correct path for any database with native rows — the guard stops the loss, it does not make full mode preserve anything.
  • The empty-id rule exists twice: enforced in numid.service.ts (EMPTY_NUMID_SQL) and reported by scripts/numid-audit.sql. They agree today (both return 1089, 1094, 1134, 1143 on dev); they are not mechanically kept in step, so change them together.
  • No un-assign. Nothing removes a NUMid once given, and nothing reports which ids were recycled from whom beyond the customer.portal-access activity-log entry.

Bank — the concept→ramo classifier is won't-build, not pending. concepto is a payee name (0 of 22,354 match a category) and TABLA RAMODOS is a property-management expense chart, not the business-line split it was assumed to be. /banco intentionally has no category dimension. Recorded here only because bank_transactions.categoryId being null on every row otherwise reads as unfinished work.


6. Security / hygiene

  • The old repo's dbConnection.php has a plaintext MySQL password committed to git history. Not carried into this platform, but the credential is already exposed and has not been rotated. Rotate regardless of this repo.
  • The pre-migrate backup step sets NODE_TLS_REJECT_UNAUTHORIZED=0 because Portainer serves a self-signed certificate. Scoped to that one step; the real fix is replacing the certificate.

Source documents

Document What it carries
../PLAN.md build sequencing, locked decisions, per-step status
../RESUME.md session history and §6 open items
INSURANCE_FEATURES_SPEC.md §1 renewal emails (built), §2 liquidación, §3 certificate, §4 carrier APIs
RECEIPT_CAPTURE_SPEC.md §1 Editor (built), §2 OCR (built), §3 multi-bank (built), §4 recycling
MASS_EMAIL_NOTIFICATIONS.md mass email + schedules, as built
STATEMENT_OCR.md · POLICY_OCR.md the two OCR intakes, as built
DEPLOY_AND_MIGRATIONS.md release chain, galactus, known caveats