feat(migration): refuse a full re-import that would delete native rows
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m51s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m42s

A full run_all.py pass truncates and rebuilds every table it owns from the
Access extract. That was harmless while the platform was a read-only mirror --
every row came from the extract, so wiping and rebuilding lost nothing. It
stopped being harmless once the platform started minting rows Access has never
heard of: allocated portal NUMids, customers created in the staff UI,
OCR-captured policies, app-booked ledger rows, uploaded documents.

REIMPORT is a button in /operaciones, so that was one click away.

native_guard.py counts what only exists here and exits 3; run_all.py runs it
before the first truncate and stops. Detecting an allocated NUMid needs the
staged Parquet -- the customer holds an ordinary-looking (utilities, DATGRAL,
'1172') ref, so "customer has no refs" cannot see it and only comparing against
the extract can. Missing staging is therefore treated as blocking rather than
as "nothing to protect".

The guard does not teach full mode to preserve anything: --sync already upserts
legacy rows against the existing refs and leaves the rest alone, and rebuilding
that inside full mode would re-implement it. --force-full (checkbox in the
REIMPORT confirm, recorded in the audit log) deletes them deliberately.

Verified against dev: clean before, exit 3 listing utilities/1172 with a
synthetic ref present, clean again after removing it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-06 21:01:01 -07:00
co-authored by Claude Opus 5
parent 6a97242fc3
commit 17d83291c3
10 changed files with 319 additions and 20 deletions
+8 -7
View File
@@ -195,13 +195,14 @@ insurance-only customer has no reason to hold a utilities id.
(`transform_customers.py:327`), so an id recycled today is handed back to its
Access owner on the next sync and the customer given it loses portal access.
**Flip it on after utilities cuts over**, or for ids deleted at the source.
- **A full re-import destroys every natively allocated id.**
`transform_customers.py:246` truncates `customers` and `customer_legacy_refs`,
then rebuilds the pool from Access alone. Until that is fixed — either by
mandating `--sync` for all future utilities loads, or by teaching the transform
to preserve refs with no Access counterpart — a NUMid issued here survives only
until the next full load. This is a prerequisite for the cutover, not a
nice-to-have.
- **A full re-import would destroy every natively allocated id — now guarded.**
`transform_customers.py:246` truncates `customers` and `customer_legacy_refs`
(and the other transforms truncate everything they own), then rebuild from
Access alone. `migration/native_guard.py` runs before any of it and refuses
when the target holds rows Access has never seen; `run_all.py --force-full`,
or the checkbox in the REIMPORT confirm, overrides and deletes them. **`--sync`
remains the correct path for any database with native rows** — the guard stops
the loss, it does not make full mode preserve anything.
- **The empty-id rule exists twice**: enforced in `numid.service.ts`
(`EMPTY_NUMID_SQL`) and reported by `scripts/numid-audit.sql`. They agree today
(both return 1089, 1094, 1134, 1143 on dev); they are not mechanically kept in