wip: ops admin panel + migration sync + crud/rbac phase-5 snapshot

Working-tree checkpoint of in-progress work carried across prior
sessions on the feat/crud-rbac branch, committed so it lands on the
remote alongside the CI changes.

- Operaciones admin panel: apps/api/src/ops (ingest upload, backup /
  restore / re-import jobs) wired into app.module + RBAC abilities, and
  the apps/web/src/app/operaciones page. docker-compose gets INGEST_DIR
  / BACKUP_DIR volumes; .gitignore excludes migration/ingest + backups.
- migration/sync.py plus transform_*.py / run_all / config / dbenv /
  blob_extract adjustments for the additive sync path.
- crud/rbac phase-5 web bits: AppShell, api/labels/types libs, globals.
- schema.prisma + PLAN/RESUME doc updates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-23 19:01:36 -07:00
co-authored by Claude Opus 4.8
parent 6ad0993a71
commit f1ef1c70b3
27 changed files with 1480 additions and 107 deletions
+7 -2
View File
@@ -128,7 +128,7 @@ Given the amount of near-duplicate/overlapping data across snapshot tables (mult
6. Shared billing/statements module (the payoff: one statement per customer spanning both utility and insurance transactions) — **DONE**. `apps/api/src/billing/` + web `/estado-cuenta` and `/estado-cuenta/[id]`. Two questions, two views: a per-customer **balances worklist** (who owes what) and a cross-customer **movement browser** (every charge and credit, filterable by line, concept, origin table and date range, with totals for the whole filtered set). The detail page is the actual statement: balance per currency, the same balance split by business line, charges broken out by concept, and the full movement list with a running balance. **Design constraint that shapes the whole module: balances are reported per currency and never collapsed into one number.** 912 of the 1,269 customers with a ledger move in both MXN and USD, the charge side is MXN-only while receipts arrive in both, and the legacy data never stored the exchange rate applied to a movement — so a single "total balance" would be a figure that never existed in the books.
7. Bank register module (`bank_transactions`/`business_line_categories` from SCOTHIA) — small, self-contained, and has no customer FK, so it can slot in independently once the core migration pipeline exists; low risk, low priority relative to the customer-facing modules.
8. VPS provisioning + Tailscale + MySQL replication setup. `utility_dbo`'s schema is now available (full dump on disk — 55 tables; see Status), so the exact replicated table/column set and inbox-table shape can be finalized against the real portal DB and the portal PHP code (`my-jorgecuadros-web`) that reads/writes it.
9. Sync worker (push replicated tables' relevant subset, poll inbox tables for payment/propane submissions) — depends on step 8. Portal write points confirmed present in `utility_dbo`: `peticion_gas` (propane requests), PayPal payment writes, `notifications_settings`, `verification_codes` — these define the VPS→internal inbox set.
9. Sync worker (push replicated tables' relevant subset, poll inbox tables for payment/propane submissions) — depends on step 8. **The separate Phase B Access additive sync is implemented:** `migration/run_all.py --sync` and the admin `SYNC` job upsert legacy-owned rows without truncating the database or touching manual rows. Portal write points confirmed present in `utility_dbo`: `peticion_gas` (propane requests), PayPal payment writes, `notifications_settings`, `verification_codes` — these define the VPS→internal inbox set.
10. Reports/email campaigns/admin — parity with old app's `reports.php`/`emailCampaigns.php` intent, rebuilt properly.
## Status
@@ -146,6 +146,9 @@ Repo scaffolded at `jorgecuadros-platform/`: npm workspaces, NestJS API with a r
- **i18n:** **Spanish-first** UI — matches the source data and staff usage.
- **CI/CD:** **Gitea Actions** on `git.mancinas.io` — build image, push to the git.mancinas.io container registry, deploy to Portainer (mirrors the `portainer-gitea-deploy` pattern already used on this LAN). Jenkins/`git.freakma.com` dropped.
- **`utility_dbo`:** resolved — full dump + portal code available (see Status).
- **Phase B Access additive sync:** implemented in `migration/run_all.py --sync` and the admin
`SYNC` job. It preserves manual rows and stable legacy-owned primary keys; end-to-end database
validation remains before production use.
## Open items (ops, not design)
@@ -156,5 +159,7 @@ Repo scaffolded at `jorgecuadros-platform/`: npm workspaces, NestJS API with a r
- Migration: automated row-count/sum reconciliation between `staging` and final schema per table group (see step 5 above), run as part of the migration script, not a manual spot-check.
- App: standard NestJS unit/integration tests per module (auth guards, Prisma queries), Playwright/Cypress e2e for the core "look up a customer, see their unified policies + services + statement" flow — the thing the whole project exists to deliver.
- Sync: once the VPS replica and inbox tables exist, verify replication lag stays low (a few seconds to low minutes) and that a payment/propane submission on the portal reliably shows up in the internal app within one polling interval, before relying on it operationally.
- **Sync:** Phase B Access additive sync now has automated CLI/admin wiring, but must be verified
against a disposable DB with stable-PK, manual-row, update, and source-delete cases. The
separate VPS/portal sync still requires VPS provisioning and inbox-table implementation.
- Before cutover: run the new app against migrated data side-by-side with the live Access files for a period, comparing balances/statements for a sample of active customers to catch migration logic errors before the Access files are retired.