Every transform resolves its target through dbenv.database_url(), which lets a
DATABASE_URL in the process environment win — that is how the API container
drives a re-import against its own database with no deploy/ directory present.
blob_extract.py was the one step that bypassed it and called load_env()
directly for the MinIO credentials, so the "Operaciones" re-import loaded all
the data and then exited 1 on:
missing /repo/deploy/.env.prod — deploy the 'prod' DB stack and write its
.env first
Give the S3 settings the same resolution as the DB URL: load_env() now returns
{} for an absent file, and setting()/require() layer the process environment on
top of it. blob_extract reads S3_ENDPOINT / S3_BUCKET and accepts either
S3_ACCESS_KEY/S3_SECRET_KEY or MINIO_ROOT_USER/MINIO_ROOT_PASSWORD, matching
the fallback order in storage.service.ts and the vars the api service already
sets in deploy/galactus/jorgecuadros-app.compose.yml. A genuinely missing
setting still fails fast, now naming the variable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Working-tree checkpoint of in-progress work carried across prior
sessions on the feat/crud-rbac branch, committed so it lands on the
remote alongside the CI changes.
- Operaciones admin panel: apps/api/src/ops (ingest upload, backup /
restore / re-import jobs) wired into app.module + RBAC abilities, and
the apps/web/src/app/operaciones page. docker-compose gets INGEST_DIR
/ BACKUP_DIR volumes; .gitignore excludes migration/ingest + backups.
- migration/sync.py plus transform_*.py / run_all / config / dbenv /
blob_extract adjustments for the additive sync path.
- crud/rbac phase-5 web bits: AppShell, api/labels/types libs, globals.
- schema.prisma + PLAN/RESUME doc updates.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
migration/transform_properties.py loads properties, property_services and
trust_accounts from staged DATMEX/PROFILE, resolving each property's customer
FK through customer_legacy_refs. Services are derived from DATMEX's own
account/route/meter fields (the authoritative data); PROFILE flags — merged
best-effort on (numer_id,casa,direccion), which matched 1519/1519 — only
refine each service's `active`. Trust accounts are 1:1 from DATMEX trust
fields; TRUSTVENCE (overlapping) deferred to reconciliation; blobs are step 4.
Loaded/validated (dev): 1519 properties (0 orphans, 1 blank id skipped),
3486 services (ELECTRIC 1118 / PROPERTY_TAX 939 / WATER 859 / GAS 335 /
OTHER 115 / FEDERAL_ZONE 76 / CABLE 41 / ALARM 3), 553 trust accounts —
counts track the PROFILE enrollment flags.
Reproducibility (asked: dev must be redoable in prod):
- migration/dbenv.py: single DB-target source = deploy/.env.<env>'s
DATABASE_URL; connect(env) + env_arg() (--env, default dev).
- transform_customers.py / transform_properties.py now take --env instead of
hardcoding .env.dev.
- migration/run_all.py: runs every step in dependency order against --env
(optional --stage re-extracts from Access first). Reproducing dev->prod is
`run_all.py --env prod` after deploying the prod stack + prisma db push.
All steps are idempotent (truncate+rebuild); re-run yields identical counts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
rmancinasandClaude Opus 5