Lock plan: macOS move, mdbtools extraction, resolve open decisions

Project moved Windows -> macOS, retiring the pyodbc + Access ODBC driver
extraction path in favor of mdbtools (verified against the real files).
Record the four now-locked decisions and the resolved utility_dbo blocker.

- PLAN.md: rework migration step 1 for mdbtools; replace "Open decisions"
  with locked Decisions (extraction=mdbtools, i18n=Spanish-first,
  CI=Gitea Actions, utility_dbo=available); update Status for the macOS
  move + portal DB in hand; finalize sync steps against real utility_dbo.
- RESUME.md: rewrite next-steps as a locked execution queue.
- migration/config.py: point SOURCE_ROOT at ~/Downloads/JorgeCuadros-Legacy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-22 17:08:43 -07:00
co-authored by Claude Opus 4.8
parent 0268ed896c
commit 22a53bc562
3 changed files with 61 additions and 18 deletions
+33 -5
View File
@@ -141,9 +141,37 @@ python load_staging.py --database-url mysql+pymysql://user:pass@host:3306/ # l
- **No Docker, no local MySQL, no local Postgres** on this machine — `docker-compose.yml` and the MySQL-target mode of `load_staging.py` are written but unexecuted here. Test both on whatever machine ends up running this for real.
- Old repo's `dbConnection.php` has a **hardcoded plaintext MySQL password** for the external DB connection, committed to git history. Not carried forward into the new platform, but worth rotating that credential regardless since it's already exposed in the old repo's history.
## 8. Suggested next session starting point
## 8. Plan locked — next actions
1. Chase down open item #1 (`utility_dbo` schema) — it blocks finalizing the sync design concretely.
2. Start the reconciliation pass (plan step 2, open item #5) against the staged Parquet data — rerun `load_staging.py --output-dir` first since the original output didn't persist.
3. Once reconciliation rules are known, write the transform-and-load scripts (plan step 3) that populate the real Prisma-managed MySQL tables from staging, starting with the `Customer`/`CustomerLegacyRef` module since every other module depends on it.
4. In parallel or after: build out the Customer module in `apps/api`/`apps/web` (list/search/detail) — the first real feature, per Build Sequencing step 3 in the plan.
**All four previously-open decisions are now locked (2026-07-22).** See `PLAN.md`
→ "Decisions (locked)" for the authoritative record:
- Extraction toolchain (macOS): **mdbtools** (installed + verified against the real files).
- i18n: **Spanish-first**.
- CI/CD: **Gitea Actions** on `git.mancinas.io` → registry → Portainer.
- `utility_dbo`: **resolved** — full dump (`utility_dbo.sql`, 1.3 GB, 55 tables) and the
portal codebase (`~/PhpstormProjects/my-jorgecuadros-web`) are both on disk.
**Environment: moved Windows → macOS.** Sources now at `~/Downloads/JorgeCuadros-Legacy/`
(all four files). This machine has Docker, MySQL/MariaDB client, Node 22, Python 3.14,
Homebrew. No Access ODBC driver, `node_modules` not installed, staging Parquet not present.
**Execution queue (in order):**
1. **Port the extraction layer to mdbtools.** Rewrite `migration/extract.py` to shell out to
`mdb-tables`/`mdb-export` instead of `pyodbc`. Keep the same public interface
(`connect`/`list_tables`/`read_table`) so `load_staging.py` and `config.py` are unchanged
beyond the already-fixed `SOURCE_ROOT`. Carry over the two hard-won fixes conceptually:
accented-column tables (mdbtools reads `PROPANO` cleanly — verified) and the corrupted `MULT`
row (mdb-export's `-b` / error handling; confirm the bad row is skipped, not fatal).
2. **Re-run staging** (`python load_staging.py --output-dir ./output`) to regenerate the staged
data on this machine, then load into a local MySQL (`docker compose up mysql`) for SQL reconciliation.
3. **Reconciliation pass** (plan step 2) against staged data — the `EFECTIVO*` variants, the
billing-period tables, `DATGRAL` vs `COBRO3`. Produce diff/dedupe rules from the data.
4. **Transform + load** (plan step 3), starting with `Customer`/`CustomerLegacyRef` — every
other module depends on it. Use the `NUM UTIL` cross-ref + name/address matching.
5. **Customer module** in `apps/api`/`apps/web` (list/search/detail) — first real feature,
Spanish-first UI. Run `npm install` at repo root first (node_modules absent here).
6. **Sync design finalization** — now unblocked: map the internal→VPS replicated subset and the
VPS→internal inbox tables against the real `utility_dbo` schema and the portal's read/write
points in `my-jorgecuadros-web` (`peticion_gas`, PayPal payments, `notifications_settings`).
Only genuinely-pending item is **VPS provisioning** (ops task — provider/size/Tailscale+MySQL).