From 81f62430b38f788045c422360d0d5cdeb3c4edeb Mon Sep 17 00:00:00 2001 From: Ricardo Mancinas Date: Wed, 22 Jul 2026 17:23:33 -0700 Subject: [PATCH] Reconciliation pass (migration step 2): decide union/de-dup rules Add migration/reconcile.py (reads staged Parquet) and the generated migration/RECONCILIATION.md. Naive full-row matching across the suspected "duplicate" groups gives a misleading ~0 overlap, so each group is probed on a deliberate business key instead. The evidence overturns all three of the plan's original assumptions: - EFECTIVO vs EFECTIVO_BACKUP: NOT a live/backup pair. `folio` is a per-table sequential number that collides (12,363 shared folio numbers, every one a different transaction); real business-key (cl,fecha,monto,conepto) overlap is 2. Near-disjoint ledgers (BACKUP ~2017-2022, EFECTIVO recent). Rule: migrate both, keyed by (source_table, folio) provenance, no folio de-dup, don't drop BACKUP. FM3 tables are a separate fee/tax/multa stream. - datos2 vs FEE ANUAL vs fee15: disjoint billing runs from different periods (2025-26 / 2018 / 2017), zero real-identity overlap. Rule: union all three, no de-dup; keep datos2.due_date. - DATGRAL vs COBRO3: COBRO3.fee is a constant 75 (a charge batch), not a filtered customer snapshot; every num_id already in DATGRAL. Rule: DATGRAL is the sole customer master, COBRO3 contributes zero customers. Also flags monedas currency variants (PESOS/Pesos/DOLLARS) for normalization at transform time. Update PLAN.md (migration step 2 outcome + corrected inventory bullets) and RESUME.md (queue: reconciliation done, transform+load next). Co-Authored-By: Claude Opus 4.8 --- PLAN.md | 11 ++- RESUME.md | 14 ++- migration/RECONCILIATION.md | 38 ++++++++ migration/reconcile.py | 183 ++++++++++++++++++++++++++++++++++++ 4 files changed, 238 insertions(+), 8 deletions(-) create mode 100644 migration/RECONCILIATION.md create mode 100644 migration/reconcile.py diff --git a/PLAN.md b/PLAN.md index ec01ceb..e08fe26 100644 --- a/PLAN.md +++ b/PLAN.md @@ -34,10 +34,10 @@ Extracted live via `pyodbc` + the Windows Access ODBC driver (`dump_schema.py`, **UTILITIES.accdb** (52 tables, ~538MB, dominated by embedded document blobs): - `DATMEX` (1,520 rows) — one row per property: address, phones, and inline utility fields for water/electric/gas/cable/property-tax/federal-zone/trust, each with its own due-date/route/account-number columns, plus 6 `LONGBINARY` columns holding scanned utility bills/IDs. -- `DATGRAL` / `COBRO3` (1,172 / 181 rows) — customer master data (name, MX + US address, phone, email, ID document, client-since date, fee, status). `COBRO3` looks like a filtered snapshot of `DATGRAL`, not a distinct entity. +- `DATGRAL` / `COBRO3` (1,172 / 181 rows) — customer master data (name, MX + US address, phone, email, ID document, client-since date, fee, status). ~~`COBRO3` looks like a filtered snapshot of `DATGRAL`~~ **Reconciliation (step 2) corrected this: `COBRO3` is a charge batch (constant fee 75), not a customer snapshot — `DATGRAL` is the sole customer master.** - `PROFILE` (1,520) — per-property service enrollment flags, joins 1:1 with `DATMEX` by `NUMERID`. -- `EFECTIVO` (13,697), `EFECTIVO FM3` (627), `EFECTIVO_BACKUP` (12,387), `CHEQUE FM3` (157) — cash/check transaction ledgers, near-identical shape, apparent year/program snapshots rather than distinct data. -- `FEE ANUAL` (1,082), `datos2` (16,000), `fee15` (1,030), `billing` (0) — recurring billing/fee transaction logs, again near-duplicate shapes across different periods/exports — needs de-duplication logic, not a straight union. +- `EFECTIVO` (13,697), `EFECTIVO FM3` (627), `EFECTIVO_BACKUP` (12,387), `CHEQUE FM3` (157) — cash/check transaction ledgers. ~~apparent year/program snapshots rather than distinct data~~ **Reconciliation (step 2) corrected this: EFECTIVO and EFECTIVO_BACKUP are near-disjoint ledgers (folio collides; only 2 business-key matches) — migrate both, no folio de-dup. FM3 tables are a separate fee stream.** +- `FEE ANUAL` (1,082), `datos2` (16,000), `fee15` (1,030), `billing` (0) — recurring billing/fee transaction logs. ~~near-duplicate shapes ... needs de-duplication~~ **Reconciliation (step 2) corrected this: disjoint billing runs from different periods (2017 / 2018 / 2025–26), zero real-identity overlap — straight union, no de-dup.** - `TRUSTVENCE` (549) — bank trust account fee due-dates. - `TIPO HIST` (2,301) — exchange-rate history (date/hour/rate), referenced by the `MONEDAS` (currency) field used throughout. - `TYPE OF TRX` (79) — ES/EN transaction-type lookup (already mirrored as `type_transactions` in the old MySQL schema — reuse that mapping). @@ -108,7 +108,10 @@ All tables get a surrogate `id` (uuid or serial) plus, where the row came from a Given the amount of near-duplicate/overlapping data across snapshot tables (multiple `EFECTIVO*` variants, multiple year-stamped billing tables, `COBRO3` vs `DATGRAL`), doing a direct Access → normalized-MySQL transform in one pass is risky — a bug loses the ability to check itself against the source. 1. **Raw staging load**: dump every non-scratch Access table 1:1 into a MySQL `staging` (per-source schema/database, e.g. `stg_utilities`/`stg_seguros`/`stg_scothia`) — same columns, minimal type coercion — via a Python script across all four source files. Already built and run against real data as `migration/load_staging.py` in the new repo — see Status below. This is the audit trail — nothing is transformed yet. **Extraction toolchain note:** the original build used `pyodbc` + the Windows Access ODBC driver; the project has since moved to a macOS machine, so the extraction layer (`migration/extract.py`) is being reworked to use **mdbtools** (`mdb-tables`/`mdb-export`, installed via Homebrew) instead. mdbtools has been verified against the real files to read table data, accented-column tables (which broke pyodbc's UTF-16 path — e.g. `PROPANO`), and per-table exports cleanly. mdbtools does **not** extract Forms/Reports/Queries, but those were already captured on Windows via DAO/COM and are frozen in `migration/objects.json` + `docs/LEGACY_DATABASES_OBJECTS.md`, so nothing is lost. The only piece needing extra handling under mdbtools is `LONGBINARY` blob/document extraction (step 4), where mdbtools emits the OLE wrapper — addressed when step 4 runs, not a blocker for steps 1–3. -2. **Reconciliation pass**: for each set of overlapping tables (the `EFECTIVO` variants, the billing-period tables, `DATGRAL` vs `COBRO3`), write SQL that diffs them and produces a report of exact duplicates vs. genuinely distinct records, before deciding the union/de-dupe rule. Don't guess the rule up front — the data decides it. +2. **Reconciliation pass** — **DONE** (`migration/reconcile.py` → `migration/RECONCILIATION.md`, run against the staged data). For each set of overlapping tables, it probes a deliberate *business key* (not naive full-row match, which gives a misleading ~0 overlap everywhere) and reports what's actually duplicate vs. distinct. **Outcome overturned all three of the plan's original "duplicate" assumptions — the union/de-dup rules below are now decided by the data:** + - **`EFECTIVO` vs `EFECTIVO_BACKUP`:** *not* a live/backup duplicate pair. `folio` is a per-table sequential number that **collides** (12,363 shared folio numbers, all carrying different transactions); on the real business key `(cl,fecha,monto,conepto)` only **2 rows** overlap. They are near-disjoint ledgers (BACKUP ≈ 2017–2022, EFECTIVO recent). **Rule: migrate both**, keyed internally by `(legacy_source_table, folio)` provenance; no folio de-dup, don't drop BACKUP. `EFECTIVO FM3`/`CHEQUE FM3` are a separate `fee/tax/multa` stream, migrated distinctly. (`monedas` needs currency normalization — `PESOS`/`Pesos`/`DOLLARS` variants.) + - **`datos2` vs `FEE ANUAL` vs `fee15`:** *not* near-duplicate exports. They are **disjoint billing runs from different periods** (`datos2` ≈2025–26, `FEE ANUAL` 2018-01-03, `fee15` 2017-01-10 — each period-table `refer` is a single constant); zero real-identity overlap. **Rule: migrate all three, no de-dup**; keep `datos2.due_date` (null for the others). + - **`DATGRAL` vs `COBRO3`:** `COBRO3` is *not* a filtered snapshot of the customer master — its `fee` is a **constant 75** for all 181 rows (a saved charge worklist / "cobro" = collection), and every `num_id` already exists in `DATGRAL`. **Rule: `DATGRAL` is the sole utilities customer master; COBRO3 contributes zero customers** — model its 181 rows as charge transactions if worth keeping, else exclude. 3. **Transform + load**: SQL/TypeScript scripts (versioned in the new repo under `migration/`) that read `staging`, apply the customer-matching and unpivot logic described above, and upsert into the real Prisma-managed tables, writing `legacy_*` provenance on every row. 4. **Document extraction**: separate one-off script pulls every `LONGBINARY` column out to files (named by provenance key), uploads to object storage, and inserts the corresponding `*_documents` metadata row. 5. **Validation**: row-count and spot-check reconciliation between `staging` and final tables (e.g., every legacy customer has exactly one `customers` row via `customer_legacy_refs`; sum of migrated transaction amounts per customer matches sum in `staging`). diff --git a/RESUME.md b/RESUME.md index 393ef18..fba8b9a 100644 --- a/RESUME.md +++ b/RESUME.md @@ -164,10 +164,16 @@ Homebrew. No Access ODBC driver, `node_modules` not installed, staging Parquet n row (mdb-export's `-b` / error handling; confirm the bad row is skipped, not fatal). 2. **Re-run staging** (`python load_staging.py --output-dir ./output`) to regenerate the staged data on this machine, then load into a local MySQL (`docker compose up mysql`) for SQL reconciliation. -3. **Reconciliation pass** (plan step 2) against staged data — the `EFECTIVO*` variants, the - billing-period tables, `DATGRAL` vs `COBRO3`. Produce diff/dedupe rules from the data. -4. **Transform + load** (plan step 3), starting with `Customer`/`CustomerLegacyRef` — every - other module depends on it. Use the `NUM UTIL` cross-ref + name/address matching. +3. **Reconciliation pass** (plan step 2) — **DONE** (`migration/reconcile.py` → `RECONCILIATION.md`). + Overturned all three "duplicate" assumptions: EFECTIVO/BACKUP are near-disjoint ledgers + (folio collides; migrate both), the billing tables are disjoint period runs (union all, no + de-dup), and COBRO3 is a charge batch not a customer snapshot (DATGRAL is sole master). The + decided union/de-dup rules are in `PLAN.md` migration step 2. +4. **Transform + load** (plan step 3) — NEXT. Start with `Customer`/`CustomerLegacyRef` (every + other module depends on it): DATGRAL (utilities) is the master; join insurance `DATGRAL` + via its `NUM UTIL` cross-ref + name/address matching; COBRO3 excluded from customers. Then + the ledger union per the reconciliation rules (both EFECTIVO tables, all three billing tables, + provenance-keyed; normalize `monedas` currency variants). 5. **Customer module** in `apps/api`/`apps/web` (list/search/detail) — first real feature, Spanish-first UI. Run `npm install` at repo root first (node_modules absent here). 6. **Sync design finalization** — now unblocked: map the internal→VPS replicated subset and the diff --git a/migration/RECONCILIATION.md b/migration/RECONCILIATION.md new file mode 100644 index 0000000..d4236bc --- /dev/null +++ b/migration/RECONCILIATION.md @@ -0,0 +1,38 @@ +# Reconciliation Report — Overlapping Legacy Tables + +_Migration plan step 2. Generated by `reconcile.py` from staged Parquet (`output/stg_utilities`). Regenerate: `./.venv/bin/python reconcile.py > RECONCILIATION.md`._ + +**Headline:** none of the three suspected "duplicate" groups are what the plan assumed. They are disjoint historical/period data or a mislabeled batch — see each group's decided rule. + +## 1. Cash ledger — `EFECTIVO` vs `EFECTIVO_BACKUP` + +- `EFECTIVO`: 13697 rows. `EFECTIVO_BACKUP`: 12387 rows. +- `folio` is per-table sequential: 13697 distinct in EFECTIVO (= row count), 12369 in BACKUP. 12363 folio *numbers* appear in both. +- **But of those 12363 shared folio numbers, 12363 carry a *different* transaction** (differ on ['cl', 'fecha', 'monto', 'conepto']). → `folio` collides; it is NOT a stable cross-table id. +- On the real business key `['cl', 'fecha', 'monto', 'conepto']`: **2 rows in both**, 13663 only in EFECTIVO, 12385 only in BACKUP. +- Date ranges are different eras: BACKUP is dominated by 2017–2022 records, EFECTIVO by recent ones. + +**Decided rule:** the two tables are **near-disjoint ledgers**, not a live/backup duplicate pair (only 2 shared payments out of ~13k+12k). Migrate **both** into `transactions`, each row keyed internally by `(legacy_source_table, folio)` provenance — do **not** de-dup on `folio` (it collides) and do **not** drop BACKUP (it holds ~12k older payments absent from EFECTIVO). The 2 business-key matches are the only possible double-counts and should be spot-checked, but at that volume they don't threaten balance integrity. + +> `EFECTIVO FM3` (627) and `CHEQUE FM3` (157) are a separate stream — `fee`/`tax`/`multa` columns instead of `monto` — and migrate as distinct transactions, not reconciled against EFECTIVO. + +> Data-quality note for the transform: `monedas` has case/spelling variants (`PESOS`/`Pesos`, `DOLARES`/`Dolares`/`DOLLARS`) — normalize currency on load. + +## 2. Billing/fee logs — `datos2` vs `FEE ANUAL` vs `fee15` + +- Rows: `datos2` 16000, `FEE ANUAL` 1082, `fee15` 1030. +- `date` spans: `datos2` ('2025-01-08 00:00:00', '2026-05-15 00:00:00'), `FEE ANUAL` ('2018-01-03 00:00:00', '2018-01-03 00:00:00'), `fee15` ('2017-01-10 00:00:00', '2017-01-10 00:00:00') — **three different periods**. +- `FEE ANUAL.refer` and `fee15.refer` are each a single constant value (1/1 distinct) — these are one-shot per-period fee runs, not general logs. +- Real-identity overlap `['numid', 'date', 'chargecredit']`: FEE ANUAL∩datos2 = 0, fee15∩datos2 = 0, FEE ANUAL∩fee15 = 0. + +**Decided rule:** `datos2`, `FEE ANUAL`, and `fee15` are **disjoint historical billing runs from different periods** (≈2025–26, 2018, 2017 respectively), not copies of one another. Migrate **all three** into the billing side of `transactions`, provenance-tagged; no de-dup is needed (zero real-identity overlap). Preserve `datos2.due_date` (the other two lack it — leave null for their rows). + +## 3. Customer master — `DATGRAL` vs `COBRO3` + +- `DATGRAL` 1172 rows / `COBRO3` 181 rows; both unique on `num_id`. +- Every `COBRO3.num_id` is in `DATGRAL` (181/181); it adds **no new customer id**. +- **`COBRO3.fee` is constant** (1 distinct value: ['75']) while `DATGRAL.fee` varies per customer — so COBRO3 is not a snapshot of the master's fee field. +- Sampling shows COBRO3 rows carry the flat charge with names/addresses denormalized (some blank), i.e. a saved *charge worklist*, not the authoritative customer record. + +**Decided rule:** `COBRO3` is a **billing/charge batch** ('cobro' = collection), not a customer table. `DATGRAL` is the sole utilities customer master. **Do not** merge COBRO3 into `customers` or let it overwrite any master field. If the flat charge has value as history, model those 181 rows as charge `transactions` (amount = the constant fee) keyed to the existing DATGRAL customers — otherwise exclude COBRO3 from the migration entirely. Either way it contributes zero new customers. + diff --git a/migration/reconcile.py b/migration/reconcile.py new file mode 100644 index 0000000..f411721 --- /dev/null +++ b/migration/reconcile.py @@ -0,0 +1,183 @@ +""" +Migration plan step 2: reconciliation pass. + +The legacy Access files carry several sets of near-duplicate / overlapping +tables (snapshot copies, per-period exports, filtered views). Before the +transform+load (step 3) can union them, we need to know — from the data, not +from a guess — which rows are exact duplicates across those tables and which +are genuinely distinct, so the de-dup / union rule is decided by evidence. + +Reads the staged Parquet produced by load_staging.py (`--output-dir`) and +prints a Markdown reconciliation report. Regenerate with: + + ./.venv/bin/python reconcile.py > RECONCILIATION.md + +Method note: naive full-row matching across these tables gives ~0 overlap, +which is a trap — it hides *why*. The interesting question is whether two +tables hold the SAME economic records under cosmetic differences, or +genuinely DIFFERENT records. So each group is probed on a deliberate +*business key* (the columns that identify the real-world thing) and the +volatile/identity columns are examined separately. Every table went through +the same mdb-export path, so identical source values serialize identically — +string comparison on a chosen key is a valid identity test. +""" + +from __future__ import annotations + +from pathlib import Path + +import pandas as pd + +STG = Path(__file__).parent / "output" / "stg_utilities" +_META = ("_legacy_source_table", "_row_num") +_NULL = "∅" + + +def load(name: str) -> pd.DataFrame: + df = pd.read_parquet(STG / f"{name}.parquet") + df = df[[c for c in df.columns if c not in _META]].copy() + for c in df.columns: + df[c] = df[c].astype("string").str.strip().fillna(_NULL) + return df + + +def keyset(df: pd.DataFrame, cols: list[str]) -> set[str]: + return set(df[cols].agg("\x1f".join, axis=1)) + + +def overlap(a: pd.DataFrame, b: pd.DataFrame, cols: list[str]): + ka, kb = keyset(a, cols), keyset(b, cols) + return len(ka & kb), len(ka - kb), len(kb - ka) + + +def constant_cols(df: pd.DataFrame) -> list[str]: + return [c for c in df.columns if df[c].nunique(dropna=False) <= 1] + + +def p(*a): + print(*a) + + +def main() -> None: + p("# Reconciliation Report — Overlapping Legacy Tables") + p("") + p("_Migration plan step 2. Generated by `reconcile.py` from staged Parquet " + "(`output/stg_utilities`). Regenerate: `./.venv/bin/python reconcile.py " + "> RECONCILIATION.md`._") + p("") + p("**Headline:** none of the three suspected \"duplicate\" groups are what " + "the plan assumed. They are disjoint historical/period data or a " + "mislabeled batch — see each group's decided rule.") + p("") + + # ------------------------------------------------------------------ # + # Group 1: EFECTIVO vs EFECTIVO_BACKUP + # ------------------------------------------------------------------ # + p("## 1. Cash ledger — `EFECTIVO` vs `EFECTIVO_BACKUP`") + p("") + ef, efb = load("efectivo"), load("efectivo_backup") + biz = ['cl', 'fecha', 'monto', 'conepto'] # what identifies a real payment + p(f"- `EFECTIVO`: {len(ef)} rows. `EFECTIVO_BACKUP`: {len(efb)} rows.") + # folio behaviour + fe = set(ef['folio']) - {_NULL} + fb = set(efb['folio']) - {_NULL} + p(f"- `folio` is per-table sequential: {ef['folio'].nunique()} distinct in " + f"EFECTIVO (= row count), {efb['folio'].nunique()} in BACKUP. " + f"{len(fe & fb)} folio *numbers* appear in both.") + both_folio = fe & fb + m = ef[ef['folio'].isin(both_folio)].drop_duplicates('folio').set_index('folio') + n = efb[efb['folio'].isin(both_folio)].drop_duplicates('folio').set_index('folio') + ci = m.index.intersection(n.index) + folio_conflict = int((m.loc[ci, biz] != n.loc[ci, biz]).any(axis=1).sum()) + p(f"- **But of those {len(ci)} shared folio numbers, {folio_conflict} carry " + f"a *different* transaction** (differ on {biz}). → `folio` collides; it is " + "NOT a stable cross-table id.") + b, oa, ob = overlap(ef, efb, biz) + p(f"- On the real business key `{biz}`: **{b} rows in both**, {oa} only in " + f"EFECTIVO, {ob} only in BACKUP.") + p("- Date ranges are different eras: BACKUP is dominated by 2017–2022 " + "records, EFECTIVO by recent ones.") + p("") + p("**Decided rule:** the two tables are **near-disjoint ledgers**, not a " + "live/backup duplicate pair (only " + str(b) + " shared payments out of " + "~13k+12k). Migrate **both** into `transactions`, each row keyed " + "internally by `(legacy_source_table, folio)` provenance — do **not** " + "de-dup on `folio` (it collides) and do **not** drop BACKUP (it holds " + "~12k older payments absent from EFECTIVO). The " + str(b) + " business-" + "key matches are the only possible double-counts and should be spot-" + "checked, but at that volume they don't threaten balance integrity.") + p("") + p("> `EFECTIVO FM3` (627) and `CHEQUE FM3` (157) are a separate stream — " + "`fee`/`tax`/`multa` columns instead of `monto` — and migrate as distinct " + "transactions, not reconciled against EFECTIVO.") + p("") + p("> Data-quality note for the transform: `monedas` has case/spelling " + "variants (`PESOS`/`Pesos`, `DOLARES`/`Dolares`/`DOLLARS`) — normalize " + "currency on load.") + p("") + + # ------------------------------------------------------------------ # + # Group 2: datos2 vs FEE ANUAL vs fee15 + # ------------------------------------------------------------------ # + p("## 2. Billing/fee logs — `datos2` vs `FEE ANUAL` vs `fee15`") + p("") + d2, fa, f15 = load("datos2"), load("fee_anual"), load("fee15") + p(f"- Rows: `datos2` {len(d2)}, `FEE ANUAL` {len(fa)}, `fee15` {len(f15)}.") + + def date_span(df): + d = sorted(x for x in df['date'].unique() if x != _NULL) + return (d[0], d[-1]) if d else ("∅", "∅") + p(f"- `date` spans: `datos2` {date_span(d2)}, `FEE ANUAL` {date_span(fa)}, " + f"`fee15` {date_span(f15)} — **three different periods**.") + p(f"- `FEE ANUAL.refer` and `fee15.refer` are each a single constant value " + f"({fa['refer'].nunique()}/{f15['refer'].nunique()} distinct) — these are " + "one-shot per-period fee runs, not general logs.") + idcols = ['numid', 'date', 'chargecredit'] + _, _, _ = overlap(fa, d2, idcols) + b1 = overlap(fa, d2, idcols)[0] + b2 = overlap(f15, d2, idcols)[0] + b3 = overlap(fa, f15, idcols)[0] + p(f"- Real-identity overlap `{idcols}`: FEE ANUAL∩datos2 = {b1}, " + f"fee15∩datos2 = {b2}, FEE ANUAL∩fee15 = {b3}.") + p("") + p("**Decided rule:** `datos2`, `FEE ANUAL`, and `fee15` are **disjoint " + "historical billing runs from different periods** (≈2025–26, 2018, 2017 " + "respectively), not copies of one another. Migrate **all three** into the " + "billing side of `transactions`, provenance-tagged; no de-dup is needed " + "(zero real-identity overlap). Preserve `datos2.due_date` (the other two " + "lack it — leave null for their rows).") + p("") + + # ------------------------------------------------------------------ # + # Group 3: DATGRAL vs COBRO3 + # ------------------------------------------------------------------ # + p("## 3. Customer master — `DATGRAL` vs `COBRO3`") + p("") + dg, cb = load("datgral"), load("cobro3") + dg_ids, cb_ids = set(dg['num_id']), set(cb['num_id']) + p(f"- `DATGRAL` {len(dg)} rows / `COBRO3` {len(cb)} rows; both unique on " + f"`num_id`.") + p(f"- Every `COBRO3.num_id` is in `DATGRAL` ({len(cb_ids & dg_ids)}/" + f"{len(cb_ids)}); it adds **no new customer id**.") + fee_const = cb['fee'].nunique() + fee_vals = list(cb['fee'].unique())[:3] + p(f"- **`COBRO3.fee` is constant** ({fee_const} distinct value: {fee_vals}) " + "while `DATGRAL.fee` varies per customer — so COBRO3 is not a snapshot of " + "the master's fee field.") + p("- Sampling shows COBRO3 rows carry the flat charge with names/addresses " + "denormalized (some blank), i.e. a saved *charge worklist*, not the " + "authoritative customer record.") + p("") + p("**Decided rule:** `COBRO3` is a **billing/charge batch** ('cobro' = " + "collection), not a customer table. `DATGRAL` is the sole utilities " + "customer master. **Do not** merge COBRO3 into `customers` or let it " + "overwrite any master field. If the flat charge has value as history, " + "model those 181 rows as charge `transactions` (amount = the constant " + "fee) keyed to the existing DATGRAL customers — otherwise exclude COBRO3 " + "from the migration entirely. Either way it contributes zero new " + "customers.") + p("") + + +if __name__ == "__main__": + main()