fix(migration): recover transaction type labels and minimum balance
Two fields the customer-facing site reads were being dropped on the way in from Access. transform_transactions.py mapped DATOS2's type string through the Access `TYPE OF TRX` table and stored NULL on a miss. That table is a stale pick-list rather than a constraint — staff free-text straight into DATOS2 — so 78 distinct values covering 3,939 rows never resolved, including BALANCE FORWARD (1,188) and ANNUAL FEE (1,116). Nothing else on `transactions` carries the type text, so those rows lost their label outright and rendered blank. Now mints a type_transactions row from the literal string when the lookup lacks it; nameEs stays NULL since only the lookup has translations. transform_customers.py never carried DATGRAL.TIPO, leaving customers.minimumBalance empty on every row despite the column existing. TIPO is the minimum-balance threshold (100/200/300/500; 1,017 of 1,172 customers carry one), not an account type as the name suggests — the customer app shows it as `minBalance`. Added to the insert list and to the ON DUPLICATE KEY UPDATE clause, without which --sync would silently skip it on existing rows. Both land on the next `run_all.py --sync` reload. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -112,6 +112,29 @@ def main():
|
||||
type_rows.append((tid, en, s(r["espa_ol"]), 0))
|
||||
type_map[en.upper()] = tid
|
||||
|
||||
def type_id_for(raw) -> str | None:
|
||||
"""Resolve a transaction type, minting one when the lookup lacks it.
|
||||
|
||||
The Access `TYPE OF TRX` table is a stale pick-list, not a constraint —
|
||||
staff free-text straight into DATOS2, so 78 values covering 3,939 rows
|
||||
(BALANCE FORWARD 1,188, ANNUAL FEE 1,116, IZZI 367, ...) appear in the
|
||||
ledger but not the lookup. Leaving those unmapped stored typeId NULL and
|
||||
lost the label outright: nothing else on `transactions` carries the type
|
||||
text, so the row rendered blank and was unrecoverable after migration.
|
||||
Minting from the literal keeps the display string; nameEs stays NULL
|
||||
because only the lookup has translations.
|
||||
"""
|
||||
en = s(raw)
|
||||
if not en:
|
||||
return None
|
||||
key = en.upper()
|
||||
tid = type_map.get(key)
|
||||
if tid is None:
|
||||
tid = str(uuid.uuid4())
|
||||
type_rows.append((tid, en, None, 0))
|
||||
type_map[key] = tid
|
||||
return tid
|
||||
|
||||
xr = load("stg_utilities", "tipo_hist")
|
||||
xr_rows = []
|
||||
for _, r in xr.iterrows():
|
||||
@@ -193,7 +216,7 @@ def main():
|
||||
td = dt(r["date"])
|
||||
if td is None:
|
||||
skip_date += 1; continue
|
||||
tid = type_map.get((s(r["type_of_trx"]) or "").upper())
|
||||
tid = type_id_for(r["type_of_trx"])
|
||||
add(cid, "UTILITY", td, dec(r["chargecredit"], Decimal(0)), "MXN",
|
||||
period=s(r["period"]), reference=s(r["refer"]), typeid=tid,
|
||||
check=s(r["cheque"]), src_db="UTILITIES", src_tbl=legacy_tbl,
|
||||
|
||||
Reference in New Issue
Block a user