rmancinas/jorgecuadros-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Transform+load: shared ledger + SCOTHIA bank register (step 3 complete)

Browse Source 
migration/transform_transactions.py unions every cash/billing ledger into
`transactions` per the reconciliation rules: both EFECTIVO tables (no folio
de-dup, near-disjoint), all three billing tables (disjoint periods), the FM3
fee stream (amount = fee+tax+multa), IVA 2015 (nominal date), and insurance
EFECTIVO (domain INSURANCE). Also loads the type_transactions (EN/ES) and
exchange_rates lookups. Customer FK resolves through customer_legacy_refs;
rows with no resolvable customer/date are skipped and counted.
Loaded (dev): 45861 transactions (UTILITY 45566 / INSURANCE 295, 0 orphans),
79 type_transactions, 2301 exchange_rates.

migration/transform_bank.py loads SCOTHIA DATOS I/E into bank_transactions as
signed amounts (income +, expense -) and TABLA RAMODOS into
business_line_categories. Deliberately customer-independent (office's own
checking account). Loaded (dev): 22354 bank_transactions (net +899,375.77),
66 categories; categoryId left null (concept->ramo classifier is future work).

run_all.py: pipeline now customers -> properties -> policies -> transactions
-> bank, all idempotent. Verified full end-to-end run against dev.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
rmancinas
2026-07-22 18:43:34 -07:00
co-authored by Claude Opus 4.8
parent 21899e99bb
commit 83e3cb8f47
4 changed files with 397 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
migration/run_all.py
+5 -3
View File
@@ -31,9 +31,11 @@ PY = sys.executable # the venv python running this orchestrator
# Dependency order — extend as later modules land (policies, transactions, bank).
STEPS = [
"transform_customers.py", # customers + customer_legacy_refs (everything FKs to these)
"transform_properties.py", # properties + services + trust accounts
"transform_policies.py", # policies + installments/vehicles/drivers/benef/claims/adjusters
"transform_customers.py", # customers + customer_legacy_refs (everything FKs to these)
"transform_properties.py", # properties + services + trust accounts
"transform_policies.py", # policies + installments/vehicles/drivers/benef/claims/adjusters
"transform_transactions.py", # shared ledger + type_transactions + exchange_rates
"transform_bank.py", # SCOTHIA bank register (no customer FK; independent)
]