Compare commits

...
3 Commits
Author SHA1 Message Date
rmancinasandClaude Opus 4.8 12a1523073 docs: record step 6, correct the EFECTIVO verdict, refresh stale state
PLAN.md:
- Migration step 2: replace the "near-disjoint ledgers, migrate both" rule
  with the corrected de-dup rule, plus a box explaining why the original
  verdict was wrong so the reversal is auditable rather than silent.
- Note that transactions.amount is signed and that currencies are never
  summed.
- Build sequencing step 6 marked done.

RESUME.md — the execution queue still stated the reverted EFECTIVO verdict
verbatim, so a fresh session reading top-to-bottom would have hit the old
rule in step 3 and the correction in step 4 with no way to tell which won.
Beyond that fix, several sections still described the pre-macOS-move world:
- §2: every source path was C:\Users\ricar\...; the repo was described as
  "not yet a git repository".
- §4.4: described the pyodbc + Access ODBC extraction rather than mdbtools.
- §6: four of five "open items" were already resolved.
- §7: documented the old Windows box. Now the macOS machine, plus the traps
  worth knowing — run_all.py vs single transforms, `next build` clobbering a
  running dev server's .next, and the mdb-export numeric formatting trap.
- §8: items were mis-numbered (5b before 5) and item 5 was work finished
  many sessions ago. Renumbered, with an explicit "next" block.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 23:29:45 -07:00
rmancinasandClaude Opus 4.8 2c6a6bf60b feat(billing): shared statements module across both business lines
Plan step 6 — the payoff of the unified customer record: a utility charge
and an insurance payment finally sit on the same page, under the same
person, with a running balance.

API (apps/api/src/billing/):
- GET /billing — cross-customer movement browser. Search over customer,
  referencia, cheque, concepto and periodo; filters for business line,
  currency, charge-vs-credit, concept, origin table and a from/to date
  range; 5 sorts. Returns totals for the whole filtered set, not just the
  page, so a filtered view can't be misread as the full ledger.
- GET /billing/balances — per-customer receivables worklist with
  owing/credit/settled buckets and 4 sorts. Raw SQL (parameterized via
  Prisma.sql): needs conditional sums per currency and per direction in
  one pass plus ordering and pagination on a computed balance, none of
  which groupBy expresses.
- GET /billing/stats, /billing/facets, /billing/customers/:id.

Web:
- /estado-cuenta — two views over the same ledger, because staff ask two
  different questions: "Saldos por cliente" (who owes what) and
  "Movimientos" (every charge and credit).
- /estado-cuenta/[id] — the statement: balance per currency, the same
  balance split by business line, charges broken out by concept, and the
  full movement list with a running balance.
- Cross-linked from the customer and property detail pages.

Two data findings shape the whole module:

1. transactions.amount is a signed ledger. Every charge type is negative
   without exception (WATER 3115/3117, ELECTRIC 2191/2191, PROPERTY TAXES
   926/926, TRUST FEE 188/188) and every deposit type positive (CHECK and
   CASH DEPOSIT, PAYPAL, all of EFECTIVO). So SUM(amount) is the balance
   and negative means the customer owes the office.

2. Currency is not summable. 912 of the 1269 customers with a ledger move
   in both MXN and USD, the charge side is MXN-only while receipts arrive
   in both, and no per-movement exchange rate was ever stored. A single
   "total balance" would be a figure that never existed in the books, so
   every total is reported per currency and the balance filter/sort takes
   a currency argument rather than collapsing.

Also: type_transactions.nameEs is entirely null (the legacy TYPE OF TRX
ESPAÑOL column is empty in all 79 rows), so Spanish concept names come
from a label map in labels.ts; the entries that are payee names rather
than categories fall through untranslated, which is correct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 23:29:19 -07:00
rmancinasandClaude Opus 4.8 9de8e4e6c0 fix(migration): de-duplicate EFECTIVO_BACKUP against EFECTIVO
The reconciliation pass ruled EFECTIVO and EFECTIVO_BACKUP "near-disjoint
ledgers" and the transform loaded both in full. That verdict was a bug, not
a finding.

reconcile.py compared the business key (cl, fecha, monto, conepto) as raw
strings, on the stated premise that "every table went through the same
mdb-export path, so identical source values serialize identically". They
don't: mdb-export formats a numeric column from its Access column type, so
the same amount is emitted as `5000` from one table and `27000.0000` from
the other. No two rows could ever match on `monto`, which is why the pass
reported 2 overlapping rows.

Canonicalizing numeric key columns first shows 12386 of EFECTIVO_BACKUP's
12387 rows already exist verbatim in EFECTIVO — same customer, same
timestamp to the second, same amount, same concept text — leaving exactly
one genuinely new row. The ledger was carrying 12386 duplicated payments,
roughly doubling every customer's historical receipt total.

- reconcile.py: add canon(), which parses a key column to a number when
  nearly every populated cell parses and re-emits it at fixed precision.
  Applied in keyset() and in the folio-conflict comparison. Rewrite the
  group-1 verdict and the module docstring's method note.
- transform_transactions.py: share a business-key `seen` set between the
  two efectivo_like() calls. EFECTIVO loads first and wins collisions.
  De-dup on the business key, never on folio — folio is per-table
  sequential and collides on 12204 different payments.
- Regenerate RECONCILIATION.md. Groups 2 and 3 re-checked under the fix;
  their verdicts are unchanged.

Ledger after re-running run_all.py --env dev: 45861 -> 33475 rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 23:28:18 -07:00
18 changed files with 3076 additions and 115 deletions
+6 -4
View File
@@ -96,7 +96,7 @@ All tables get a surrogate `id` (uuid or serial) plus, where the row came from a
- `service_documents` (extracted blobs), `trust_accounts` (from `TRUSTVENCE`). - `service_documents` (extracted blobs), `trust_accounts` (from `TRUSTVENCE`).
**Shared financial ledger** (one office, one set of books — no reason to keep insurance and utility transactions in separate schemas): **Shared financial ledger** (one office, one set of books — no reason to keep insurance and utility transactions in separate schemas):
- `transactions` — unifies utilities' `EFECTIVO`/`EFECTIVO FM3`/`EFECTIVO_BACKUP`/`FEE ANUAL`/`datos2`/`fee15`/`billing`/`CHEQUE FM3`/`IVA 2015` and insurance's `EFECTIVO`, tagged by `domain` (utility/insurance/trust) and carrying the provenance columns so the de-duplication across those overlapping snapshot tables is traceable, not destructive. - `transactions` — unifies utilities' `EFECTIVO`/`EFECTIVO FM3`/`EFECTIVO_BACKUP`/`FEE ANUAL`/`datos2`/`fee15`/`billing`/`CHEQUE FM3`/`IVA 2015` and insurance's `EFECTIVO`, tagged by `domain` (utility/insurance/trust) and carrying the provenance columns so the de-duplication across those overlapping snapshot tables is traceable, not destructive. **`amount` is signed:** negative = charge (cargo), positive = credit (abono), so `SUM(amount)` per customer per currency *is* the balance — negative means the customer owes the office. The two currencies are never summed together (see the billing module note in Build sequencing step 6).
- `exchange_rates` (from `TIPO HIST`), `type_transactions` (carry over ES/EN lookup as-is). - `exchange_rates` (from `TIPO HIST`), `type_transactions` (carry over ES/EN lookup as-is).
- `bank_transactions` — the company's own operating bank register, from SCOTHIA's `DATOS E`/`DATOS I` unified into one signed-amount table (income positive, expense negative) with a `category` FK to `business_line_categories` (from `TABLA RAMODOS`) and a `cleared`/`operado` flag. This is deliberately **separate** from customer-facing `transactions` — it's the office's own bank reconciliation book, not money owed by/to a customer — but sharing the `business_line_categories` lookup lets you eventually answer "how much of our actual bank activity ties back to insurance vs. utilities vs. trust," which is a natural reporting win from unifying these three sources. - `bank_transactions` — the company's own operating bank register, from SCOTHIA's `DATOS E`/`DATOS I` unified into one signed-amount table (income positive, expense negative) with a `category` FK to `business_line_categories` (from `TABLA RAMODOS`) and a `cleared`/`operado` flag. This is deliberately **separate** from customer-facing `transactions` — it's the office's own bank reconciliation book, not money owed by/to a customer — but sharing the `business_line_categories` lookup lets you eventually answer "how much of our actual bank activity ties back to insurance vs. utilities vs. trust," which is a natural reporting win from unifying these three sources.
- `business_line_categories` (from `TABLA RAMODOS`). - `business_line_categories` (from `TABLA RAMODOS`).
@@ -108,8 +108,10 @@ All tables get a surrogate `id` (uuid or serial) plus, where the row came from a
Given the amount of near-duplicate/overlapping data across snapshot tables (multiple `EFECTIVO*` variants, multiple year-stamped billing tables, `COBRO3` vs `DATGRAL`), doing a direct Access → normalized-MySQL transform in one pass is risky — a bug loses the ability to check itself against the source. Given the amount of near-duplicate/overlapping data across snapshot tables (multiple `EFECTIVO*` variants, multiple year-stamped billing tables, `COBRO3` vs `DATGRAL`), doing a direct Access → normalized-MySQL transform in one pass is risky — a bug loses the ability to check itself against the source.
1. **Raw staging load**: dump every non-scratch Access table 1:1 into a MySQL `staging` (per-source schema/database, e.g. `stg_utilities`/`stg_seguros`/`stg_scothia`) — same columns, minimal type coercion — via a Python script across all four source files. Already built and run against real data as `migration/load_staging.py` in the new repo — see Status below. This is the audit trail — nothing is transformed yet. **Extraction toolchain note:** the original build used `pyodbc` + the Windows Access ODBC driver; the project has since moved to a macOS machine, so the extraction layer (`migration/extract.py`) is being reworked to use **mdbtools** (`mdb-tables`/`mdb-export`, installed via Homebrew) instead. mdbtools has been verified against the real files to read table data, accented-column tables (which broke pyodbc's UTF-16 path — e.g. `PROPANO`), and per-table exports cleanly. mdbtools does **not** extract Forms/Reports/Queries, but those were already captured on Windows via DAO/COM and are frozen in `migration/objects.json` + `docs/LEGACY_DATABASES_OBJECTS.md`, so nothing is lost. The only piece needing extra handling under mdbtools is `LONGBINARY` blob/document extraction (step 4), where mdbtools emits the OLE wrapper — addressed when step 4 runs, not a blocker for steps 13. 1. **Raw staging load**: dump every non-scratch Access table 1:1 into a MySQL `staging` (per-source schema/database, e.g. `stg_utilities`/`stg_seguros`/`stg_scothia`) — same columns, minimal type coercion — via a Python script across all four source files. Already built and run against real data as `migration/load_staging.py` in the new repo — see Status below. This is the audit trail — nothing is transformed yet. **Extraction toolchain note:** the original build used `pyodbc` + the Windows Access ODBC driver; the project has since moved to a macOS machine, so the extraction layer (`migration/extract.py`) is being reworked to use **mdbtools** (`mdb-tables`/`mdb-export`, installed via Homebrew) instead. mdbtools has been verified against the real files to read table data, accented-column tables (which broke pyodbc's UTF-16 path — e.g. `PROPANO`), and per-table exports cleanly. mdbtools does **not** extract Forms/Reports/Queries, but those were already captured on Windows via DAO/COM and are frozen in `migration/objects.json` + `docs/LEGACY_DATABASES_OBJECTS.md`, so nothing is lost. The only piece needing extra handling under mdbtools is `LONGBINARY` blob/document extraction (step 4), where mdbtools emits the OLE wrapper — addressed when step 4 runs, not a blocker for steps 13.
2. **Reconciliation pass****DONE** (`migration/reconcile.py``migration/RECONCILIATION.md`, run against the staged data). For each set of overlapping tables, it probes a deliberate *business key* (not naive full-row match, which gives a misleading ~0 overlap everywhere) and reports what's actually duplicate vs. distinct. **Outcome overturned all three of the plan's original "duplicate" assumptions — the union/de-dup rules below are now decided by the data:** 2. **Reconciliation pass****DONE** (`migration/reconcile.py``migration/RECONCILIATION.md`, run against the staged data). For each set of overlapping tables, it probes a deliberate *business key* (not naive full-row match, which gives a misleading ~0 overlap everywhere) and reports what's actually duplicate vs. distinct. The union/de-dup rules below are decided by the data:
- **`EFECTIVO` vs `EFECTIVO_BACKUP`:** *not* a live/backup duplicate pair. `folio` is a per-table sequential number that **collides** (12,363 shared folio numbers, all carrying different transactions); on the real business key `(cl,fecha,monto,conepto)` only **2 rows** overlap. They are near-disjoint ledgers (BACKUP ≈ 20172022, EFECTIVO recent). **Rule: migrate both**, keyed internally by `(legacy_source_table, folio)` provenance; no folio de-dup, don't drop BACKUP. `EFECTIVO FM3`/`CHEQUE FM3` are a separate `fee/tax/multa` stream, migrated distinctly. (`monedas` needs currency normalization — `PESOS`/`Pesos`/`DOLLARS` variants.) - **`EFECTIVO` vs `EFECTIVO_BACKUP`: `EFECTIVO_BACKUP` is a stale backup copy — de-dup it. (Corrected 2026-07-22; see the box below.)** On the canonicalized business key `(cl,fecha,monto,conepto)`, **12,386 of BACKUP's 12,387 rows already exist verbatim in `EFECTIVO`** — same customer, same timestamp to the second, same amount, same concept text — leaving exactly **1** genuinely new row. `folio` is a per-table sequential number that **collides** (12,363 shared numbers, 12,204 of them on different payments), so it can never be the de-dup key. **Rule: load `EFECTIVO` in full; from `EFECTIVO_BACKUP` load only business-key-new rows.** `EFECTIVO FM3`/`CHEQUE FM3` are a separate `fee/tax/multa` stream, migrated distinctly. (`monedas` needs currency normalization — `PESOS`/`Pesos`/`DOLLARS` variants.)
> **Why this was wrong the first time.** The original pass reported only **2** overlapping rows and concluded the two tables were "near-disjoint ledgers, migrate both". That verdict came from a bug in `reconcile.py`, which compared business-key columns as raw strings on the premise that "every table went through the same mdb-export path, so identical source values serialize identically". They don't: `mdb-export` formats a numeric column from its *Access column type*, so the same amount is emitted as `5000` from one table and `27000.0000` from the other, and no two rows could ever match on `monto`. `reconcile.py` now canonicalizes numeric key columns before comparing. The bad rule had already been loaded: the ledger carried 45,861 rows with **12,386 duplicated payments**, roughly doubling every customer's historical receipt total — which would have made every balance and statement in step 6 wrong. Re-running `run_all.py` brings the ledger to **33,475** rows. Groups 2 and 3 below were re-checked under the fix and their verdicts are unchanged.
- **`datos2` vs `FEE ANUAL` vs `fee15`:** *not* near-duplicate exports. They are **disjoint billing runs from different periods** (`datos2` ≈202526, `FEE ANUAL` 2018-01-03, `fee15` 2017-01-10 — each period-table `refer` is a single constant); zero real-identity overlap. **Rule: migrate all three, no de-dup**; keep `datos2.due_date` (null for the others). - **`datos2` vs `FEE ANUAL` vs `fee15`:** *not* near-duplicate exports. They are **disjoint billing runs from different periods** (`datos2` ≈202526, `FEE ANUAL` 2018-01-03, `fee15` 2017-01-10 — each period-table `refer` is a single constant); zero real-identity overlap. **Rule: migrate all three, no de-dup**; keep `datos2.due_date` (null for the others).
- **`DATGRAL` vs `COBRO3`:** `COBRO3` is *not* a filtered snapshot of the customer master — its `fee` is a **constant 75** for all 181 rows (a saved charge worklist / "cobro" = collection), and every `num_id` already exists in `DATGRAL`. **Rule: `DATGRAL` is the sole utilities customer master; COBRO3 contributes zero customers** — model its 181 rows as charge transactions if worth keeping, else exclude. - **`DATGRAL` vs `COBRO3`:** `COBRO3` is *not* a filtered snapshot of the customer master — its `fee` is a **constant 75** for all 181 rows (a saved charge worklist / "cobro" = collection), and every `num_id` already exists in `DATGRAL`. **Rule: `DATGRAL` is the sole utilities customer master; COBRO3 contributes zero customers** — model its 181 rows as charge transactions if worth keeping, else exclude.
3. **Transform + load**: SQL/TypeScript scripts (versioned in the new repo under `migration/`) that read `staging`, apply the customer-matching and unpivot logic described above, and upsert into the real Prisma-managed tables, writing `legacy_*` provenance on every row. 3. **Transform + load**: SQL/TypeScript scripts (versioned in the new repo under `migration/`) that read `staging`, apply the customer-matching and unpivot logic described above, and upsert into the real Prisma-managed tables, writing `legacy_*` provenance on every row.
@@ -123,7 +125,7 @@ Given the amount of near-duplicate/overlapping data across snapshot tables (mult
3. Customer module (list/search/detail — the unified view is the core deliverable) backed by finished migration steps 35 for customers only. 3. Customer module (list/search/detail — the unified view is the core deliverable) backed by finished migration steps 35 for customers only.
4. Insurance module (policies, vehicles, beneficiaries, claims) on top of the same customer records. 4. Insurance module (policies, vehicles, beneficiaries, claims) on top of the same customer records.
5. Utilities module (properties, services, trust accounts) on top of the same customer records. 5. Utilities module (properties, services, trust accounts) on top of the same customer records.
6. Shared billing/statements module (the payoff: one statement per customer spanning both utility and insurance transactions). 6. Shared billing/statements module (the payoff: one statement per customer spanning both utility and insurance transactions)**DONE**. `apps/api/src/billing/` + web `/estado-cuenta` and `/estado-cuenta/[id]`. Two questions, two views: a per-customer **balances worklist** (who owes what) and a cross-customer **movement browser** (every charge and credit, filterable by line, concept, origin table and date range, with totals for the whole filtered set). The detail page is the actual statement: balance per currency, the same balance split by business line, charges broken out by concept, and the full movement list with a running balance. **Design constraint that shapes the whole module: balances are reported per currency and never collapsed into one number.** 912 of the 1,269 customers with a ledger move in both MXN and USD, the charge side is MXN-only while receipts arrive in both, and the legacy data never stored the exchange rate applied to a movement — so a single "total balance" would be a figure that never existed in the books.
7. Bank register module (`bank_transactions`/`business_line_categories` from SCOTHIA) — small, self-contained, and has no customer FK, so it can slot in independently once the core migration pipeline exists; low risk, low priority relative to the customer-facing modules. 7. Bank register module (`bank_transactions`/`business_line_categories` from SCOTHIA) — small, self-contained, and has no customer FK, so it can slot in independently once the core migration pipeline exists; low risk, low priority relative to the customer-facing modules.
8. VPS provisioning + Tailscale + MySQL replication setup. `utility_dbo`'s schema is now available (full dump on disk — 55 tables; see Status), so the exact replicated table/column set and inbox-table shape can be finalized against the real portal DB and the portal PHP code (`my-jorgecuadros-web`) that reads/writes it. 8. VPS provisioning + Tailscale + MySQL replication setup. `utility_dbo`'s schema is now available (full dump on disk — 55 tables; see Status), so the exact replicated table/column set and inbox-table shape can be finalized against the real portal DB and the portal PHP code (`my-jorgecuadros-web`) that reads/writes it.
9. Sync worker (push replicated tables' relevant subset, poll inbox tables for payment/propane submissions) — depends on step 8. Portal write points confirmed present in `utility_dbo`: `peticion_gas` (propane requests), PayPal payment writes, `notifications_settings`, `verification_codes` — these define the VPS→internal inbox set. 9. Sync worker (push replicated tables' relevant subset, poll inbox tables for payment/propane submissions) — depends on step 8. Portal write points confirmed present in `utility_dbo`: `peticion_gas` (propane requests), PayPal payment writes, `notifications_settings`, `verification_codes` — these define the VPS→internal inbox set.
+176 -78
View File
@@ -4,11 +4,11 @@ Comprehensive state-of-the-world doc for picking this project back up. Read this
before doing anything else in a fresh session — it front-loads everything that before doing anything else in a fresh session — it front-loads everything that
took multiple rounds of investigation to establish. took multiple rounds of investigation to establish.
**Companion doc:** the full architecture/migration plan is copied into this **Companion doc:** the full architecture/migration plan is [`PLAN.md`](PLAN.md) in
repo as [`PLAN.md`](PLAN.md) (source of truth is this repo — **that is the source of truth for the design.** (It began as
`C:\Users\ricar\.claude\plans\logical-yawning-tome.md` — copy here if that `~/.claude/plans/logical-yawning-tome.md` on the old Windows machine; that copy is
one gets updated further). This file is the "what happened and what's next" gone and no longer authoritative.) This file is the "what happened and what's next"
companion to that plan, not a replacement for it. Read both. companion, not a replacement. Read both.
--- ---
@@ -32,28 +32,37 @@ infrastructure decisions below.
## 2. Where everything lives (file paths) ## 2. Where everything lives (file paths)
**Source data (do not modify — read-only references):** > Paths below are the **current macOS machine**. The project moved Windows → macOS on
- `C:\Users\ricar\Downloads\Jorge\UTILITIES.accdb` — utilities business, 52 tables, ~538MB > 2026-07-22; anything still written as `C:\Users\ricar\...` in older notes is stale.
- `C:\Users\ricar\Downloads\Jorge\SEGUROS 16.mdb` — insurance frontend shell, **empty**, all data is in `_be`
- `C:\Users\ricar\Downloads\Jorge\SEGUROS 16_be.mdb` — insurance backend, 64 tables, ~882MB **Source data (do not modify — read-only references), all in `~/Downloads/JorgeCuadros-Legacy/`:**
- `C:\Users\ricar\Downloads\Jorge\SCOTHIA.mdb` — office's own Scotiabank checking register ("chequera"), 7 tables, ~3MB - `UTILITIES.accdb` — utilities business, 52 tables, ~538MB
- `SEGUROS 16.mdb` — insurance frontend shell, **no data tables**, but holds *all* of the
insurance line's Reports/Forms/Queries
- `SEGUROS 16_be.mdb` — insurance backend, 64 tables, ~882MB
- `SCOTHIA.mdb` — office's own Scotiabank checking register ("chequera"), 7 tables, ~3MB
- `utility_dbo.sql` — customer portal's live DB dump (1.3 GB, 55 tables)
- `jorgecuadros.sql` — older/partial export (38 MB, 11 tables), **not** the portal live DB
- **Full structural reference for all three, usable without Windows or the original files:** [`docs/LEGACY_DATABASES.md`](docs/LEGACY_DATABASES.md) — every table, every column with type/nullability, the cross-reference keys between the three databases, and every known data-quality quirk (the UTF-16 decode bug, the corrupted `MULT` row, near-duplicate snapshot tables, etc.), all generated from a live read of the real files via `migration/catalog_schema.py`. Regenerate it if the source files change; the raw JSON it's built from is checked in at `migration/catalog.json`. - **Full structural reference for all three, usable without Windows or the original files:** [`docs/LEGACY_DATABASES.md`](docs/LEGACY_DATABASES.md) — every table, every column with type/nullability, the cross-reference keys between the three databases, and every known data-quality quirk (the UTF-16 decode bug, the corrupted `MULT` row, near-duplicate snapshot tables, etc.), all generated from a live read of the real files via `migration/catalog_schema.py`. Regenerate it if the source files change; the raw JSON it's built from is checked in at `migration/catalog.json`.
- **Queries/Forms/Reports reference:** [`docs/LEGACY_DATABASES_OBJECTS.md`](docs/LEGACY_DATABASES_OBJECTS.md) — none of this is visible via ODBC/`pyodbc`; it required DAO COM automation (`migration/catalog_objects.py`, needs `pywin32`) instead. Found 311 Reports, 271 Forms, and 1,274 Queries (751 "real," the rest Access-internal hidden subquery caches) across the three populated files — importantly, `SEGUROS 16.mdb` (which has zero data tables) turned out to hold *all* of the insurance line's Reports/Forms/Queries; `SEGUROS 16_be.mdb` is confirmed pure data storage. The real queries' full SQL text is the best available record of actual business logic (billing math, renewal batching) — worth reading before reimplementing any given feature from scratch. Raw JSON checked in at `migration/objects.json`. - **Queries/Forms/Reports reference:** [`docs/LEGACY_DATABASES_OBJECTS.md`](docs/LEGACY_DATABASES_OBJECTS.md) — none of this is visible via ODBC/`pyodbc`; it required DAO COM automation (`migration/catalog_objects.py`, needs `pywin32`) instead. Found 311 Reports, 271 Forms, and 1,274 Queries (751 "real," the rest Access-internal hidden subquery caches) across the three populated files — importantly, `SEGUROS 16.mdb` (which has zero data tables) turned out to hold *all* of the insurance line's Reports/Forms/Queries; `SEGUROS 16_be.mdb` is confirmed pure data storage. The real queries' full SQL text is the best available record of actual business logic (billing math, renewal batching) — worth reading before reimplementing any given feature from scratch. Raw JSON checked in at `migration/objects.json`.
- `C:\Users\ricar\Downloads\jorgecuadros_app.sql` and `jorgecuadros_app (1).sql` — MySQL dumps of the customer-portal's **tracking/analytics** sidecar DB (`browse_tracking`, `devices` push-tokens, `task_tracking`) from `mysql.freakma.com`. **Not** the portal's real data DB — see open item #1 below. - `jorgecuadros_app.sql` / `jorgecuadros_app (1).sql` (on the old machine) — MySQL dumps of the portal's **tracking/analytics** sidecar DB (`browse_tracking`, `devices` push-tokens, `task_tracking`). **Not** the portal's real data DB; superseded by `utility_dbo.sql` above.
**Customer-facing portal (out of scope to rebuild, but the sync target):**
- `~/PhpstormProjects/my-jorgecuadros-web` — PHP/`mysqli`, ~397 files, core in `scripts/functions.php`. Reads/writes `utility_dbo`.
**Old internal app (reference-only, not being built on):** **Old internal app (reference-only, not being built on):**
- `C:\Users\ricar\Downloads\Jorge\jorgecuadros-intra-webapp` — PHP, MySQL (`webapp_jorgecuadros`). Schema at `db\webapp_jorgecuadros.sql` is a useful reference for field mappings/business logic. Code itself is not being reused — see §4. - `jorgecuadros-intra-webapp` (on the old machine) — PHP, MySQL (`webapp_jorgecuadros`). Its `db/webapp_jorgecuadros.sql` is a useful reference for field mappings/business logic. Code itself is not reused — see §4.
**New platform (the actual deliverable, in progress):** **New platform (the actual deliverable):**
- `C:\Users\ricar\Downloads\Jorge\jorgecuadros-platform` — the new repo. Not yet a git repository (no commits made this session — user hasn't asked for any yet). - `~/WebstormProjects/jorgecuadros-platform` — the repo. **Is** a git repo, branch `master`, 21 commits, remote `git.mancinas.io/rmancinas/jorgecuadros-platform`.
**The plan document:** **The plan document:**
- `C:\Users\ricar\.claude\plans\logical-yawning-tome.md` — full architecture, source-data inventory per table, target data model, migration strategy, infrastructure/sync design, open decisions, build sequencing. **This is the source of truth for the design** — this RESUME.md summarizes it plus session/environment state that isn't in the plan itself. - [`PLAN.md`](PLAN.md) in this repo — full architecture, source-data inventory per table, target data model, migration strategy, infrastructure/sync design, locked decisions, build sequencing. **This is now the source of truth for the design** (the original `~/.claude/plans/logical-yawning-tome.md` lived on the old Windows machine). This RESUME.md is the "what happened / what's next" companion.
**Ephemeral / will NOT persist across sessions** (session-scoped temp directory): **Staged data (gitignored, regenerable):**
- `C:\Users\ricar\AppData\Local\Temp\claude\...\scratchpad\` — contained exploratory helper scripts (`dump_schema.py`, `summarize_schema.py`, `test_decode_fix.py`) and the schema JSON/txt dumps used during initial analysis, plus a Parquet staging output from one run of the migration script. **None of this needs to be recovered** — the real, permanent versions of the useful scripts are in `jorgecuadros-platform/migration/`, and the Parquet output can be regenerated in under 2 minutes by rerunning `load_staging.py` (see §6). - `migration/output/stg_utilities|stg_seguros|stg_scothia/*.parquet` — regenerate in ~2 min with `load_staging.py --output-dir ./output`. Every transform step reads from here.
## 3. Key decisions made this session ## 3. Key decisions (locked — see `PLAN.md` → "Decisions (locked)")
| Decision | Answer | Why | | Decision | Answer | Why |
|---|---|---| |---|---|---|
@@ -65,9 +74,10 @@ infrastructure decisions below.
| Infrastructure | Internal server (private) + new VPS (Tailscale-linked) running a MySQL replica | Internal server has no inbound internet exposure; shared hosting can't be a replication target; a VPS you control can be both a real replication node and internet-reachable for the portal | | Infrastructure | Internal server (private) + new VPS (Tailscale-linked) running a MySQL replica | Internal server has no inbound internet exposure; shared hosting can't be a replication target; a VPS you control can be both a real replication node and internet-reachable for the portal |
| Auth mechanism | Session-based (Passport + `express-session`), Argon2 password hashing | Implemented already — see §5 | | Auth mechanism | Session-based (Passport + `express-session`), Argon2 password hashing | Implemented already — see §5 |
## 4. What was actually built and verified this session ## 4. What is built and verified
Everything below was **run and confirmed working**, not just written: Everything below was **run and confirmed working**, not just written. §8 carries the
per-module detail and the running status; this section is the structural tour.
### 4.1 Repo scaffold ### 4.1 Repo scaffold
- `jorgecuadros-platform/` — npm workspaces (`apps/*`, `packages/*`) - `jorgecuadros-platform/` — npm workspaces (`apps/*`, `packages/*`)
@@ -94,7 +104,7 @@ Regenerate the client any time with:
cd jorgecuadros-platform cd jorgecuadros-platform
DATABASE_URL="mysql://user:pass@localhost:3306/placeholder" npx prisma generate --schema=packages/database/prisma/schema.prisma DATABASE_URL="mysql://user:pass@localhost:3306/placeholder" npx prisma generate --schema=packages/database/prisma/schema.prisma
``` ```
(A real `DATABASE_URL` isn't needed for `generate`/`validate`, just a syntactically valid one — no live DB was available in this session, see §7.) (A real `DATABASE_URL` isn't needed for `generate`/`validate`, just a syntactically valid one. A live dev DB *is* available now — see §7 — so `prisma db push` works too.)
### 4.3 Docker Compose / Dockerfiles ### 4.3 Docker Compose / Dockerfiles
- `docker-compose.yml``mysql:8.4` + `api` + `web` services, healthchecked. - `docker-compose.yml``mysql:8.4` + `api` + `web` services, healthchecked.
@@ -103,17 +113,20 @@ DATABASE_URL="mysql://user:pass@localhost:3306/placeholder" npx prisma generate
- **Not run** — this environment has no Docker installed (`docker --version` fails). Untested beyond visual review; verify on a machine with Docker before relying on it. - **Not run** — this environment has no Docker installed (`docker --version` fails). Untested beyond visual review; verify on a machine with Docker before relying on it.
### 4.4 Migration pipeline (`migration/`) — run end-to-end against real data ### 4.4 Migration pipeline (`migration/`) — run end-to-end against real data
- `config.py` — manifest of the 3 Access source files (paths + per-source exclude lists for confirmed-scratch tables, with reasoning in comments) - `config.py` — manifest of the Access source files (`SOURCE_ROOT` + per-source exclude lists for confirmed-scratch tables, with reasoning in comments)
- `extract.py`connects via `pyodbc` + the Windows Access ODBC driver (`Microsoft Access Driver (*.mdb, *.accdb)`, 64-bit — confirmed installed on this machine). Two real bugs found and fixed here: - `extract.py`shells out to **mdbtools** (`mdb-tables` / `mdb-export`, Homebrew). Rewritten from the original `pyodbc` + Windows Access ODBC version during the macOS move; public interface (`connect`/`list_tables`/`read_table`) unchanged. mdbtools also sidesteps both bugs the pyodbc path needed workarounds for: it reads accented-column tables (`PROPANO`, `FALTANTES AGUA`, `TIT`) cleanly instead of hitting a UTF-16 decode error, and it doesn't abort a whole table on `MULT`'s corrupted row.
1. `cursor.columns()` hits a UTF-16 decode bug on some tables (confirmed: `PROPANO`, `FALTANTES AGUA`, `TIT`) — fixed by reading column names from `cursor.description` after a `SELECT *` instead. - What mdbtools **cannot** do is read Forms/Reports/Queries. Those were already captured on Windows via DAO COM and are frozen in `migration/objects.json` + `docs/LEGACY_DATABASES_OBJECTS.md` — nothing is lost, but they can't be re-extracted on this machine.
2. `cursor.fetchall()` aborts an entire table on the first corrupted row — confirmed on `MULT` (Jet/ACE-level "Record is deleted" error, HY109). Fixed by fetching row-by-row in a try/except, skipping and logging just the bad row. Recovered 763 of 764 rows in `MULT`. **Verified the cursor advances correctly and doesn't infinite-loop on the bad row** before trusting this for a full run. - `load_staging.py` — dumps every non-excluded table into either Parquet (`--output-dir`, no DB needed) or MySQL (`--database-url`, one database per source: `stg_utilities`/`stg_seguros`/`stg_scothia`). 82 tables staged, zero unhandled errors.
- `load_staging.py` — dumps every non-excluded table into either Parquet (`--output-dir`, no DB needed) or MySQL (`--database-url`, one database per source: `stg_utilities`/`stg_seguros`/`stg_scothia`). **Actually run** in Parquet mode against all three Access files: **82 tables staged successfully, zero unhandled errors**, `MULT`'s corrupted row correctly skipped and logged. - `reconcile.py``RECONCILIATION.md` — the duplicate/distinct pass (step 2). See §8 step 3.
- `requirements.txt``pyodbc`, `pandas`, `pyarrow`, `sqlalchemy`, `pymysql`. - `transform_*.py`, `prune_empty_customers.py`, `blob_extract.py` — steps 34, all idempotent (truncate + rebuild).
- `run_all.py`**the entry point.** Runs every step in dependency order. See the ⚠️ in §7 for why you should never run a single transform on its own.
- `dbenv.py``--env <name>` reads `deploy/.env.<name>` for the target DB.
- `requirements.txt``pandas`, `pyarrow`, `sqlalchemy`, `pymysql`, `boto3` (no `pyodbc` — that was the Windows path).
To rerun (from `jorgecuadros-platform/migration`, after `pip install -r requirements.txt`): To rerun (from `migration/`, venv at `migration/.venv`):
```bash ```bash
python load_staging.py --output-dir ./output # Parquet, no DB needed — always works ./.venv/bin/python load_staging.py --output-dir ./output # re-extract from Access (needs mdbtools + the source files)
python load_staging.py --database-url mysql+pymysql://user:pass@host:3306/ # loads into real MySQL once available ./.venv/bin/python run_all.py --env dev # full transform+load; add --stage to re-extract first
``` ```
## 5. Infrastructure & sync architecture (designed, not yet built) ## 5. Infrastructure & sync architecture (designed, not yet built)
@@ -124,22 +137,47 @@ python load_staging.py --database-url mysql+pymysql://user:pass@host:3306/ # l
- **Internal → VPS:** one-way native MySQL replication (binlog/GTID) for the subset of data the portal needs to read (statements, balances, customer profile). Internal-only tables (staff notes, adjuster info, activity logs) are deliberately excluded from what replicates. - **Internal → VPS:** one-way native MySQL replication (binlog/GTID) for the subset of data the portal needs to read (statements, balances, customer profile). Internal-only tables (staff notes, adjuster info, activity logs) are deliberately excluded from what replicates.
- **VPS → Internal:** the portal also *writes* (payment submissions, propane orders) — one-way replication can't carry that back, and multi-master MySQL replication was deliberately ruled out as too fragile for this system's size. Instead: unreplicated "inbox" tables on the VPS (`payment_submissions`, `propane_order_requests`) that the portal writes to directly, polled every 15 min by a worker on the internal server (over Tailscale) that turns new rows into real records. - **VPS → Internal:** the portal also *writes* (payment submissions, propane orders) — one-way replication can't carry that back, and multi-master MySQL replication was deliberately ruled out as too fragile for this system's size. Instead: unreplicated "inbox" tables on the VPS (`payment_submissions`, `propane_order_requests`) that the portal writes to directly, polled every 15 min by a worker on the internal server (over Tailscale) that turns new rows into real records.
## 6. Open items — need input/access before certain next steps can proceed ## 6. Open items
1. **`utility_dbo` schema is still unknown.** This is the customer portal's actual live data database (referenced in the old app via `getExternalDBConnection()` at `mysql.freakma.com`, used there only for `email_alert_log`, but the mobile app almost certainly reads/writes statements, payments, and propane orders directly against it). The two SQL dumps provided (`jorgecuadros_app.sql`, `jorgecuadros_app (1).sql`) turned out to be a **separate** analytics/tracking database, not this one. When asked, the user pointed back to `SEGUROS 16_be.mdb` — worth revisiting; it's possible the intent was "the source data ultimately comes from the Access files" rather than "here is utility_dbo's schema." **Without the real `utility_dbo` schema, the sync worker's exact target tables/columns for the inbox pattern can't be finalized.** Ask for an export or read-only credentials, same as how the Access files and tracking-DB dumps were provided. **Resolved since this section was first written** (kept as a pointer, not a to-do):
2. **VPS not yet provisioned** — provider (Hetzner vs DigitalOcean), size, and Tailscale/MySQL setup on it are pending. Ops task, not something done in this session. `utility_dbo` schema (full dump on disk), CI/CD (Gitea Actions), i18n (Spanish-first), and
3. **CI/hosting** — keep Jenkins + `git.freakma.com`, or move to GitHub Actions if the new repo lives elsewhere? the reconciliation pass (done, then corrected) are all closed. See §3 and §8.
4. **i18n** — nearly all source data and, presumably, staff usage is in Spanish; old app's code/UI was English-labeled. Confirm Spanish-first / bilingual / English before frontend work goes deep.
5. **Reconciliation pass not started** (migration plan step 2) — the near-duplicate snapshot tables (`EFECTIVO`/`EFECTIVO FM3`/`EFECTIVO_BACKUP`, `FEE ANUAL`/`datos2`/`fee15`/`billing`, `DATGRAL` vs `COBRO3`) need a diff/dedupe pass against the staged data before any transform-and-load into the real schema. This is explicitly *not* a "guess the rule up front" thing — the plan calls for writing SQL against the staged Parquet/MySQL data to see what's actually duplicate vs. distinct.
## 7. Environment notes (this machine, in case it matters for reproducing) **Still open:**
1. **VPS not yet provisioned** — provider (Hetzner vs DigitalOcean), size, Tailscale + MySQL
replica setup. Pure ops task; the design is settled (§5). This is the only genuinely
blocking item left on the roadmap.
2. **Sync worker not built** — unblocked now that `utility_dbo` and the portal code are on
disk, but depends on the VPS existing. Portal write points to poll: `peticion_gas`,
PayPal payments, `notifications_settings`, `verification_codes`.
3. **Old external-DB credential** — the old repo's `dbConnection.php` has a hardcoded
plaintext MySQL password committed to git history. Not carried into the new platform,
but rotate it regardless; it is already exposed.
4. **`bank_transactions.categoryId` is null on all 22354 rows** — the concept→ramo
classifier was deferred. Needed before any "insurance vs utilities vs trust" split of
the office's own bank activity.
5. **`TRASPASOS PAYPAL` is a clearing account, not a customer** — carries -7.03M MXN over
309 movements and therefore tops the adeudo worklist. Deliberately not special-cased in
code; needs a business decision on how to model it.
- Windows, PowerShell primary, Git Bash also available. ## 7. Environment notes (current macOS machine)
- Node v25.2.0, npm 11.6.2 — no pnpm, yarn is present but npm workspaces were used throughout.
- Python 3.9 (`C:\Python39`), `pip install`'d this session: `pyodbc`, `pandas`, `pyarrow`, `psycopg2` (installed but no longer used after the MySQL switch — harmless to leave or remove). - macOS (Darwin 25.5.0), zsh. Node v22.23.0. Python 3.14.6 in `migration/.venv`. Homebrew, Docker, MySQL/MariaDB client all present.
- MS Access ODBC driver confirmed installed: **64-bit** `Microsoft Access Driver (*.mdb, *.accdb)` (matches 64-bit Python — this pairing matters, a 32/64-bit mismatch would break `pyodbc.connect`). - **mdbtools** installed via Homebrew — the extraction toolchain. No Access ODBC driver (and none needed).
- **No Docker, no local MySQL, no local Postgres** on this machine — `docker-compose.yml` and the MySQL-target mode of `load_staging.py` are written but unexecuted here. Test both on whatever machine ends up running this for real. - **`npm` is pnpm-aliased**, and pnpm ignores the `workspaces` field. Consequences:
- Old repo's `dbConnection.php` has a **hardcoded plaintext MySQL password** for the external DB connection, committed to git history. Not carried forward into the new platform, but worth rotating that credential regardless since it's already exposed in the old repo's history. - there is **no root `node_modules/.bin`**. Binaries live per-app: `apps/api/node_modules/.bin/nest`, `apps/web/node_modules/.bin/next`.
- Prisma CLI is run as `npx prisma@5`.
- **Dev servers** (both must be up to use the UI):
- API `cd apps/api && ./node_modules/.bin/nest start --watch``:3001`
- Web `cd apps/web && ./node_modules/.bin/next dev``:3000`
- Dev login: `admin@jorgecuadros.local`, password from `apps/api/scripts/seed-user.mjs` (`SEED_PASSWORD` env overrides the default).
- **Dev DB**: `192.168.4.212:3307` (cubex Swarm stack `jorgecuadros-dev-db`). Credentials in gitignored `deploy/.env.dev`. **MinIO** for documents: `192.168.4.212:9100`, bucket `jorgecuadros-documents`.
**Traps worth knowing before you lose an hour to one:**
- ⚠️ **Never run a single `transform_*.py` on its own — use `run_all.py`.** Each step truncates what it owns, so a lone run orphans everything downstream. `prune_empty_customers.py` must re-run after any ledger change, and `blob_extract.py` must follow properties + policies or the uploaded MinIO objects end up with no rows pointing at them.
- ⚠️ **Never run `next build` while `next dev` is running** — they share `.next` and the dev server starts serving blank white pages. Recovery: kill the dev server, `rm -rf apps/web/.next`, restart.
- ⚠️ **`mdb-export` formats numerics per Access column type** (`5000` from one table, `27000.0000` from another). Never string-compare staged Parquet numerics across two tables — canonicalize first. This exact trap produced a wrong, *locked* migration decision that shipped 12386 duplicate rows into the ledger (§8 step 4).
- Shell on this machine: `head` is aliased to an HTTP HEAD tool — use `/usr/bin/head`. `grep --include=*.md` trips zsh globbing — quote the pattern.
## 8. Plan locked — next actions ## 8. Plan locked — next actions
@@ -151,24 +189,36 @@ python load_staging.py --database-url mysql+pymysql://user:pass@host:3306/ # l
- `utility_dbo`: **resolved** — full dump (`utility_dbo.sql`, 1.3 GB, 55 tables) and the - `utility_dbo`: **resolved** — full dump (`utility_dbo.sql`, 1.3 GB, 55 tables) and the
portal codebase (`~/PhpstormProjects/my-jorgecuadros-web`) are both on disk. portal codebase (`~/PhpstormProjects/my-jorgecuadros-web`) are both on disk.
**Environment: moved Windows → macOS.** Sources now at `~/Downloads/JorgeCuadros-Legacy/` **Environment: moved Windows → macOS** (2026-07-22). See §7 for the current machine.
(all four files). This machine has Docker, MySQL/MariaDB client, Node 22, Python 3.14,
Homebrew. No Access ODBC driver, `node_modules` not installed, staging Parquet not present.
**Execution queue (in order):** **Execution queue.** Steps 16 below are **done**; they are kept because each carries the
1. **Port the extraction layer to mdbtools.** Rewrite `migration/extract.py` to shell out to data findings and corrections that came out of doing it. Skip to the ⏭ marker at the end
for what's actually next.
1. ~~**Port the extraction layer to mdbtools.**~~ **DONE.** Rewrote `migration/extract.py` to shell out to
`mdb-tables`/`mdb-export` instead of `pyodbc`. Keep the same public interface `mdb-tables`/`mdb-export` instead of `pyodbc`. Keep the same public interface
(`connect`/`list_tables`/`read_table`) so `load_staging.py` and `config.py` are unchanged (`connect`/`list_tables`/`read_table`) so `load_staging.py` and `config.py` are unchanged
beyond the already-fixed `SOURCE_ROOT`. Carry over the two hard-won fixes conceptually: beyond the already-fixed `SOURCE_ROOT`. Carry over the two hard-won fixes conceptually:
accented-column tables (mdbtools reads `PROPANO` cleanly — verified) and the corrupted `MULT` accented-column tables (mdbtools reads `PROPANO` cleanly — verified) and the corrupted `MULT`
row (mdb-export's `-b` / error handling; confirm the bad row is skipped, not fatal). row (mdb-export's `-b` / error handling; confirm the bad row is skipped, not fatal).
2. **Re-run staging** (`python load_staging.py --output-dir ./output`) to regenerate the staged 2. ~~**Re-run staging**~~ **DONE** — staged Parquet regenerated on this machine
data on this machine, then load into a local MySQL (`docker compose up mysql`) for SQL reconciliation. (`load_staging.py --output-dir ./output`), 82 tables.
3. **Reconciliation pass** (plan step 2) — **DONE** (`migration/reconcile.py``RECONCILIATION.md`). 3. **Reconciliation pass** (plan step 2) — **DONE** (`migration/reconcile.py``RECONCILIATION.md`),
Overturned all three "duplicate" assumptions: EFECTIVO/BACKUP are near-disjoint ledgers **and corrected 2026-07-22.** Current verdicts:
(folio collides; migrate both), the billing tables are disjoint period runs (union all, no - `EFECTIVO_BACKUP` is a **stale backup copy of `EFECTIVO`** — 12386 of its 12387 rows are
de-dup), and COBRO3 is a charge batch not a customer snapshot (DATGRAL is sole master). The verbatim duplicates (customer + timestamp-to-the-second + amount + concept text), leaving
decided union/de-dup rules are in `PLAN.md` migration step 2. 1 new row. Load EFECTIVO in full, de-dup BACKUP on the business key. **Never de-dup on
`folio`** — it is per-table sequential and collides (12363 shared numbers, 12204 of them
on different payments).
- The billing tables (`datos2`/`FEE ANUAL`/`fee15`) are disjoint period runs — union all,
no de-dup.
- `COBRO3` is a charge batch, not a customer snapshot — `DATGRAL` is the sole master.
⚠️ **This file and `PLAN.md` previously said the opposite about EFECTIVO** ("near-disjoint
ledgers, migrate both"). That was a bug, not a finding — see the Shared ledger entry in
step 4 below for the root cause and the fix. If you read a doc, comment, or commit message
from before 2026-07-22 that says "migrate both, no folio de-dup", it is stale.
The authoritative rules live in `PLAN.md` migration step 2.
4. **Transform + load** (plan step 3) — IN PROGRESS. 4. **Transform + load** (plan step 3) — IN PROGRESS.
- **Customers — DONE** (`migration/transform_customers.py`). Loaded into the dev DB: 1682 - **Customers — DONE** (`migration/transform_customers.py`). Loaded into the dev DB: 1682
customers (1172 utilities master + 510 insurance-only), 2242 legacy refs (all traceable), customers (1172 utilities master + 510 insurance-only), 2242 legacy refs (all traceable),
@@ -185,12 +235,23 @@ Homebrew. No Access ODBC driver, `node_modules` not installed, staging Parquet n
adjusters. Unmodeled coverage columns preserved verbatim in `coveragesJson`. Verified a adjusters. Unmodeled coverage columns preserved verbatim in `coveragesJson`. Verified a
unified customer (EARWOOD, DAVID) carrying both a utility property+services and 2 MULT unified customer (EARWOOD, DAVID) carrying both a utility property+services and 2 MULT
policies — the cross-line customer view works at the data layer. policies — the cross-line customer view works at the data layer.
- **Shared ledger — DONE** (`migration/transform_transactions.py`): 45861 transactions - **Shared ledger — DONE** (`migration/transform_transactions.py`): **33475** transactions
(UTILITY 45566 / INSURANCE 295, 0 orphans) unioning both EFECTIVO tables (13696+12386, (UTILITY 33180 / INSURANCE 295, 0 orphans) unioning EFECTIVO (13695) **plus only the 1
no folio de-dup), all three billing tables (datos2/FEE ANUAL/fee15), the FM3 fee stream business-key-new row from EFECTIVO_BACKUP**, all three billing tables
(amount=fee+tax+multa), IVA 2015 (nominal date), and insurance EFECTIVO — per the (datos2/FEE ANUAL/fee15), the FM3 fee stream (amount=fee+tax+multa), IVA 2015 (nominal
reconciliation rules; plus 79 `type_transactions` (EN/ES) and 2301 `exchange_rates`. date), and insurance EFECTIVO; plus 79 `type_transactions` and 2301 `exchange_rates`.
Skipped 22 no-customer + 303 no-date (mostly datos2 blanks). Skipped 22 no-customer + 272 no-date + **12417 EFECTIVO_BACKUP duplicates**.
**Corrected 2026-07-22 — this used to load 45861 rows.** `reconcile.py` had string-compared
`monto`, which mdb-export serializes at a different precision per Access column type
(`5000` vs `27000.0000`), so it saw 2 overlapping rows instead of 12386 and ruled
EFECTIVO_BACKUP an independent ledger. It is a stale backup copy: 12386 of its 12387 rows
match an EFECTIVO row on customer + timestamp-to-the-second + amount + concept text. The
ledger was double-counting those payments, roughly doubling every customer's historical
receipt total. Both `reconcile.py` (canonicalizes numeric key columns now) and
`transform_transactions.py` (de-dups on the business key, never on `folio` — folio
collides) are fixed, and `run_all.py --env dev` has been re-run end to end.
**Lesson for any future reconciliation: never compare mdb-export output as raw strings
across two tables — canonicalize numerics first.**
- **Bank register — DONE** (`migration/transform_bank.py`): 22354 `bank_transactions` from - **Bank register — DONE** (`migration/transform_bank.py`): 22354 `bank_transactions` from
SCOTHIA DATOS I/E as signed amounts (income +, expense -; net +899,375.77), 66 SCOTHIA DATOS I/E as signed amounts (income +, expense -; net +899,375.77), 66
`business_line_categories`. No customer FK; categoryId left null (concept->ramo classifier `business_line_categories`. No customer FK; categoryId left null (concept->ramo classifier
@@ -233,21 +294,58 @@ Homebrew. No Access ODBC driver, `node_modules` not installed, staging Parquet n
geographic filter. `PropertyService.notes` means something different per kind geographic filter. `PropertyService.notes` means something different per kind
(municipality / CFE PAR-IMPAR cycle / gas supply type), so the UI labels it per kind. (municipality / CFE PAR-IMPAR cycle / gas supply type), so the UI labels it per kind.
240 of 1519 properties have no service rows at all — surfaced as its own bucket. 240 of 1519 properties have no service rows at all — surfaced as its own bucket.
- **NEXT:** plan step 6 — the shared billing/statements view across both business lines. - **Billing / statements module (plan step 6) — DONE**: `apps/api/src/billing/`
- Full pipeline reproducible in one command: `run_all.py --env <env>` runs customers -> (`GET /billing` movement browser with search over customer / referencia / cheque /
properties -> policies -> transactions -> bank in order (all idempotent); add `--stage` concepto / periodo, filters for línea, moneda, cargo-vs-abono, concepto (typeId), origin
to re-extract from the Access files first. Verified end-to-end against dev. table and a from/to date range, 5 sorts, and **totals for the whole filtered set**;
5b. **Infra done:** dev MySQL deployed to the cubex Swarm via Portainer API as stack `GET /billing/balances` per-customer balances with owing/credit/settled buckets and 4
`jorgecuadros-dev-db` (MySQL 8.4, `192.168.4.212:3307`, node `cubex` labeled sorts; `/billing/stats`, `/billing/facets`, `/billing/customers/:id`) + web
`jorgecuadros_db=true`); Prisma schema pushed (26 tables). Stack file: `/estado-cuenta` (two tabs: "Saldos por cliente" worklist and "Movimientos" ledger) and
`deploy/jorgecuadros-db.stack.yml` (same file deploys prod as `jorgecuadros-prod-db` :3306). `/estado-cuenta/[id]` (the statement: balance per currency, the same balance split by
Creds in gitignored `deploy/.env.dev`. NOTE: machine `npm` is pnpm-aliased and pnpm ignores business line, cargos por concepto with proportional bars, and the full movement table
the `workspaces` field — full workspace install needs `pnpm-workspace.yaml` or real npm; for with a running balance). Cross-links from the customer and property detail pages.
now Prisma CLI is run via `npx prisma@5`. **Data findings:**
5. **Customer module** in `apps/api`/`apps/web` (list/search/detail) — first real feature, (a) `transactions.amount` is a *signed* ledger — every charge type is negative without
Spanish-first UI. Run `npm install` at repo root first (node_modules absent here). exception (WATER 3115/3117, ELECTRIC 2191/2191, PROPERTY TAXES 926/926, TRUST FEE
6. **Sync design finalization** — now unblocked: map the internal→VPS replicated subset and the 188/188) and every deposit type positive (CHECK/CASH DEPOSIT, PAYPAL, all of EFECTIVO),
VPS→internal inbox tables against the real `utility_dbo` schema and the portal's read/write so `SUM(amount)` is the balance and negative = the customer owes.
points in `my-jorgecuadros-web` (`peticion_gas`, PayPal payments, `notifications_settings`). (b) **Currency is not summable.** 912 of the 1269 customers with a ledger move in both
MXN and USD; the charge side (datos2/FEE ANUAL/fee15) is MXN-only while receipts arrive
in both, and no per-movement exchange rate was ever stored. Every figure in the module is
per currency; the balance filter/sort takes a currency argument rather than collapsing.
(c) `type_transactions.nameEs` is **entirely null** — the legacy `TYPE OF TRX` table has
an `ESPAÑOL` column but all 79 rows are empty, so the API can only return English names.
`labels.ts:TX_TYPE_LABELS` supplies Spanish for the real service/payment categories; the
rest of the 79 "types" are payee names (LORETO GONZALEZ, ALBERCAS VALLARTA…) that fall
through untranslated, which is correct.
(d) The biggest debtor by far is **"TRASPASOS PAYPAL"** (-7.03M MXN over 309 movements) —
a house/clearing account, not a person. Left in rather than special-cased, but it will
head the adeudo worklist until someone decides how to model it.
- Full pipeline reproducible in one command: `run_all.py --env <env>` runs customers →
properties → policies → transactions → prune → bank → blobs in order (all idempotent);
add `--stage` to re-extract from the Access files first. Verified end-to-end against dev.
Only genuinely-pending item is **VPS provisioning** (ops task — provider/size/Tailscale+MySQL). 5. **Infra****DONE.** Dev MySQL deployed to the cubex Swarm via the Portainer API as stack
`jorgecuadros-dev-db` (MySQL 8.4, `192.168.4.212:3307`, node `cubex` labeled
`jorgecuadros_db=true`); Prisma schema pushed (26 tables). Stack file
`deploy/jorgecuadros-db.stack.yml` deploys prod from the same file as
`jorgecuadros-prod-db` on :3306. MinIO for documents deployed as `jorgecuadros-dev-minio`.
6. **Staff web UI****DONE** for all four modules (§8 step 4: clientes, polizas, servicios,
estado-cuenta). Spanish-first, session-cookie auth against the API, verified against real
migrated data.
---
**NEXT — where to pick up:**
- **Plan step 7: bank register module.** SCOTHIA data is already migrated (22354
`bank_transactions` + 66 `business_line_categories`) and has **no customer FK**, so it is
self-contained and low-risk — API + `/banco` browser only. Blocked on nothing.
Ties into open item §6.4 (`categoryId` is null on every row).
- **Plan step 89: VPS + sync worker.** Blocked on VPS provisioning (§6.1) — the only real
external dependency left.
- **Plan step 10: reports / email campaigns / admin.**
- **Uncommitted work:** the billing module and the ledger de-dup fix are written, built and
verified but **not committed**`git status` is dirty at the time of writing. The browser
visual pass on `/estado-cuenta` was also never completed (it needs an interactive login).
+2
View File
@@ -6,6 +6,7 @@ import { AuthModule } from "./auth/auth.module";
import { CustomersModule } from "./customers/customers.module"; import { CustomersModule } from "./customers/customers.module";
import { PoliciesModule } from "./policies/policies.module"; import { PoliciesModule } from "./policies/policies.module";
import { PropertiesModule } from "./properties/properties.module"; import { PropertiesModule } from "./properties/properties.module";
import { BillingModule } from "./billing/billing.module";
import { AppController } from "./app.controller"; import { AppController } from "./app.controller";
@Module({ @Module({
@@ -17,6 +18,7 @@ import { AppController } from "./app.controller";
CustomersModule, CustomersModule,
PoliciesModule, PoliciesModule,
PropertiesModule, PropertiesModule,
BillingModule,
], ],
controllers: [AppController], controllers: [AppController],
}) })
+116
View File
@@ -0,0 +1,116 @@
import { Controller, Get, Param, Query, UseGuards } from "@nestjs/common";
import { TransactionDomain } from "@jorgecuadros/database";
import { AuthenticatedGuard } from "../auth/authenticated.guard";
import {
BalanceFilter,
BalanceSort,
BillingService,
LedgerCurrency,
LedgerDirection,
MovementSort,
} from "./billing.service";
const DOMAINS: TransactionDomain[] = ["UTILITY", "INSURANCE", "TRUST"];
const CURRENCIES: LedgerCurrency[] = ["MXN", "USD"];
const DIRECTIONS: LedgerDirection[] = ["charge", "credit"];
const BALANCES: BalanceFilter[] = ["all", "owing", "credit", "settled"];
const MOVEMENT_SORTS: MovementSort[] = [
"date_desc",
"date_asc",
"amount_desc",
"amount_asc",
"customer",
];
const BALANCE_SORTS: BalanceSort[] = [
"owing_desc",
"credit_desc",
"recent",
"customer",
];
function one<T>(allowed: T[], value: string | undefined): T | undefined {
return allowed.includes(value as T) ? (value as T) : undefined;
}
/** A `YYYY-MM-DD` bound; anything unparseable is treated as absent. */
function parseDate(v: string | undefined, endOfDay = false): Date | undefined {
if (!v) return undefined;
const d = new Date(endOfDay ? `${v}T23:59:59.999Z` : `${v}T00:00:00.000Z`);
return Number.isNaN(d.getTime()) ? undefined : d;
}
@UseGuards(AuthenticatedGuard)
@Controller("billing")
export class BillingController {
constructor(private readonly billing: BillingService) {}
@Get("stats")
stats() {
return this.billing.stats();
}
@Get("facets")
facets() {
return this.billing.facets();
}
/** Per-customer balances — the receivables worklist. */
@Get("balances")
balances(
@Query("query") query?: string,
@Query("page") page?: string,
@Query("pageSize") pageSize?: string,
@Query("currency") currency?: string,
@Query("balance") balance?: string,
@Query("domain") domain?: string,
@Query("sort") sort?: string,
) {
return this.billing.balances({
query,
page: Math.max(1, Number(page) || 1),
pageSize: Math.min(100, Math.max(1, Number(pageSize) || 25)),
currency: one(CURRENCIES, currency) ?? "MXN",
balance: one(BALANCES, balance) ?? "all",
domain: one(DOMAINS, domain),
sort: one(BALANCE_SORTS, sort) ?? "owing_desc",
});
}
/** One customer's full statement across both business lines. */
@Get("customers/:id")
statement(@Param("id") id: string) {
return this.billing.statement(id);
}
/** Cross-customer movement browser. */
@Get()
movements(
@Query("query") query?: string,
@Query("page") page?: string,
@Query("pageSize") pageSize?: string,
@Query("domain") domain?: string,
@Query("currency") currency?: string,
@Query("direction") direction?: string,
@Query("typeId") typeId?: string,
@Query("source") source?: string,
@Query("customerId") customerId?: string,
@Query("from") from?: string,
@Query("to") to?: string,
@Query("sort") sort?: string,
) {
return this.billing.movements({
query,
page: Math.max(1, Number(page) || 1),
pageSize: Math.min(100, Math.max(1, Number(pageSize) || 25)),
domain: one(DOMAINS, domain),
currency: one(CURRENCIES, currency),
direction: one(DIRECTIONS, direction),
typeId: typeId || undefined,
source: source || undefined,
customerId: customerId || undefined,
from: parseDate(from),
to: parseDate(to, true),
sort: one(MOVEMENT_SORTS, sort) ?? "date_desc",
});
}
}
+9
View File
@@ -0,0 +1,9 @@
import { Module } from "@nestjs/common";
import { BillingController } from "./billing.controller";
import { BillingService } from "./billing.service";
@Module({
controllers: [BillingController],
providers: [BillingService],
})
export class BillingModule {}
+725
View File
@@ -0,0 +1,725 @@
import { Injectable, NotFoundException } from "@nestjs/common";
import { Prisma, TransactionDomain } from "@jorgecuadros/database";
import { PrismaService } from "../prisma/prisma.service";
/**
* Shared billing / statements module — plan step 6.
*
* SIGN CONVENTION (established from the migrated data, not assumed):
* `transactions.amount` is a *signed* ledger amount.
* - negative = cargo (a charge: a utility bill, predial, trust fee, HOA due,
* insurance premium the office paid or billed on the customer's behalf).
* Every legacy `type_of_trx` on the charge side is negative without
* exception — WATER 3115/3117 negative, ELECTRIC 2191/2191, PROPERTY TAXES
* 926/926, TRUST FEE 188/188.
* - positive = abono (a credit: CHECK DEPOSIT, CASH DEPOSIT, PAYPAL, and
* every `EFECTIVO` cash receipt).
* So `SUM(amount)` is the balance: negative means the customer owes the office,
* positive means the customer is in credit.
*
* CURRENCY IS NOT SUMMABLE. 912 of the 1269 customers with a ledger have
* movements in both MXN and USD, and the charge side (`datos2`/`FEE ANUAL`/
* `fee15`) is MXN-only while receipts arrive in both. Applying a historical
* exchange rate to a 20-year ledger to produce one number would invent a figure
* the source data never had, so every total in this module is reported *per
* currency* and never collapsed. Balance filters and sorts therefore operate on
* one caller-chosen currency at a time.
*/
/** Which side of the ledger a movement is on. */
export type LedgerDirection = "charge" | "credit";
/** Balance buckets for the receivables worklist, on the selected currency. */
export type BalanceFilter = "all" | "owing" | "credit" | "settled";
export type MovementSort =
| "date_desc"
| "date_asc"
| "amount_desc"
| "amount_asc"
| "customer";
export type BalanceSort = "owing_desc" | "credit_desc" | "recent" | "customer";
export type LedgerCurrency = "MXN" | "USD";
export interface MovementParams {
query?: string;
page: number;
pageSize: number;
domain?: TransactionDomain;
currency?: LedgerCurrency;
direction?: LedgerDirection;
typeId?: string;
source?: string;
customerId?: string;
/** Inclusive ISO date bounds on `transactionDate`. */
from?: Date;
to?: Date;
sort: MovementSort;
}
export interface BalanceParams {
query?: string;
page: number;
pageSize: number;
currency: LedgerCurrency;
balance: BalanceFilter;
/** Restricts the whole balance to one business line. */
domain?: TransactionDomain;
sort: BalanceSort;
}
/** Raw shape of the per-customer balance aggregate. */
interface BalanceRow {
id: string;
name: string;
nameSource: string | null;
nameMissing: number;
city: string | null;
state: string | null;
movements: bigint | number | string;
balanceMxn: Prisma.Decimal | null;
balanceUsd: Prisma.Decimal | null;
chargesMxn: Prisma.Decimal | null;
creditsMxn: Prisma.Decimal | null;
chargesUsd: Prisma.Decimal | null;
creditsUsd: Prisma.Decimal | null;
utilityMovements: bigint | number | string;
insuranceMovements: bigint | number | string;
lastMovement: Date | null;
}
/**
* Raw-query counts come back in three shapes depending on the aggregate:
* `COUNT(*)` as bigint, `SUM(bool)` as a decimal *string*, and plain numbers.
* Normalize all of them before they reach the client as JSON.
*/
function num(v: bigint | number | string | null | undefined): number {
if (v === null || v === undefined) return 0;
return typeof v === "number" ? v : Number(v);
}
function dec(v: Prisma.Decimal | null | undefined): string {
return (v ?? new Prisma.Decimal(0)).toFixed(2);
}
@Injectable()
export class BillingService {
constructor(private readonly prisma: PrismaService) {}
private movementWhere(p: MovementParams): Prisma.TransactionWhereInput {
const and: Prisma.TransactionWhereInput[] = [];
if (p.query && p.query.trim()) {
const q = p.query.trim();
and.push({
OR: [
{ customer: { name: { contains: q } } },
{ reference: { contains: q } },
{ checkNumber: { contains: q } },
{ message: { contains: q } },
{ period: { contains: q } },
],
});
}
if (p.domain) and.push({ domain: p.domain });
if (p.currency) and.push({ currency: p.currency });
// A charge is strictly negative and a credit strictly positive; the ~193
// zero-amount rows are neither and are excluded from both sides on purpose.
if (p.direction === "charge") and.push({ amount: { lt: 0 } });
if (p.direction === "credit") and.push({ amount: { gt: 0 } });
if (p.typeId) and.push({ typeId: p.typeId });
if (p.source) and.push({ legacySourceTable: p.source });
if (p.customerId) and.push({ customerId: p.customerId });
if (p.from || p.to) {
and.push({
transactionDate: {
...(p.from ? { gte: p.from } : {}),
...(p.to ? { lte: p.to } : {}),
},
});
}
return and.length ? { AND: and } : {};
}
private movementOrderBy(
sort: MovementSort,
): Prisma.TransactionOrderByWithRelationInput[] {
switch (sort) {
case "date_asc":
return [{ transactionDate: "asc" }];
case "amount_desc":
return [{ amount: "desc" }];
case "amount_asc":
return [{ amount: "asc" }];
case "customer":
return [
{ customer: { nameMissing: "asc" } },
{ customer: { name: "asc" } },
{ transactionDate: "desc" },
];
default:
return [{ transactionDate: "desc" }];
}
}
/** Cross-customer movement browser — every charge and credit, filterable. */
async movements(params: MovementParams) {
const where = this.movementWhere(params);
const [total, rows] = await this.prisma.$transaction([
this.prisma.transaction.count({ where }),
this.prisma.transaction.findMany({
where,
skip: (params.page - 1) * params.pageSize,
take: params.pageSize,
orderBy: this.movementOrderBy(params.sort),
select: {
id: true,
transactionDate: true,
domain: true,
amount: true,
currency: true,
reference: true,
period: true,
checkNumber: true,
message: true,
legacySourceTable: true,
type: { select: { nameEn: true, nameEs: true } },
customer: {
select: { id: true, name: true, nameSource: true, city: true },
},
},
}),
]);
// Totals for the *filtered set*, not just the page — the number staff read
// off a filtered view ("how much did we bill for water in April") has to
// cover everything the filter matched.
const totals = await this.prisma.transaction.groupBy({
by: ["currency"],
where,
_sum: { amount: true },
_count: { _all: true },
});
const charges = await this.prisma.transaction.groupBy({
by: ["currency"],
where: { AND: [where, { amount: { lt: 0 } }] },
_sum: { amount: true },
_count: { _all: true },
});
const credits = await this.prisma.transaction.groupBy({
by: ["currency"],
where: { AND: [where, { amount: { gt: 0 } }] },
_sum: { amount: true },
_count: { _all: true },
});
const chargeMap = new Map(charges.map((c) => [c.currency, c]));
const creditMap = new Map(credits.map((c) => [c.currency, c]));
return {
items: rows.map((r) => ({
id: r.id,
transactionDate: r.transactionDate,
domain: r.domain,
amount: r.amount,
currency: r.currency,
direction: r.amount.lessThan(0) ? "charge" : "credit",
reference: r.reference,
period: r.period,
checkNumber: r.checkNumber,
message: r.message,
source: r.legacySourceTable,
type: r.type,
customerId: r.customer.id,
customerName: r.customer.name,
customerNameSource: r.customer.nameSource,
customerCity: r.customer.city,
})),
total,
page: params.page,
pageSize: params.pageSize,
pageCount: Math.ceil(total / params.pageSize),
totals: totals.map((t) => ({
currency: t.currency,
net: t._sum.amount,
count: t._count._all,
charges: chargeMap.get(t.currency)?._sum.amount ?? null,
chargeCount: chargeMap.get(t.currency)?._count._all ?? 0,
credits: creditMap.get(t.currency)?._sum.amount ?? null,
creditCount: creditMap.get(t.currency)?._count._all ?? 0,
})),
};
}
/**
* Receivables worklist: one row per customer with a ledger, carrying both
* currency balances, filtered/sorted on the caller's chosen currency.
*
* Raw SQL rather than Prisma `groupBy` because this needs conditional sums
* per currency *and* per direction in a single pass, plus ordering and
* pagination on a computed balance — none of which groupBy expresses.
*/
async balances(params: BalanceParams) {
const { query, page, pageSize, currency, balance, domain, sort } = params;
const filters: Prisma.Sql[] = [];
if (domain) filters.push(Prisma.sql`t.domain = ${domain}`);
const txFilter = filters.length
? Prisma.sql`AND ${Prisma.join(filters, " AND ")}`
: Prisma.empty;
const nameFilter =
query && query.trim()
? Prisma.sql`AND (c.name LIKE ${`%${query.trim()}%`} OR c.city LIKE ${`%${query.trim()}%`})`
: Prisma.empty;
// The balance column the filter and sort act on.
const bal =
currency === "USD"
? Prisma.sql`SUM(CASE WHEN t.currency = 'USD' THEN t.amount ELSE 0 END)`
: Prisma.sql`SUM(CASE WHEN t.currency = 'MXN' THEN t.amount ELSE 0 END)`;
// "Owing" is a *negative* balance (see the sign convention above). The
// 0.005 threshold keeps rounding dust out of both worklists.
let having = Prisma.empty;
if (balance === "owing") having = Prisma.sql`HAVING ${bal} < -0.005`;
else if (balance === "credit") having = Prisma.sql`HAVING ${bal} > 0.005`;
else if (balance === "settled")
having = Prisma.sql`HAVING ${bal} BETWEEN -0.005 AND 0.005`;
let orderBy: Prisma.Sql;
switch (sort) {
case "credit_desc":
orderBy = Prisma.sql`ORDER BY ${bal} DESC`;
break;
case "recent":
orderBy = Prisma.sql`ORDER BY MAX(t.transactionDate) DESC`;
break;
case "customer":
orderBy = Prisma.sql`ORDER BY c.nameMissing ASC, c.name ASC`;
break;
default:
// Deepest debt first — the point of the worklist.
orderBy = Prisma.sql`ORDER BY ${bal} ASC`;
}
const rows = await this.prisma.$queryRaw<BalanceRow[]>`
SELECT
c.id,
c.name,
c.nameSource,
c.nameMissing,
c.city,
c.state,
COUNT(*) AS movements,
SUM(CASE WHEN t.currency = 'MXN' THEN t.amount ELSE 0 END) AS balanceMxn,
SUM(CASE WHEN t.currency = 'USD' THEN t.amount ELSE 0 END) AS balanceUsd,
SUM(CASE WHEN t.currency = 'MXN' AND t.amount < 0 THEN t.amount ELSE 0 END) AS chargesMxn,
SUM(CASE WHEN t.currency = 'MXN' AND t.amount > 0 THEN t.amount ELSE 0 END) AS creditsMxn,
SUM(CASE WHEN t.currency = 'USD' AND t.amount < 0 THEN t.amount ELSE 0 END) AS chargesUsd,
SUM(CASE WHEN t.currency = 'USD' AND t.amount > 0 THEN t.amount ELSE 0 END) AS creditsUsd,
SUM(t.domain = 'UTILITY') AS utilityMovements,
SUM(t.domain = 'INSURANCE') AS insuranceMovements,
MAX(t.transactionDate) AS lastMovement
FROM customers c
JOIN transactions t ON t.customerId = c.id
WHERE 1 = 1 ${nameFilter} ${txFilter}
GROUP BY c.id, c.name, c.nameSource, c.nameMissing, c.city, c.state
${having}
${orderBy}
LIMIT ${pageSize} OFFSET ${(page - 1) * pageSize}
`;
const counted = await this.prisma.$queryRaw<{ total: bigint | number | string }[]>`
SELECT COUNT(*) AS total FROM (
SELECT c.id
FROM customers c
JOIN transactions t ON t.customerId = c.id
WHERE 1 = 1 ${nameFilter} ${txFilter}
GROUP BY c.id
${having}
) x
`;
const total = num(counted[0]?.total);
return {
items: rows.map((r) => ({
id: r.id,
name: r.name,
nameSource: r.nameSource,
city: r.city,
state: r.state,
movements: num(r.movements),
utilityMovements: num(r.utilityMovements),
insuranceMovements: num(r.insuranceMovements),
lastMovement: r.lastMovement,
balances: [
{
currency: "MXN",
balance: dec(r.balanceMxn),
charges: dec(r.chargesMxn),
credits: dec(r.creditsMxn),
},
{
currency: "USD",
balance: dec(r.balanceUsd),
charges: dec(r.chargesUsd),
credits: dec(r.creditsUsd),
},
],
})),
total,
page,
pageSize,
pageCount: Math.ceil(total / pageSize),
currency,
};
}
/** Top-line figures for the billing page header. */
async stats() {
const [movements, ledgerCustomers, byCurrency, byDomain] = await Promise.all([
this.prisma.transaction.count(),
this.prisma.transaction
.findMany({ distinct: ["customerId"], select: { customerId: true } })
.then((r) => r.length),
this.prisma.transaction.groupBy({
by: ["currency"],
_sum: { amount: true },
_count: { _all: true },
}),
this.prisma.transaction.groupBy({
by: ["domain", "currency"],
_sum: { amount: true },
_count: { _all: true },
}),
]);
const charges = await this.prisma.transaction.groupBy({
by: ["currency"],
where: { amount: { lt: 0 } },
_sum: { amount: true },
_count: { _all: true },
});
const credits = await this.prisma.transaction.groupBy({
by: ["currency"],
where: { amount: { gt: 0 } },
_sum: { amount: true },
_count: { _all: true },
});
const chargeMap = new Map(charges.map((c) => [c.currency, c]));
const creditMap = new Map(credits.map((c) => [c.currency, c]));
// How many customers sit on each side of the line, per currency — the
// headline for a receivables view. Counted in SQL; a customer can be
// "owing" in MXN and "in credit" in USD, and both are true at once.
const sides = await this.prisma.$queryRaw<
{
currency: string;
owing: bigint | number | string;
inCredit: bigint | number | string;
}[]
>`
SELECT currency,
SUM(bal < -0.005) AS owing,
SUM(bal > 0.005) AS inCredit
FROM (
SELECT customerId, currency, SUM(amount) AS bal
FROM transactions GROUP BY customerId, currency
) x
GROUP BY currency
`;
const sideMap = new Map(sides.map((s) => [s.currency, s]));
const [firstRow, lastRow] = await Promise.all([
this.prisma.transaction.findFirst({
orderBy: { transactionDate: "asc" },
select: { transactionDate: true },
}),
this.prisma.transaction.findFirst({
orderBy: { transactionDate: "desc" },
select: { transactionDate: true },
}),
]);
// Customers whose ledger spans both business lines — the whole reason this
// module is one view instead of two.
const crossLine = await this.prisma.$queryRaw<{ n: bigint | number | string }[]>`
SELECT COUNT(*) AS n FROM (
SELECT customerId FROM transactions
GROUP BY customerId HAVING COUNT(DISTINCT domain) > 1
) x
`;
return {
movements,
ledgerCustomers,
crossLineCustomers: num(crossLine[0]?.n),
firstMovement: firstRow?.transactionDate ?? null,
lastMovement: lastRow?.transactionDate ?? null,
byCurrency: byCurrency.map((c) => ({
currency: c.currency,
net: c._sum.amount,
count: c._count._all,
charges: chargeMap.get(c.currency)?._sum.amount ?? null,
chargeCount: chargeMap.get(c.currency)?._count._all ?? 0,
credits: creditMap.get(c.currency)?._sum.amount ?? null,
creditCount: creditMap.get(c.currency)?._count._all ?? 0,
owing: num(sideMap.get(c.currency)?.owing),
inCredit: num(sideMap.get(c.currency)?.inCredit),
})),
byDomain: byDomain.map((d) => ({
domain: d.domain,
currency: d.currency,
net: d._sum.amount,
count: d._count._all,
})),
};
}
/** Filter dropdown options for the movement browser. */
async facets() {
const types = await this.prisma.transaction.groupBy({
by: ["typeId"],
where: { typeId: { not: null } },
_count: { _all: true },
orderBy: { _count: { typeId: "desc" } },
});
const typeRows = await this.prisma.typeTransaction.findMany({
where: { id: { in: types.map((t) => t.typeId as string) } },
select: { id: true, nameEn: true, nameEs: true },
});
const typeMap = new Map(typeRows.map((t) => [t.id, t]));
const sources = await this.prisma.transaction.groupBy({
by: ["legacySourceTable"],
_count: { _all: true },
orderBy: { _count: { legacySourceTable: "desc" } },
});
const years = await this.prisma.$queryRaw<
{ year: number; count: bigint | number | string }[]
>`
SELECT YEAR(transactionDate) AS year, COUNT(*) AS count
FROM transactions GROUP BY year ORDER BY year DESC
`;
return {
types: types
.map((t) => {
const row = typeMap.get(t.typeId as string);
return {
id: t.typeId as string,
name: row?.nameEs || row?.nameEn || "—",
count: t._count._all,
};
})
.filter((t) => t.name !== "—"),
sources: sources.map((s) => ({
name: s.legacySourceTable ?? "—",
count: s._count._all,
})),
years: years.map((y) => ({ year: Number(y.year), count: num(y.count) })),
};
}
/**
* One customer's statement across both business lines.
*
* Returns the *whole* ledger rather than a page of it: the heaviest customer
* carries 365 movements (mean 26), and a running balance is meaningless if
* the client only holds a slice. The running balance is accumulated per
* currency in chronological order, then the list is handed back newest-first
* with each row's balance-after already attached.
*/
async statement(customerId: string) {
const customer = await this.prisma.customer.findUnique({
where: { id: customerId },
select: {
id: true,
name: true,
nameSource: true,
addressLine1: true,
city: true,
state: true,
phone: true,
mobile: true,
email: true,
customerSince: true,
preferredCurrency: true,
status: true,
_count: { select: { properties: true, policies: true } },
},
});
if (!customer) {
throw new NotFoundException(`Customer ${customerId} not found`);
}
const rows = await this.prisma.transaction.findMany({
where: { customerId },
orderBy: [{ transactionDate: "asc" }, { id: "asc" }],
select: {
id: true,
transactionDate: true,
domain: true,
amount: true,
currency: true,
reference: true,
period: true,
checkNumber: true,
message: true,
legacySourceTable: true,
type: { select: { nameEn: true, nameEs: true } },
},
});
const running = new Map<string, Prisma.Decimal>();
const movements = rows.map((r) => {
const prev = running.get(r.currency) ?? new Prisma.Decimal(0);
const next = prev.plus(r.amount);
running.set(r.currency, next);
return {
id: r.id,
transactionDate: r.transactionDate,
domain: r.domain,
amount: r.amount,
currency: r.currency,
direction: r.amount.lessThan(0) ? "charge" : "credit",
reference: r.reference,
period: r.period,
checkNumber: r.checkNumber,
message: r.message,
source: r.legacySourceTable,
type: r.type,
/** Balance in this row's currency after applying it. */
balanceAfter: next.toFixed(2),
};
});
movements.reverse();
// Per-currency summary, and the same split by business line so the two
// ledgers are visibly one statement without being illegally added up.
const perCurrency = new Map<
string,
{
currency: string;
charges: Prisma.Decimal;
credits: Prisma.Decimal;
chargeCount: number;
creditCount: number;
count: number;
first: Date | null;
last: Date | null;
}
>();
const perDomain = new Map<
string,
{
domain: TransactionDomain;
currency: string;
charges: Prisma.Decimal;
credits: Prisma.Decimal;
count: number;
}
>();
for (const r of rows) {
const c =
perCurrency.get(r.currency) ??
{
currency: r.currency,
charges: new Prisma.Decimal(0),
credits: new Prisma.Decimal(0),
chargeCount: 0,
creditCount: 0,
count: 0,
first: null as Date | null,
last: null as Date | null,
};
c.count += 1;
if (r.amount.lessThan(0)) {
c.charges = c.charges.plus(r.amount);
c.chargeCount += 1;
} else if (r.amount.greaterThan(0)) {
c.credits = c.credits.plus(r.amount);
c.creditCount += 1;
}
if (!c.first) c.first = r.transactionDate;
c.last = r.transactionDate;
perCurrency.set(r.currency, c);
const dk = `${r.domain}|${r.currency}`;
const d =
perDomain.get(dk) ??
{
domain: r.domain,
currency: r.currency,
charges: new Prisma.Decimal(0),
credits: new Prisma.Decimal(0),
count: 0,
};
d.count += 1;
if (r.amount.lessThan(0)) d.charges = d.charges.plus(r.amount);
else if (r.amount.greaterThan(0)) d.credits = d.credits.plus(r.amount);
perDomain.set(dk, d);
}
// Where the money goes, per charge type — the question a customer asks
// when they query their balance.
const byType = new Map<
string,
{ name: string; currency: string; total: Prisma.Decimal; count: number }
>();
for (const r of rows) {
if (!r.amount.lessThan(0)) continue;
const name = r.type?.nameEs || r.type?.nameEn || "Sin clasificar";
const key = `${name}|${r.currency}`;
const e =
byType.get(key) ??
{ name, currency: r.currency, total: new Prisma.Decimal(0), count: 0 };
e.total = e.total.plus(r.amount);
e.count += 1;
byType.set(key, e);
}
return {
customer: {
...customer,
propertyCount: customer._count.properties,
policyCount: customer._count.policies,
},
summary: [...perCurrency.values()].map((c) => ({
currency: c.currency,
charges: c.charges.toFixed(2),
credits: c.credits.toFixed(2),
balance: c.charges.plus(c.credits).toFixed(2),
chargeCount: c.chargeCount,
creditCount: c.creditCount,
count: c.count,
firstMovement: c.first,
lastMovement: c.last,
})),
byDomain: [...perDomain.values()].map((d) => ({
domain: d.domain,
currency: d.currency,
charges: d.charges.toFixed(2),
credits: d.credits.toFixed(2),
balance: d.charges.plus(d.credits).toFixed(2),
count: d.count,
})),
byType: [...byType.values()]
.map((t) => ({
name: t.name,
currency: t.currency,
total: t.total.toFixed(2),
count: t.count,
}))
.sort((a, b) => Number(a.total) - Number(b.total)),
movements,
};
}
}
+11
View File
@@ -93,6 +93,7 @@ function Detail({ id }: { id: string }) {
<PropiedadesSection properties={data.properties} /> <PropiedadesSection properties={data.properties} />
<PolizasSection policies={data.policies} /> <PolizasSection policies={data.policies} />
<EstadoCuentaSection <EstadoCuentaSection
customerId={data.id}
summary={data.transactionSummary} summary={data.transactionSummary}
transactions={data.transactions} transactions={data.transactions}
/> />
@@ -574,9 +575,11 @@ function InstallmentRow({ inst }: { inst: Installment }) {
/* ------------------------------------------------------- Estado de cuenta */ /* ------------------------------------------------------- Estado de cuenta */
function EstadoCuentaSection({ function EstadoCuentaSection({
customerId,
summary, summary,
transactions, transactions,
}: { }: {
customerId: string;
summary: TransactionSummaryRow[]; summary: TransactionSummaryRow[];
transactions: Transaction[]; transactions: Transaction[];
}) { }) {
@@ -639,6 +642,14 @@ function EstadoCuentaSection({
</div> </div>
)} )}
</div> </div>
{transactions.length > 0 && (
<p className="section-note">
<Link href={`/estado-cuenta/${customerId}`} className="inline-link">
Ver estado de cuenta completo
</Link>{" "}
con saldo, saldo corrido y desglose por línea de negocio.
</p>
)}
</section> </section>
); );
} }
@@ -0,0 +1,500 @@
"use client";
import { useEffect, useMemo, useState } from "react";
import Link from "next/link";
import { AppShell } from "@/components/AppShell";
import { getStatement } from "@/lib/api";
import {
balancePhrase,
balanceTone,
directionLabel,
domainLabel,
formatDate,
formatMoney,
formatNumber,
ledgerSourceLabel,
SIN_NOMBRE,
txTypeLabel,
} from "@/lib/labels";
import type {
LedgerCurrency,
Statement,
StatementMovement,
TransactionDomain,
} from "@/lib/types";
/**
* One customer's statement across both business lines — the payoff of plan
* step 6 and, ultimately, of the whole unified-customer project: a utility
* charge and an insurance payment finally sit on the same page, under the same
* person, with a running balance.
*
* The running balance is per currency (the API accumulates it chronologically
* before handing the list back newest-first), so the movement table is scoped
* to one currency at a time — a column that alternated between pesos and
* dollars would be a meaningless number.
*/
export default function EstadoCuentaDetailPage({
params,
}: {
params: { id: string };
}) {
const { id } = params;
return (
<AppShell>
<StatementView id={id} />
</AppShell>
);
}
function StatementView({ id }: { id: string }) {
const [data, setData] = useState<Statement | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const [currency, setCurrency] = useState<LedgerCurrency | null>(null);
const [domain, setDomain] = useState<TransactionDomain | "">("");
useEffect(() => {
let alive = true;
setLoading(true);
setError(null);
getStatement(id)
.then((d) => {
if (!alive) return;
setData(d);
// Default to the currency the customer actually moves the most in.
const busiest = [...d.summary].sort((a, b) => b.count - a.count)[0];
setCurrency(busiest?.currency ?? "MXN");
setLoading(false);
})
.catch((e) => {
if (!alive) return;
setError(
e?.status === 404
? "No encontramos este cliente."
: e?.message ?? "No se pudo cargar el estado de cuenta.",
);
setLoading(false);
});
return () => {
alive = false;
};
}, [id]);
const movements = useMemo(() => {
if (!data || !currency) return [];
return data.movements.filter(
(m) => m.currency === currency && (!domain || m.domain === domain),
);
}, [data, currency, domain]);
if (loading) return <StatementSkeleton />;
if (error)
return (
<>
<BackLink />
<div className="state-error" role="alert">
{error}
</div>
</>
);
if (!data || !currency) return null;
const active = data.summary.find((s) => s.currency === currency);
return (
<div className="rise">
<BackLink />
<Hero data={data} />
<section className="section">
<SectionHead rule="cuenta" title="Saldo por moneda" />
{data.summary.length === 0 ? (
<div className="card">
<div className="empty-inline">
Este cliente no tiene movimientos registrados.
</div>
</div>
) : (
<div className="summary-grid">
{data.summary.map((s) => {
const tone = balanceTone(s.balance);
return (
<button
type="button"
key={s.currency}
className={`summary-card bal-card ${tone}${
currency === s.currency ? " selected" : ""
}`}
onClick={() => setCurrency(s.currency)}
aria-pressed={currency === s.currency}
>
<div className="summary-domain">
Saldo en {s.currency} · {balancePhrase(s.balance)}
</div>
<div className={`summary-total bal-amount ${tone}`}>
{formatMoney(s.balance, s.currency)}
</div>
<div className="bal-breakdown">
<span className="tx-amount neg">
{formatMoney(s.charges, s.currency)}
</span>
<span className="bal-breakdown-label">
{formatNumber(s.chargeCount)} cargos
</span>
<span className="tx-amount pos">
{formatMoney(s.credits, s.currency)}
</span>
<span className="bal-breakdown-label">
{formatNumber(s.creditCount)} abonos
</span>
</div>
<div className="summary-count">
{formatDate(s.firstMovement)} a {formatDate(s.lastMovement)}
</div>
</button>
);
})}
</div>
)}
<p className="section-note">
Los saldos se muestran por separado en cada moneda. La contabilidad
heredada registró los cargos únicamente en pesos y los recibos en
ambas monedas, sin guardar el tipo de cambio aplicado a cada
movimiento, por lo que sumarlas produciría una cifra que nunca existió
en los libros.
</p>
</section>
<PorLineaSection data={data} currency={currency} />
<ConceptosSection data={data} currency={currency} />
<section className="section">
<SectionHead
rule="cuenta"
title="Movimientos"
count={movements.length}
countSuffix={movements.length === 1 ? "movimiento" : "movimientos"}
/>
<div className="filter-row">
<label className="filter-field">
<span className="filter-label">Moneda</span>
<select
className="input select"
value={currency}
onChange={(e) => setCurrency(e.target.value as LedgerCurrency)}
>
{data.summary.map((s) => (
<option key={s.currency} value={s.currency}>
{s.currency} ({formatNumber(s.count)})
</option>
))}
</select>
</label>
<label className="filter-field">
<span className="filter-label">Línea de negocio</span>
<select
className="input select"
value={domain}
onChange={(e) =>
setDomain(e.target.value as TransactionDomain | "")
}
>
<option value="">Ambas líneas</option>
<option value="UTILITY">Servicios</option>
<option value="INSURANCE">Seguros</option>
</select>
</label>
</div>
<div className="card">
{movements.length === 0 ? (
<div className="empty-inline">
Sin movimientos en {currency}
{domain ? ` para ${domainLabel(domain)}` : ""}.
</div>
) : (
<div className="tx-scroll">
<table className="tx-table">
<thead>
<tr>
<th>Fecha</th>
<th>Línea</th>
<th>Concepto</th>
<th>Referencia</th>
<th className="num">Cargo / Abono</th>
<th className="num">Saldo</th>
</tr>
</thead>
<tbody>
{movements.map((m) => (
<StatementRow key={m.id} m={m} />
))}
</tbody>
</table>
</div>
)}
{domain && movements.length > 0 && (
<div className="section-note" style={{ padding: "0 16px 14px" }}>
La columna de saldo es el saldo acumulado del cliente en{" "}
{currency} sobre <strong>todas</strong> sus líneas filtrar por
línea oculta filas, no las descuenta.
</div>
)}
</div>
{active && (
<p className="section-note">
Saldo final en {currency}:{" "}
<strong>{formatMoney(active.balance, currency)}</strong> (
{balancePhrase(active.balance).toLowerCase()}).
</p>
)}
</section>
</div>
);
}
function BackLink() {
return (
<Link href="/estado-cuenta" className="back-link">
Volver a Estado de cuenta
</Link>
);
}
function Hero({ data }: { data: Statement }) {
const c = data.customer;
const location = [c.city?.replace(/,\s*$/, ""), c.state]
.filter(Boolean)
.join(", ");
const facts: { label: string; value: string }[] = [
{ label: "Cliente desde", value: formatDate(c.customerSince) },
{ label: "Propiedades", value: String(c.propertyCount) },
{ label: "Pólizas", value: String(c.policyCount) },
{ label: "Teléfono", value: c.phone || c.mobile || "—" },
{ label: "Correo", value: c.email || "—" },
];
return (
<div className="detail-hero">
<div className="hero-top">
<div>
<h1
className={`hero-name${
c.name === SIN_NOMBRE ? " hero-name-missing" : ""
}`}
>
{c.name}
</h1>
{location && <div className="hero-provenance">{location}</div>}
{c.nameSource && (
<div className="hero-provenance">
Nombre recuperado de {c.nameSource} el registro original no
tenía nombre.
</div>
)}
</div>
<div className="hero-badges">
{c.propertyCount > 0 && (
<span className="badge badge-servicios">
<span className="dot" /> Servicios
</span>
)}
{c.policyCount > 0 && (
<span className="badge badge-seguros">
<span className="dot" /> Seguros
</span>
)}
<span className={`badge ${c.status ? "badge-on-dark" : "badge-negative"}`}>
{c.status ? "Activo" : "Inactivo"}
</span>
</div>
</div>
<div className="hero-facts">
{facts.map((f) => (
<div key={f.label}>
<div className="hero-fact-label">{f.label}</div>
<div className="hero-fact-value">{f.value}</div>
</div>
))}
</div>
<div className="hero-links">
<Link href={`/clientes/${c.id}`} className="btn btn-outline">
Ver ficha del cliente
</Link>
</div>
</div>
);
}
/** The cross-line split — the same balance, broken out by business line. */
function PorLineaSection({
data,
currency,
}: {
data: Statement;
currency: LedgerCurrency;
}) {
const rows = data.byDomain.filter((d) => d.currency === currency);
if (rows.length === 0) return null;
return (
<section className="section">
<SectionHead rule="cuenta" title={`Por línea de negocio · ${currency}`} />
<div className="summary-grid">
{rows.map((r) => (
<div className={`summary-card ${r.domain}`} key={r.domain}>
<div className="summary-domain">
<span className={`tx-dot ${r.domain}`} />
{domainLabel(r.domain)}
</div>
<div className={`summary-total bal-amount ${balanceTone(r.balance)}`}>
{formatMoney(r.balance, currency)}
</div>
<div className="bal-breakdown">
<span className="tx-amount neg">
{formatMoney(r.charges, currency)}
</span>
<span className="bal-breakdown-label">en cargos</span>
<span className="tx-amount pos">
{formatMoney(r.credits, currency)}
</span>
<span className="bal-breakdown-label">en abonos</span>
</div>
<div className="summary-count">
{formatNumber(r.count)}{" "}
{r.count === 1 ? "movimiento" : "movimientos"}
</div>
</div>
))}
</div>
</section>
);
}
/** Where the charges went — the question a customer asks about their balance. */
function ConceptosSection({
data,
currency,
}: {
data: Statement;
currency: LedgerCurrency;
}) {
const rows = data.byType.filter((t) => t.currency === currency).slice(0, 12);
if (rows.length === 0) return null;
const largest = Math.abs(Number(rows[0]?.total ?? 0)) || 1;
return (
<section className="section">
<SectionHead rule="servicios" title={`Cargos por concepto · ${currency}`} />
<div className="card">
<div className="concept-list">
{rows.map((t) => (
<div className="concept-row" key={`${t.name}-${t.currency}`}>
<div className="concept-name">
{txTypeLabel({ nameEn: t.name })}
<span className="concept-count">
{formatNumber(t.count)}{" "}
{t.count === 1 ? "cargo" : "cargos"}
</span>
</div>
<div className="concept-bar" aria-hidden>
<span
style={{
width: `${Math.max(
2,
(Math.abs(Number(t.total)) / largest) * 100,
)}%`,
}}
/>
</div>
<div className="concept-total tx-amount neg">
{formatMoney(t.total, currency)}
</div>
</div>
))}
</div>
</div>
</section>
);
}
function StatementRow({ m }: { m: StatementMovement }) {
const concept = m.message || m.period || null;
return (
<tr>
<td className="mono" style={{ whiteSpace: "nowrap" }}>
{formatDate(m.transactionDate)}
</td>
<td className="tx-domain-cell">
<span className={`tx-dot ${m.domain}`} />
{domainLabel(m.domain)}
</td>
<td>
{txTypeLabel(m.type)}
{concept && <div className="tx-concept">{concept}</div>}
</td>
<td className="tx-ref">
{m.reference || m.checkNumber || "—"}
<div className="tx-concept">{ledgerSourceLabel(m.source)}</div>
</td>
<td className="num">
<span className={`tx-amount ${m.direction === "charge" ? "neg" : "pos"}`}>
{formatMoney(m.amount, m.currency)}
</span>
<span className="tx-cur">{directionLabel(m.direction)}</span>
</td>
<td className="num">
<span className={`bal-running ${balanceTone(m.balanceAfter)}`}>
{formatMoney(m.balanceAfter, m.currency)}
</span>
</td>
</tr>
);
}
function SectionHead({
rule,
title,
count,
countSuffix,
}: {
rule: string;
title: string;
count?: number;
countSuffix?: string;
}) {
return (
<div className="section-head">
<span className={`section-rule ${rule}`} aria-hidden />
<h2 className="section-title">{title}</h2>
{count != null && (
<span className="section-count">
{formatNumber(count)} {countSuffix ?? ""}
</span>
)}
</div>
);
}
function StatementSkeleton() {
return (
<div aria-hidden>
<div className="skeleton" style={{ height: 18, width: 180 }} />
<div
className="skeleton"
style={{ height: 150, marginTop: 16, borderRadius: 16 }}
/>
<div
className="skeleton"
style={{ height: 320, marginTop: 24, borderRadius: 16 }}
/>
</div>
);
}
+845
View File
@@ -0,0 +1,845 @@
"use client";
import { useCallback, useEffect, useRef, useState } from "react";
import Link from "next/link";
import { AppShell } from "@/components/AppShell";
import {
getBillingFacets,
getBillingStats,
listBalances,
listMovements,
} from "@/lib/api";
import {
balancePhrase,
balanceTone,
directionLabel,
domainLabel,
formatDate,
formatMoney,
formatNumber,
ledgerSourceLabel,
SIN_NOMBRE,
txTypeLabel,
} from "@/lib/labels";
import type {
BalanceFilter,
BalanceListItem,
BalanceListResponse,
BalanceSort,
BillingFacets,
BillingStats,
LedgerCurrency,
LedgerDirection,
MovementListItem,
MovementListResponse,
MovementSort,
TransactionDomain,
} from "@/lib/types";
/**
* Shared billing / statements browser — plan step 6.
*
* Two views over the same ledger, because staff ask two different questions:
* - "Saldos": who owes what, one row per customer. The receivables worklist.
* - "Movimientos": every individual charge and credit, filterable — the
* answer to "what did we bill for water in April".
*
* Both are cross-line: a customer's utility charges and insurance movements sit
* in the same ledger, which is the point of the unified customer record.
*
* Balances are always shown *per currency* and never added together — see the
* currency note in `billing.service.ts`.
*/
type View = "saldos" | "movimientos";
const BALANCE_FILTERS: { key: BalanceFilter; label: string }[] = [
{ key: "owing", label: "Con adeudo" },
{ key: "credit", label: "Con saldo a favor" },
{ key: "settled", label: "En ceros" },
{ key: "all", label: "Todos" },
];
const BALANCE_SORTS: { key: BalanceSort; label: string }[] = [
{ key: "owing_desc", label: "Mayor adeudo primero" },
{ key: "credit_desc", label: "Mayor saldo a favor primero" },
{ key: "recent", label: "Movimiento más reciente" },
{ key: "customer", label: "Cliente (AZ)" },
];
const MOVEMENT_SORTS: { key: MovementSort; label: string }[] = [
{ key: "date_desc", label: "Fecha (más reciente)" },
{ key: "date_asc", label: "Fecha (más antigua)" },
{ key: "amount_asc", label: "Cargo más grande" },
{ key: "amount_desc", label: "Abono más grande" },
{ key: "customer", label: "Cliente (AZ)" },
];
const DIRECTIONS: { key: LedgerDirection | ""; label: string }[] = [
{ key: "", label: "Cargos y abonos" },
{ key: "charge", label: "Sólo cargos" },
{ key: "credit", label: "Sólo abonos" },
];
const DOMAINS: { key: TransactionDomain | ""; label: string }[] = [
{ key: "", label: "Ambas líneas" },
{ key: "UTILITY", label: "Servicios" },
{ key: "INSURANCE", label: "Seguros" },
];
export default function EstadoCuentaPage() {
return (
<AppShell>
<BillingBrowser />
</AppShell>
);
}
function BillingBrowser() {
const [stats, setStats] = useState<BillingStats | null>(null);
const [facets, setFacets] = useState<BillingFacets | null>(null);
const [view, setView] = useState<View>("saldos");
// The currency every balance figure is filtered and sorted on. MXN is the
// default because the charge side of the ledger is MXN-only.
const [currency, setCurrency] = useState<LedgerCurrency>("MXN");
const [query, setQuery] = useState("");
const [domain, setDomain] = useState<TransactionDomain | "">("");
const [balanceFilter, setBalanceFilter] = useState<BalanceFilter>("owing");
const [balanceSort, setBalanceSort] = useState<BalanceSort>("owing_desc");
const [direction, setDirection] = useState<LedgerDirection | "">("");
const [typeId, setTypeId] = useState("");
const [source, setSource] = useState("");
const [from, setFrom] = useState("");
const [to, setTo] = useState("");
const [movementSort, setMovementSort] = useState<MovementSort>("date_desc");
const [balances, setBalances] = useState<BalanceListResponse | null>(null);
const [movements, setMovements] = useState<MovementListResponse | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const debounceRef = useRef<ReturnType<typeof setTimeout>>();
useEffect(() => {
getBillingStats().then(setStats).catch(() => setStats(null));
getBillingFacets().then(setFacets).catch(() => setFacets(null));
}, []);
const runSearch = useCallback(
(p: number) => {
setLoading(true);
setError(null);
const done = (fn: () => void) => {
fn();
setLoading(false);
};
if (view === "saldos") {
listBalances({
query: query || undefined,
currency,
balance: balanceFilter,
domain: domain || undefined,
sort: balanceSort,
page: p,
pageSize: 25,
})
.then((res) => done(() => setBalances(res)))
.catch((e) => {
setError(e?.message ?? "No se pudieron cargar los saldos.");
setLoading(false);
});
} else {
listMovements({
query: query || undefined,
currency,
domain: domain || undefined,
direction: direction || undefined,
typeId: typeId || undefined,
source: source || undefined,
from: from || undefined,
to: to || undefined,
sort: movementSort,
page: p,
pageSize: 25,
})
.then((res) => done(() => setMovements(res)))
.catch((e) => {
setError(e?.message ?? "No se pudieron cargar los movimientos.");
setLoading(false);
});
}
},
[
view,
query,
currency,
domain,
balanceFilter,
balanceSort,
direction,
typeId,
source,
from,
to,
movementSort,
],
);
useEffect(() => {
if (debounceRef.current) clearTimeout(debounceRef.current);
debounceRef.current = setTimeout(() => runSearch(1), 280);
return () => {
if (debounceRef.current) clearTimeout(debounceRef.current);
};
}, [runSearch]);
function goToPage(p: number) {
runSearch(p);
if (typeof window !== "undefined")
window.scrollTo({ top: 0, behavior: "smooth" });
}
/** Jumping in from a headline count should land on the matching worklist. */
function pickBalance(f: BalanceFilter, cur?: LedgerCurrency) {
setView("saldos");
setBalanceFilter(f);
if (cur) setCurrency(cur);
setBalanceSort(f === "credit" ? "credit_desc" : "owing_desc");
}
const data = view === "saldos" ? balances : movements;
const filtered =
query !== "" ||
domain !== "" ||
(view === "saldos"
? balanceFilter !== "owing" || balanceSort !== "owing_desc"
: direction !== "" ||
typeId !== "" ||
source !== "" ||
from !== "" ||
to !== "" ||
movementSort !== "date_desc");
function clearFilters() {
setQuery("");
setDomain("");
setCurrency("MXN");
setBalanceFilter("owing");
setBalanceSort("owing_desc");
setDirection("");
setTypeId("");
setSource("");
setFrom("");
setTo("");
setMovementSort("date_desc");
}
return (
<>
<div className="page-head rise">
<p className="eyebrow">Cobranza y facturación</p>
<h1 className="page-title">Estado de cuenta</h1>
<BillingStatStrip
stats={stats}
currency={currency}
balanceFilter={view === "saldos" ? balanceFilter : null}
onPickBalance={pickBalance}
/>
<LedgerTotalsStrip stats={stats} />
</div>
<div className="toolbar">
<div className="search-box">
<span className="search-icon" aria-hidden>
</span>
<input
className="input search-input"
type="search"
value={query}
onChange={(e) => setQuery(e.target.value)}
placeholder={
view === "saldos"
? "Buscar por cliente o ciudad…"
: "Buscar por cliente, referencia, cheque, concepto…"
}
aria-label="Buscar en el estado de cuenta"
/>
</div>
<div className="seg" role="tablist" aria-label="Vista">
{(
[
{ key: "saldos" as View, label: "Saldos por cliente" },
{ key: "movimientos" as View, label: "Movimientos" },
]
).map((v) => (
<button
key={v.key}
type="button"
role="tab"
aria-selected={view === v.key}
className={`seg-btn ${view === v.key ? "active" : ""}`}
onClick={() => setView(v.key)}
>
{v.label}
</button>
))}
</div>
</div>
<div className="filter-row">
<label className="filter-field">
<span className="filter-label">Moneda</span>
<select
className="input select"
value={currency}
onChange={(e) => setCurrency(e.target.value as LedgerCurrency)}
>
<option value="MXN">Pesos (MXN)</option>
<option value="USD">Dólares (USD)</option>
</select>
</label>
<label className="filter-field">
<span className="filter-label">Línea de negocio</span>
<select
className="input select"
value={domain}
onChange={(e) => setDomain(e.target.value as TransactionDomain | "")}
>
{DOMAINS.map((d) => (
<option key={d.key} value={d.key}>
{d.label}
</option>
))}
</select>
</label>
{view === "saldos" ? (
<>
<label className="filter-field">
<span className="filter-label">Saldo</span>
<select
className="input select"
value={balanceFilter}
onChange={(e) =>
setBalanceFilter(e.target.value as BalanceFilter)
}
>
{BALANCE_FILTERS.map((b) => (
<option key={b.key} value={b.key}>
{b.label}
</option>
))}
</select>
</label>
<label className="filter-field">
<span className="filter-label">Ordenar por</span>
<select
className="input select"
value={balanceSort}
onChange={(e) => setBalanceSort(e.target.value as BalanceSort)}
>
{BALANCE_SORTS.map((s) => (
<option key={s.key} value={s.key}>
{s.label}
</option>
))}
</select>
</label>
</>
) : (
<>
<label className="filter-field">
<span className="filter-label">Movimiento</span>
<select
className="input select"
value={direction}
onChange={(e) =>
setDirection(e.target.value as LedgerDirection | "")
}
>
{DIRECTIONS.map((d) => (
<option key={d.key} value={d.key}>
{d.label}
</option>
))}
</select>
</label>
<label className="filter-field">
<span className="filter-label">Concepto</span>
<select
className="input select"
value={typeId}
onChange={(e) => setTypeId(e.target.value)}
>
<option value="">Todos los conceptos</option>
{facets?.types.map((t) => (
<option key={t.id} value={t.id}>
{txTypeLabel({ nameEn: t.name })} ({formatNumber(t.count)})
</option>
))}
</select>
</label>
<label className="filter-field">
<span className="filter-label">Origen</span>
<select
className="input select"
value={source}
onChange={(e) => setSource(e.target.value)}
>
<option value="">Todos los orígenes</option>
{facets?.sources.map((s) => (
<option key={s.name} value={s.name}>
{ledgerSourceLabel(s.name)} ({formatNumber(s.count)})
</option>
))}
</select>
</label>
<label className="filter-field">
<span className="filter-label">Desde</span>
<input
className="input"
type="date"
value={from}
onChange={(e) => setFrom(e.target.value)}
/>
</label>
<label className="filter-field">
<span className="filter-label">Hasta</span>
<input
className="input"
type="date"
value={to}
onChange={(e) => setTo(e.target.value)}
/>
</label>
<label className="filter-field">
<span className="filter-label">Ordenar por</span>
<select
className="input select"
value={movementSort}
onChange={(e) =>
setMovementSort(e.target.value as MovementSort)
}
>
{MOVEMENT_SORTS.map((s) => (
<option key={s.key} value={s.key}>
{s.label}
</option>
))}
</select>
</label>
</>
)}
{filtered && (
<button
type="button"
className="btn btn-ghost filter-clear"
onClick={clearFilters}
>
Limpiar filtros
</button>
)}
</div>
{data && !loading && !error && (
<div className="result-meta" aria-live="polite">
{data.total === 0
? "Sin resultados"
: view === "saldos"
? `${formatNumber(data.total)} ${
data.total === 1 ? "cliente" : "clientes"
} · saldo en ${currency}`
: `${formatNumber(data.total)} ${
data.total === 1 ? "movimiento" : "movimientos"
}`}
{query ? ` para “${query}` : ""}
</div>
)}
{view === "movimientos" && movements && !loading && (
<FilteredTotals totals={movements.totals} />
)}
{error ? (
<div className="state-error" role="alert">
{error}
</div>
) : loading ? (
<ListSkeleton />
) : data && data.total === 0 ? (
<EmptyState query={query} view={view} />
) : view === "saldos" ? (
<>
<div className="cust-list">
{balances?.items.map((b) => (
<BalanceRow key={b.id} b={b} currency={currency} />
))}
</div>
{balances && balances.pageCount > 1 && (
<Pager
page={balances.page}
pageCount={balances.pageCount}
onChange={goToPage}
/>
)}
</>
) : (
<>
<div className="card">
<div className="tx-scroll">
<table className="tx-table">
<thead>
<tr>
<th>Fecha</th>
<th>Cliente</th>
<th>Línea</th>
<th>Concepto</th>
<th>Referencia</th>
<th className="num">Monto</th>
</tr>
</thead>
<tbody>
{movements?.items.map((m) => (
<MovementRow key={m.id} m={m} />
))}
</tbody>
</table>
</div>
</div>
{movements && movements.pageCount > 1 && (
<Pager
page={movements.page}
pageCount={movements.pageCount}
onChange={goToPage}
/>
)}
</>
)}
</>
);
}
/** Headline counts; the owing/credit cells double as worklist shortcuts. */
function BillingStatStrip({
stats,
currency,
balanceFilter,
onPickBalance,
}: {
stats: BillingStats | null;
currency: LedgerCurrency;
balanceFilter: BalanceFilter | null;
onPickBalance: (f: BalanceFilter, cur?: LedgerCurrency) => void;
}) {
if (!stats) {
return (
<div className="stat-strip" aria-hidden>
{Array.from({ length: 5 }).map((_, i) => (
<div className="stat-cell" key={i}>
<div className="skeleton" style={{ height: 25, width: "60%" }} />
<div
className="skeleton"
style={{ height: 11, width: "80%", marginTop: 8 }}
/>
</div>
))}
</div>
);
}
const cur = stats.byCurrency.find((c) => c.currency === currency);
return (
<div className="stat-strip">
<button
type="button"
className={`stat-cell stat-cell-btn accent${
balanceFilter === "owing" ? " selected" : ""
}`}
onClick={() => onPickBalance("owing")}
aria-pressed={balanceFilter === "owing"}
>
<div className="stat-value">{formatNumber(cur?.owing ?? 0)}</div>
<div className="stat-label">Clientes con adeudo ({currency})</div>
</button>
<button
type="button"
className={`stat-cell stat-cell-btn${
balanceFilter === "credit" ? " selected" : ""
}`}
onClick={() => onPickBalance("credit")}
aria-pressed={balanceFilter === "credit"}
>
<div className="stat-value">{formatNumber(cur?.inCredit ?? 0)}</div>
<div className="stat-label">Con saldo a favor ({currency})</div>
</button>
<button
type="button"
className={`stat-cell stat-cell-btn${
balanceFilter === "all" ? " selected" : ""
}`}
onClick={() => onPickBalance("all")}
aria-pressed={balanceFilter === "all"}
>
<div className="stat-value">{formatNumber(stats.ledgerCustomers)}</div>
<div className="stat-label">Clientes con movimientos</div>
</button>
<div className="stat-cell">
<div className="stat-value">{formatNumber(stats.movements)}</div>
<div className="stat-label">
Movimientos · {formatDate(stats.firstMovement)} a{" "}
{formatDate(stats.lastMovement)}
</div>
</div>
<div className="stat-cell">
<div className="stat-value">
{formatNumber(stats.crossLineCustomers)}
</div>
<div className="stat-label">Con movimientos en ambas líneas</div>
</div>
</div>
);
}
/**
* Charges vs credits for the whole ledger, per currency. Kept as two separate
* chips rather than one figure: the two currencies are never added together.
*/
function LedgerTotalsStrip({ stats }: { stats: BillingStats | null }) {
if (!stats || stats.byCurrency.length === 0) return null;
return (
<div className="mix-strip">
<span className="premium-caption">Movimiento histórico</span>
{stats.byCurrency.map((c) => (
<div className="ledger-chip" key={c.currency}>
<span className="ledger-chip-cur">{c.currency}</span>
<span className="ledger-chip-figs">
<span className="tx-amount neg">
{formatMoney(c.charges, c.currency)}
</span>
<span className="ledger-chip-label">
en cargos · {formatNumber(c.chargeCount)}
</span>
</span>
<span className="ledger-chip-figs">
<span className="tx-amount pos">
{formatMoney(c.credits, c.currency)}
</span>
<span className="ledger-chip-label">
en abonos · {formatNumber(c.creditCount)}
</span>
</span>
</div>
))}
</div>
);
}
/** Totals for everything the current movement filter matched, not just the page. */
function FilteredTotals({
totals,
}: {
totals: MovementListResponse["totals"];
}) {
if (totals.length === 0) return null;
return (
<div className="filtered-totals">
{totals.map((t) => (
<div className="filtered-total" key={t.currency}>
<span className="filtered-total-cur">{t.currency}</span>
<span>
<strong className="tx-amount neg">
{formatMoney(t.charges, t.currency)}
</strong>{" "}
cargos
</span>
<span>
<strong className="tx-amount pos">
{formatMoney(t.credits, t.currency)}
</strong>{" "}
abonos
</span>
<span className="filtered-total-net">
Neto <strong>{formatMoney(t.net, t.currency)}</strong>
</span>
</div>
))}
</div>
);
}
function BalanceRow({
b,
currency,
}: {
b: BalanceListItem;
currency: LedgerCurrency;
}) {
const selected =
b.balances.find((x) => x.currency === currency) ?? b.balances[0];
const other = b.balances.find((x) => x.currency !== currency);
const tone = balanceTone(selected.balance);
const location = [b.city?.replace(/,\s*$/, ""), b.state]
.filter(Boolean)
.join(", ");
return (
<Link href={`/estado-cuenta/${b.id}`} className="cust-row bal-row">
<div className="cust-main">
<div className="cust-name">
<span className={b.name === SIN_NOMBRE ? "cust-name-missing" : undefined}>
{b.name}
</span>
{b.utilityMovements > 0 && (
<span className="badge badge-servicios">
<span className="dot" /> Servicios
</span>
)}
{b.insuranceMovements > 0 && (
<span className="badge badge-seguros">
<span className="dot" /> Seguros
</span>
)}
</div>
<div className="cust-sub">
{location && <span>{location}</span>}
{location && <span className="sep">·</span>}
<span>
{formatNumber(b.movements)}{" "}
{b.movements === 1 ? "movimiento" : "movimientos"}
</span>
<span className="sep">·</span>
<span>último {formatDate(b.lastMovement)}</span>
</div>
</div>
<div className="bal-side">
<div className={`bal-amount ${tone}`}>
{formatMoney(selected.balance, selected.currency)}
</div>
<div className={`bal-phrase ${tone}`}>
{balancePhrase(selected.balance)} · {selected.currency}
</div>
{other && Math.abs(Number(other.balance)) >= 0.005 && (
<div className="bal-other">
{formatMoney(other.balance, other.currency)} en {other.currency}
</div>
)}
</div>
</Link>
);
}
function MovementRow({ m }: { m: MovementListItem }) {
return (
<tr>
<td className="mono" style={{ whiteSpace: "nowrap" }}>
{formatDate(m.transactionDate)}
</td>
<td>
<Link href={`/estado-cuenta/${m.customerId}`} className="inline-link">
<span
className={
m.customerName === SIN_NOMBRE ? "cust-name-missing" : undefined
}
>
{m.customerName}
</span>
</Link>
</td>
<td className="tx-domain-cell">
<span className={`tx-dot ${m.domain}`} />
{domainLabel(m.domain)}
</td>
<td>
{txTypeLabel(m.type)}
{m.message && <div className="tx-concept">{m.message}</div>}
</td>
<td className="tx-ref">
{m.reference || m.checkNumber || "—"}
<div className="tx-concept">{ledgerSourceLabel(m.source)}</div>
</td>
<td className="num">
<span className={`tx-amount ${m.direction === "charge" ? "neg" : "pos"}`}>
{formatMoney(m.amount, m.currency)}
</span>
<span className="tx-cur">
{m.currency} · {directionLabel(m.direction)}
</span>
</td>
</tr>
);
}
function Pager({
page,
pageCount,
onChange,
}: {
page: number;
pageCount: number;
onChange: (p: number) => void;
}) {
return (
<nav className="pager" aria-label="Paginación">
<button
type="button"
className="btn btn-outline"
onClick={() => onChange(page - 1)}
disabled={page <= 1}
>
Anterior
</button>
<span className="pager-info">
Página <strong>{page}</strong> de {pageCount}
</span>
<button
type="button"
className="btn btn-outline"
onClick={() => onChange(page + 1)}
disabled={page >= pageCount}
>
Siguiente
</button>
</nav>
);
}
function ListSkeleton() {
return (
<div className="cust-list" aria-hidden>
{Array.from({ length: 8 }).map((_, i) => (
<div className="skeleton skel-row" key={i} />
))}
</div>
);
}
function EmptyState({ query, view }: { query: string; view: View }) {
return (
<div className="state-box">
<div className="state-glyph" aria-hidden>
</div>
<h3>Sin resultados</h3>
<p>
{query
? `No encontramos ${
view === "saldos" ? "clientes" : "movimientos"
} para “${query}”.`
: `No hay ${
view === "saldos" ? "saldos" : "movimientos"
} que coincidan con los filtros.`}
</p>
</div>
);
}
+225
View File
@@ -1820,3 +1820,228 @@ button {
white-space: nowrap; white-space: nowrap;
border: 0; border: 0;
} }
/* ================================================================
Estado de cuenta (billing / statements) — plan step 6
================================================================ */
/* A balance is signed: negative means the customer owes the office, positive
means they hold a credit. `flat` is a real state (settled to zero), not a
fallback, so it gets its own muted treatment rather than inheriting either. */
.bal-amount {
font-family: var(--font-mono);
font-weight: 700;
font-feature-settings: "tnum" 1;
white-space: nowrap;
}
.bal-amount.owing {
color: var(--negative);
}
.bal-amount.credit {
color: var(--positive);
}
.bal-amount.flat {
color: var(--muted);
}
.bal-running {
font-family: var(--font-mono);
font-size: 12.5px;
font-feature-settings: "tnum" 1;
white-space: nowrap;
color: var(--ink-soft);
}
.bal-running.owing {
color: var(--negative);
}
.bal-running.credit {
color: var(--positive);
}
.bal-row .bal-side {
text-align: right;
display: flex;
flex-direction: column;
gap: 2px;
align-items: flex-end;
min-width: 160px;
}
.bal-side .bal-amount {
font-size: 16px;
}
.bal-phrase {
font-size: 11px;
font-weight: 600;
letter-spacing: 0.03em;
text-transform: uppercase;
}
.bal-phrase.owing {
color: var(--negative);
}
.bal-phrase.credit {
color: var(--positive);
}
.bal-phrase.flat {
color: var(--muted);
}
/* The customer's other-currency balance, shown so a peso figure is never
mistaken for the whole picture. */
.bal-other {
font-size: 11.5px;
color: var(--muted);
font-family: var(--font-mono);
}
@media (max-width: 640px) {
.bal-row .bal-side {
align-items: flex-start;
text-align: left;
min-width: 0;
}
}
/* Currency cards on the statement double as the movement-table currency
switch, so they are buttons, not divs. */
.bal-card {
text-align: left;
cursor: pointer;
font: inherit;
border: 1px solid var(--line);
transition: border-color 0.15s ease, transform 0.15s ease;
}
.bal-card:hover {
transform: translateY(-1px);
}
.bal-card.selected {
border-color: var(--brand-600);
box-shadow: 0 0 0 1px var(--brand-600);
}
.bal-breakdown {
display: grid;
grid-template-columns: auto 1fr;
gap: 1px 8px;
margin-top: 8px;
font-size: 12px;
align-items: baseline;
}
.bal-breakdown-label {
color: var(--muted);
font-size: 11.5px;
}
/* Ledger-wide charge/credit totals in the page header. */
.ledger-chip {
display: flex;
align-items: center;
gap: 14px;
padding: 8px 14px;
border: 1px solid var(--line);
border-radius: 10px;
background: var(--surface-2);
}
.ledger-chip-cur {
font-weight: 700;
font-size: 12px;
letter-spacing: 0.06em;
color: var(--muted);
}
.ledger-chip-figs {
display: flex;
flex-direction: column;
line-height: 1.3;
}
.ledger-chip-label {
font-size: 11px;
color: var(--muted);
}
/* Totals for the current movement filter — deliberately above the table, so a
filtered view can't be read as if it were the whole ledger. */
.filtered-totals {
display: flex;
flex-wrap: wrap;
gap: 10px;
margin-bottom: 12px;
}
.filtered-total {
display: flex;
flex-wrap: wrap;
align-items: baseline;
gap: 14px;
padding: 9px 14px;
border: 1px solid var(--line);
border-radius: 10px;
background: var(--surface-2);
font-size: 12.5px;
color: var(--muted);
}
.filtered-total-cur {
font-weight: 700;
letter-spacing: 0.06em;
color: var(--ink-soft);
}
.filtered-total-net strong {
font-family: var(--font-mono);
color: var(--ink);
}
/* Charges broken out by concept, with a proportional bar. */
.concept-list {
display: flex;
flex-direction: column;
}
.concept-row {
display: grid;
grid-template-columns: minmax(150px, 1.2fr) minmax(60px, 2fr) auto;
gap: 14px;
align-items: center;
padding: 9px 0;
border-bottom: 1px solid var(--line);
}
.concept-row:last-child {
border-bottom: none;
}
.concept-name {
font-size: 13.5px;
font-weight: 600;
display: flex;
flex-direction: column;
}
.concept-count {
font-size: 11.5px;
font-weight: 500;
color: var(--muted);
}
.concept-bar {
height: 7px;
background: var(--surface-2);
border-radius: 4px;
overflow: hidden;
}
.concept-bar span {
display: block;
height: 100%;
border-radius: 4px;
background: var(--negative);
opacity: 0.55;
}
.concept-total {
text-align: right;
font-size: 13px;
}
@media (max-width: 640px) {
.concept-row {
grid-template-columns: 1fr auto;
}
.concept-bar {
display: none;
}
}
/* Cross-link out of a detail hero (statement -> customer file). */
.hero-links {
margin-top: 16px;
display: flex;
gap: 10px;
flex-wrap: wrap;
position: relative;
z-index: 1;
}
+4 -1
View File
@@ -438,7 +438,10 @@ function MovimientosSection({ data }: { data: PropertyDetail }) {
<div className="section-note" style={{ padding: "0 16px 14px" }}> <div className="section-note" style={{ padding: "0 16px 14px" }}>
Los movimientos pertenecen al cliente, no a esta propiedad: el Los movimientos pertenecen al cliente, no a esta propiedad: el
sistema anterior nunca ligó un pago a una propiedad concreta. Ver el{" "} sistema anterior nunca ligó un pago a una propiedad concreta. Ver el{" "}
<Link href={`/clientes/${data.customerId}`} className="inline-link"> <Link
href={`/estado-cuenta/${data.customerId}`}
className="inline-link"
>
estado de cuenta completo estado de cuenta completo
</Link> </Link>
. .
+1
View File
@@ -15,6 +15,7 @@ const NAV = [
{ href: "/clientes", label: "Clientes" }, { href: "/clientes", label: "Clientes" },
{ href: "/servicios", label: "Propiedades" }, { href: "/servicios", label: "Propiedades" },
{ href: "/polizas", label: "Pólizas" }, { href: "/polizas", label: "Pólizas" },
{ href: "/estado-cuenta", label: "Estado de cuenta" },
]; ];
export function AppShell({ children }: { children: ReactNode }) { export function AppShell({ children }: { children: ReactNode }) {
+82
View File
@@ -3,10 +3,19 @@
import type { import type {
AuthUser, AuthUser,
BalanceFilter,
BalanceListResponse,
BalanceSort,
BillingFacets,
BillingStats,
BusinessLine, BusinessLine,
CustomerDetail, CustomerDetail,
CustomerListResponse, CustomerListResponse,
CustomerStats, CustomerStats,
LedgerCurrency,
LedgerDirection,
MovementListResponse,
MovementSort,
PolicyDetail, PolicyDetail,
PolicyFacets, PolicyFacets,
PolicyListResponse, PolicyListResponse,
@@ -19,6 +28,8 @@ import type {
PropertySort, PropertySort,
PropertyStats, PropertyStats,
ServiceKind, ServiceKind,
Statement,
TransactionDomain,
TrustFilter, TrustFilter,
} from "./types"; } from "./types";
@@ -212,3 +223,74 @@ export function getProperty(
): Promise<PropertyDetail> { ): Promise<PropertyDetail> {
return apiFetch<PropertyDetail>(`/properties/${id}?days=${days}`); return apiFetch<PropertyDetail>(`/properties/${id}?days=${days}`);
} }
/* ------------------------------------------- Billing / statements module */
export interface MovementQuery {
query?: string;
page?: number;
pageSize?: number;
domain?: TransactionDomain;
currency?: LedgerCurrency;
direction?: LedgerDirection;
typeId?: string;
source?: string;
customerId?: string;
/** `YYYY-MM-DD`, inclusive on both ends. */
from?: string;
to?: string;
sort?: MovementSort;
}
export function listMovements(q: MovementQuery): Promise<MovementListResponse> {
const params = new URLSearchParams();
if (q.query) params.set("query", q.query);
if (q.page) params.set("page", String(q.page));
if (q.pageSize) params.set("pageSize", String(q.pageSize));
if (q.domain) params.set("domain", q.domain);
if (q.currency) params.set("currency", q.currency);
if (q.direction) params.set("direction", q.direction);
if (q.typeId) params.set("typeId", q.typeId);
if (q.source) params.set("source", q.source);
if (q.customerId) params.set("customerId", q.customerId);
if (q.from) params.set("from", q.from);
if (q.to) params.set("to", q.to);
if (q.sort) params.set("sort", q.sort);
const qs = params.toString();
return apiFetch<MovementListResponse>(`/billing${qs ? `?${qs}` : ""}`);
}
export interface BalanceQuery {
query?: string;
page?: number;
pageSize?: number;
currency?: LedgerCurrency;
balance?: BalanceFilter;
domain?: TransactionDomain;
sort?: BalanceSort;
}
export function listBalances(q: BalanceQuery): Promise<BalanceListResponse> {
const params = new URLSearchParams();
if (q.query) params.set("query", q.query);
if (q.page) params.set("page", String(q.page));
if (q.pageSize) params.set("pageSize", String(q.pageSize));
if (q.currency) params.set("currency", q.currency);
if (q.balance) params.set("balance", q.balance);
if (q.domain) params.set("domain", q.domain);
if (q.sort) params.set("sort", q.sort);
const qs = params.toString();
return apiFetch<BalanceListResponse>(`/billing/balances${qs ? `?${qs}` : ""}`);
}
export function getBillingStats(): Promise<BillingStats> {
return apiFetch<BillingStats>("/billing/stats");
}
export function getBillingFacets(): Promise<BillingFacets> {
return apiFetch<BillingFacets>("/billing/facets");
}
export function getStatement(customerId: string): Promise<Statement> {
return apiFetch<Statement>(`/billing/customers/${customerId}`);
}
+96
View File
@@ -1,6 +1,7 @@
// Spanish label maps + formatting helpers. Single source of truth for i18n. // Spanish label maps + formatting helpers. Single source of truth for i18n.
import type { import type {
LedgerDirection,
PolicyStatus, PolicyStatus,
ServiceKind, ServiceKind,
TransactionDomain, TransactionDomain,
@@ -123,6 +124,101 @@ export function expiryPhrase(days: number | null): string | null {
return `venció hace ${past} ${past === 1 ? "día" : "días"}`; return `venció hace ${past} ${past === 1 ? "día" : "días"}`;
} }
// ----- ledger / estado de cuenta -----
/**
* A charge is negative and a credit positive (see `billing.service.ts`), so the
* balance is the plain sum. These are the two words the office uses.
*/
export const DIRECTION_LABELS: Record<LedgerDirection, string> = {
charge: "Cargo",
credit: "Abono",
};
export function directionLabel(d: LedgerDirection): string {
return DIRECTION_LABELS[d] ?? d;
}
/**
* Spanish names for the legacy `TYPE OF TRX` lookup.
*
* The lookup ships an `ESPAÑOL` column, but it is **empty in the source** — all
* 79 rows are null — so the API can only return the English name. This map
* covers the entries that are real service/payment categories; the rest of the
* 79 are payee names (LORETO GONZALEZ, ALBERCAS VALLARTA…) that shouldn't be
* translated anyway, and fall through to the raw value.
*/
export const TX_TYPE_LABELS: Record<string, string> = {
WATER: "Agua",
ELECTRIC: "Electricidad",
TELEPHONE: "Teléfono",
"PROPERTY TAXES": "Predial",
"FEDERAL ZONE": "Zona federal",
"GAS BUTANO": "Gas butano",
"GAS REFILL": "Recarga de gas",
"TRUST FEE": "Cuota de fideicomiso",
"HOA DUES": "Cuota de asociación",
"ALARM SYSTEM": "Sistema de alarma",
"HOUSE INSURANCE": "Seguro de casa",
"AUTO INSURANCE": "Seguro de auto",
"CHECK DEPOSIT": "Depósito con cheque",
"CASH DEPOSIT": "Depósito en efectivo",
PAYPAL: "PayPal",
"RETURNED CHECK": "Cheque devuelto",
"ACCOUNT CANCELED": "Cuenta cancelada",
"BANK FEE": "Comisión bancaria",
"BANK INTEREST": "Interés bancario",
SECURITY: "Vigilancia",
BALANCE: "Saldo",
ACCOUNTANT: "Contador",
"RENEWAL CONCESSION": "Renovación de concesión",
};
export function txTypeLabel(
type: { nameEs?: string | null; nameEn?: string | null } | null | undefined,
): string {
const raw = type?.nameEs || type?.nameEn;
if (!raw) return "Sin clasificar";
return TX_TYPE_LABELS[raw.toUpperCase()] ?? raw;
}
/**
* Legacy table a movement came from. Shown so a staff member checking a
* surprising figure can trace it back to the Access table it was migrated from.
*/
export const LEDGER_SOURCE_LABELS: Record<string, string> = {
datos2: "Facturación 202526",
"FEE ANUAL": "Cuota anual 2018",
fee15: "Cuota anual 2017",
"IVA 2015": "IVA 2015",
EFECTIVO: "Recibos de caja",
EFECTIVO_BACKUP: "Recibos de caja (respaldo)",
"EFECTIVO FM3": "Trámites FM3",
"CHEQUE FM3": "Trámites FM3 (cheque)",
};
export function ledgerSourceLabel(source: string | null | undefined): string {
if (!source) return "—";
return LEDGER_SOURCE_LABELS[source] ?? source;
}
/**
* Balance wording. Negative = the customer owes the office; positive = the
* customer is in credit (they have money on account).
*/
export function balancePhrase(balance: string | number): string {
const n = typeof balance === "string" ? Number(balance) : balance;
if (!Number.isFinite(n) || Math.abs(n) < 0.005) return "Sin saldo";
return n < 0 ? "Adeudo" : "A favor";
}
/** CSS-class suffix matching `balancePhrase`, for colouring a figure. */
export function balanceTone(balance: string | number): "owing" | "credit" | "flat" {
const n = typeof balance === "string" ? Number(balance) : balance;
if (!Number.isFinite(n) || Math.abs(n) < 0.005) return "flat";
return n < 0 ? "owing" : "credit";
}
// ----- formatting ----- // ----- formatting -----
export function formatMoney( export function formatMoney(
+173
View File
@@ -470,6 +470,179 @@ export interface TransactionSummaryRow {
count: number; count: number;
} }
/* ------------------------------------------- Billing / statements module */
/**
* Which side of the ledger a movement sits on. `transactions.amount` is signed:
* a charge (cargo) is negative, a credit (abono) is positive, so the balance is
* simply the sum — negative means the customer owes the office.
*/
export type LedgerDirection = "charge" | "credit";
/** The only two currencies in the ledger. Totals are never summed across them. */
export type LedgerCurrency = "MXN" | "USD";
export type BalanceFilter = "all" | "owing" | "credit" | "settled";
export type MovementSort =
| "date_desc"
| "date_asc"
| "amount_desc"
| "amount_asc"
| "customer";
export type BalanceSort = "owing_desc" | "credit_desc" | "recent" | "customer";
export interface Movement {
id: string;
transactionDate: string | null;
domain: TransactionDomain;
amount: string;
currency: LedgerCurrency;
direction: LedgerDirection;
reference: string | null;
period: string | null;
checkNumber: string | null;
message: string | null;
/** Legacy table the row came from — `datos2`, `EFECTIVO`, `fee15`, … */
source: string | null;
type: TransactionType | null;
}
export interface MovementListItem extends Movement {
customerId: string;
customerName: string;
customerNameSource: string | null;
customerCity: string | null;
}
export interface CurrencyTotals {
currency: LedgerCurrency;
net: string | null;
count: number;
charges: string | null;
chargeCount: number;
credits: string | null;
creditCount: number;
}
export interface MovementListResponse {
items: MovementListItem[];
total: number;
page: number;
pageSize: number;
pageCount: number;
/** Totals for the whole filtered set, not just the current page. */
totals: CurrencyTotals[];
}
export interface CurrencyBalance {
currency: LedgerCurrency;
balance: string;
charges: string;
credits: string;
}
export interface BalanceListItem {
id: string;
name: string;
nameSource: string | null;
city: string | null;
state: string | null;
movements: number;
utilityMovements: number;
insuranceMovements: number;
lastMovement: string | null;
balances: CurrencyBalance[];
}
export interface BalanceListResponse {
items: BalanceListItem[];
total: number;
page: number;
pageSize: number;
pageCount: number;
currency: LedgerCurrency;
}
export interface BillingStats {
movements: number;
ledgerCustomers: number;
/** Customers whose ledger spans utilities *and* insurance. */
crossLineCustomers: number;
firstMovement: string | null;
lastMovement: string | null;
byCurrency: (CurrencyTotals & { owing: number; inCredit: number })[];
byDomain: {
domain: TransactionDomain;
currency: LedgerCurrency;
net: string | null;
count: number;
}[];
}
export interface BillingFacets {
types: Facet[];
sources: { name: string; count: number }[];
years: { year: number; count: number }[];
}
export interface StatementSummary {
currency: LedgerCurrency;
charges: string;
credits: string;
balance: string;
chargeCount: number;
creditCount: number;
count: number;
firstMovement: string | null;
lastMovement: string | null;
}
export interface StatementDomainRow {
domain: TransactionDomain;
currency: LedgerCurrency;
charges: string;
credits: string;
balance: string;
count: number;
}
export interface StatementTypeRow {
name: string;
currency: LedgerCurrency;
total: string;
count: number;
}
export interface StatementMovement extends Movement {
/** Balance in this row's currency after the movement was applied. */
balanceAfter: string;
}
export interface Statement {
customer: {
id: string;
name: string;
nameSource: string | null;
addressLine1: string | null;
city: string | null;
state: string | null;
phone: string | null;
mobile: string | null;
email: string | null;
customerSince: string | null;
preferredCurrency: string | null;
status: boolean;
propertyCount: number;
policyCount: number;
};
summary: StatementSummary[];
byDomain: StatementDomainRow[];
byType: StatementTypeRow[];
movements: StatementMovement[];
}
export interface CustomerDetail { export interface CustomerDetail {
id: string; id: string;
name: string; name: string;
+7 -5
View File
@@ -2,17 +2,19 @@
_Migration plan step 2. Generated by `reconcile.py` from staged Parquet (`output/stg_utilities`). Regenerate: `./.venv/bin/python reconcile.py > RECONCILIATION.md`._ _Migration plan step 2. Generated by `reconcile.py` from staged Parquet (`output/stg_utilities`). Regenerate: `./.venv/bin/python reconcile.py > RECONCILIATION.md`._
**Headline:** none of the three suspected "duplicate" groups are what the plan assumed. They are disjoint historical/period data or a mislabeled batch — see each group's decided rule. **Headline:** `EFECTIVO_BACKUP` *is* a duplicate of `EFECTIVO` and must not be double-loaded; the billing tables are genuinely disjoint period runs; `COBRO3` is a mislabeled charge batch, not a customer master. See each group's decided rule.
## 1. Cash ledger — `EFECTIVO` vs `EFECTIVO_BACKUP` ## 1. Cash ledger — `EFECTIVO` vs `EFECTIVO_BACKUP`
- `EFECTIVO`: 13697 rows. `EFECTIVO_BACKUP`: 12387 rows. - `EFECTIVO`: 13697 rows. `EFECTIVO_BACKUP`: 12387 rows.
- `folio` is per-table sequential: 13697 distinct in EFECTIVO (= row count), 12369 in BACKUP. 12363 folio *numbers* appear in both. - `folio` is per-table sequential: 13697 distinct in EFECTIVO (= row count), 12369 in BACKUP. 12363 folio *numbers* appear in both.
- **But of those 12363 shared folio numbers, 12363 carry a *different* transaction** (differ on ['cl', 'fecha', 'monto', 'conepto']). → `folio` collides; it is NOT a stable cross-table id. - **But of those 12363 shared folio numbers, 12204 carry a *different* transaction** (differ on ['cl', 'fecha', 'monto', 'conepto']). → `folio` collides; it is NOT a stable cross-table id, and it cannot be the de-dup key.
- On the real business key `['cl', 'fecha', 'monto', 'conepto']`: **2 rows in both**, 13663 only in EFECTIVO, 12385 only in BACKUP. - On the real business key `['cl', 'fecha', 'monto', 'conepto']`, canonicalized: **12386 rows in both**, 1279 only in EFECTIVO, 1 only in BACKUP — i.e. all but 1 of BACKUP's 12387 rows already exist verbatim in EFECTIVO, same customer, same timestamp to the second, same amount, same concept text.
- Date ranges are different eras: BACKUP is dominated by 20172022 records, EFECTIVO by recent ones. - Yearly row counts track each other almost exactly from 2006 to 2023 (e.g. 2017: 1036 vs 994), which is what a stale copy looks like — not two ledgers covering different eras.
**Decided rule:** the two tables are **near-disjoint ledgers**, not a live/backup duplicate pair (only 2 shared payments out of ~13k+12k). Migrate **both** into `transactions`, each row keyed internally by `(legacy_source_table, folio)` provenance — do **not** de-dup on `folio` (it collides) and do **not** drop BACKUP (it holds ~12k older payments absent from EFECTIVO). The 2 business-key matches are the only possible double-counts and should be spot-checked, but at that volume they don't threaten balance integrity. **Decided rule (corrected):** `EFECTIVO_BACKUP` is a **stale backup copy of `EFECTIVO`**, not an independent ledger. Load `EFECTIVO` in full, and load from `EFECTIVO_BACKUP` only the rows whose canonicalized business key is absent from `EFECTIVO` (1 row(s)). Loading both in full double-counts 12386 payments and doubles nearly every customer's historical receipt total, which makes any statement or balance view wrong. De-dup on the business key, **not** on `folio` (it collides).
> This reverses the original verdict in this report, which read `{b}` as 2. That number came from string-comparing `monto`, which `mdb-export` serializes with a different precision per table (`5000` vs `27000.0000`) — see the module docstring.
> `EFECTIVO FM3` (627) and `CHEQUE FM3` (157) are a separate stream — `fee`/`tax`/`multa` columns instead of `monto` — and migrate as distinct transactions, not reconciled against EFECTIVO. > `EFECTIVO FM3` (627) and `CHEQUE FM3` (157) are a separate stream — `fee`/`tax`/`multa` columns instead of `monto` — and migrate as distinct transactions, not reconciled against EFECTIVO.
+57 -21
View File
@@ -17,9 +17,17 @@ which is a trap — it hides *why*. The interesting question is whether two
tables hold the SAME economic records under cosmetic differences, or tables hold the SAME economic records under cosmetic differences, or
genuinely DIFFERENT records. So each group is probed on a deliberate genuinely DIFFERENT records. So each group is probed on a deliberate
*business key* (the columns that identify the real-world thing) and the *business key* (the columns that identify the real-world thing) and the
volatile/identity columns are examined separately. Every table went through volatile/identity columns are examined separately.
the same mdb-export path, so identical source values serialize identically —
string comparison on a chosen key is a valid identity test. Method correction (2026-07-22): this file previously assumed that "every table
went through the same mdb-export path, so identical source values serialize
identically". That is false. `mdb-export` formats a numeric column according to
its *Access column type*, so the same amount is emitted as `5000` from one table
and `27000.0000` from another. String-comparing a numeric key column therefore
reports near-zero overlap between tables holding identical records — which is
exactly what produced the original (wrong) "EFECTIVO / EFECTIVO_BACKUP are
near-disjoint ledgers" verdict. Every key column is now canonicalized (numeric
columns parsed and re-formatted to a fixed precision) before comparison.
""" """
from __future__ import annotations from __future__ import annotations
@@ -41,8 +49,26 @@ def load(name: str) -> pd.DataFrame:
return df return df
def canon(sr: pd.Series) -> pd.Series:
"""Canonicalize one key column so it compares across tables.
A column that parses as a number in (nearly) every populated cell is
re-emitted at fixed precision, which erases the per-table formatting
mdb-export applies from the Access column type. Anything else (dates,
free text) is left as the already-stripped string.
"""
populated = sr.ne(_NULL)
if not populated.any():
return sr
num = pd.to_numeric(sr.where(populated), errors="coerce")
if num.notna().sum() < 0.9 * populated.sum():
return sr
return num.map(lambda v: _NULL if pd.isna(v) else f"{v:.4f}").astype("string")
def keyset(df: pd.DataFrame, cols: list[str]) -> set[str]: def keyset(df: pd.DataFrame, cols: list[str]) -> set[str]:
return set(df[cols].agg("\x1f".join, axis=1)) keyed = pd.DataFrame({c: canon(df[c]) for c in cols})
return set(keyed.agg("\x1f".join, axis=1))
def overlap(a: pd.DataFrame, b: pd.DataFrame, cols: list[str]): def overlap(a: pd.DataFrame, b: pd.DataFrame, cols: list[str]):
@@ -65,9 +91,10 @@ def main() -> None:
"(`output/stg_utilities`). Regenerate: `./.venv/bin/python reconcile.py " "(`output/stg_utilities`). Regenerate: `./.venv/bin/python reconcile.py "
"> RECONCILIATION.md`._") "> RECONCILIATION.md`._")
p("") p("")
p("**Headline:** none of the three suspected \"duplicate\" groups are what " p("**Headline:** `EFECTIVO_BACKUP` *is* a duplicate of `EFECTIVO` and must "
"the plan assumed. They are disjoint historical/period data or a " "not be double-loaded; the billing tables are genuinely disjoint period "
"mislabeled batch — see each group's decided rule.") "runs; `COBRO3` is a mislabeled charge batch, not a customer master. See "
"each group's decided rule.")
p("") p("")
# ------------------------------------------------------------------ # # ------------------------------------------------------------------ #
@@ -88,24 +115,33 @@ def main() -> None:
m = ef[ef['folio'].isin(both_folio)].drop_duplicates('folio').set_index('folio') m = ef[ef['folio'].isin(both_folio)].drop_duplicates('folio').set_index('folio')
n = efb[efb['folio'].isin(both_folio)].drop_duplicates('folio').set_index('folio') n = efb[efb['folio'].isin(both_folio)].drop_duplicates('folio').set_index('folio')
ci = m.index.intersection(n.index) ci = m.index.intersection(n.index)
folio_conflict = int((m.loc[ci, biz] != n.loc[ci, biz]).any(axis=1).sum()) mk = pd.DataFrame({c: canon(m.loc[ci, c]) for c in biz})
nk = pd.DataFrame({c: canon(n.loc[ci, c]) for c in biz})
folio_conflict = int((mk != nk).any(axis=1).sum())
p(f"- **But of those {len(ci)} shared folio numbers, {folio_conflict} carry " p(f"- **But of those {len(ci)} shared folio numbers, {folio_conflict} carry "
f"a *different* transaction** (differ on {biz}). → `folio` collides; it is " f"a *different* transaction** (differ on {biz}). → `folio` collides; it is "
"NOT a stable cross-table id.") "NOT a stable cross-table id, and it cannot be the de-dup key.")
b, oa, ob = overlap(ef, efb, biz) b, oa, ob = overlap(ef, efb, biz)
p(f"- On the real business key `{biz}`: **{b} rows in both**, {oa} only in " p(f"- On the real business key `{biz}`, canonicalized: **{b} rows in both**, "
f"EFECTIVO, {ob} only in BACKUP.") f"{oa} only in EFECTIVO, {ob} only in BACKUP — i.e. all but {ob} of "
p("- Date ranges are different eras: BACKUP is dominated by 20172022 " f"BACKUP's {len(efb)} rows already exist verbatim in EFECTIVO, same "
"records, EFECTIVO by recent ones.") "customer, same timestamp to the second, same amount, same concept text.")
p("- Yearly row counts track each other almost exactly from 2006 to 2023 "
"(e.g. 2017: 1036 vs 994), which is what a stale copy looks like — not "
"two ledgers covering different eras.")
p("") p("")
p("**Decided rule:** the two tables are **near-disjoint ledgers**, not a " p("**Decided rule (corrected):** `EFECTIVO_BACKUP` is a **stale backup copy "
"live/backup duplicate pair (only " + str(b) + " shared payments out of " "of `EFECTIVO`**, not an independent ledger. Load `EFECTIVO` in full, and "
"~13k+12k). Migrate **both** into `transactions`, each row keyed " "load from `EFECTIVO_BACKUP` only the rows whose canonicalized business "
"internally by `(legacy_source_table, folio)` provenance — do **not** " f"key is absent from `EFECTIVO` ({ob} row(s)). Loading both in full "
"de-dup on `folio` (it collides) and do **not** drop BACKUP (it holds " f"double-counts {b} payments and doubles nearly every customer's "
"~12k older payments absent from EFECTIVO). The " + str(b) + " business-" "historical receipt total, which makes any statement or balance view "
"key matches are the only possible double-counts and should be spot-" "wrong. De-dup on the business key, **not** on `folio` (it collides).")
"checked, but at that volume they don't threaten balance integrity.") p("")
p("> This reverses the original verdict in this report, which put that "
"overlap at 2. That number came from string-comparing `monto`, which `mdb-export` "
"serializes with a different precision per table (`5000` vs "
"`27000.0000`) — see the module docstring.")
p("") p("")
p("> `EFECTIVO FM3` (627) and `CHEQUE FM3` (157) are a separate stream — " p("> `EFECTIVO FM3` (627) and `CHEQUE FM3` (157) are a separate stream — "
"`fee`/`tax`/`multa` columns instead of `monto` — and migrate as distinct " "`fee`/`tax`/`multa` columns instead of `monto` — and migrate as distinct "
+41 -6
View File
@@ -4,7 +4,12 @@ Migration plan step 3 (shared ledger): unify every cash/billing ledger into
Runs AFTER transform_customers.py (customer FK is required). Runs AFTER transform_customers.py (customer FK is required).
Union rules come from the reconciliation pass (RECONCILIATION.md): Union rules come from the reconciliation pass (RECONCILIATION.md):
- EFECTIVO + EFECTIVO_BACKUP -> both (near-disjoint ledgers; no folio de-dup) - EFECTIVO + EFECTIVO_BACKUP -> EFECTIVO in full, plus only the BACKUP rows
whose business key is absent from EFECTIVO
(BACKUP is a stale copy: 12386 of its 12387
rows are verbatim duplicates). De-dup on the
business key, never on folio (folio collides
across the two tables).
- datos2 + FEE ANUAL + fee15 -> all three (disjoint period runs; no de-dup) - datos2 + FEE ANUAL + fee15 -> all three (disjoint period runs; no de-dup)
- EFECTIVO FM3 / CHEQUE FM3 -> distinct fee stream (amount = fee+tax+multa) - EFECTIVO FM3 / CHEQUE FM3 -> distinct fee stream (amount = fee+tax+multa)
- IVA 2015 -> its own snapshot (no date column -> nominal) - IVA 2015 -> its own snapshot (no date column -> nominal)
@@ -117,7 +122,7 @@ def main():
# --- transactions --- # --- transactions ---
tx = [] tx = []
skip_cust = skip_date = 0 skip_cust = skip_date = skip_dupe = 0
def add(cid, domain, tdate, amount, currency, *, period=None, reference=None, def add(cid, domain, tdate, amount, currency, *, period=None, reference=None,
typeid=None, check=None, message=None, src_db=None, src_tbl=None, legacy=None): typeid=None, check=None, message=None, src_db=None, src_tbl=None, legacy=None):
@@ -125,10 +130,33 @@ def main():
amount if amount is not None else Decimal(0), currency, None, check, amount if amount is not None else Decimal(0), currency, None, check,
message, 0, src_db, src_tbl, legacy)) message, 0, src_db, src_tbl, legacy))
def efectivo_like(src, name, domain, custmap, src_db, legacy_tbl): # Business key of a real cash payment. `folio` is deliberately excluded: it
nonlocal skip_cust, skip_date # is a per-table sequential number that collides between EFECTIVO and
# EFECTIVO_BACKUP (12363 shared numbers, 12204 of them on different
# payments), so it identifies nothing across tables.
def biz_key(r):
return (
norm_id(r["cl"]),
s(r["fecha"]),
dec(r["monto"], Decimal(0)),
s(r["conepto"]),
)
def efectivo_like(src, name, domain, custmap, src_db, legacy_tbl, *, seen=None):
"""Load an EFECTIVO-shaped cash ledger.
`seen` (a set) makes the load de-duplicating: keys are added to it as
rows load, and a row whose key is already present is skipped. That is
how EFECTIVO_BACKUP contributes only its genuinely-new rows.
"""
nonlocal skip_cust, skip_date, skip_dupe
df = load(src, name) df = load(src, name)
for _, r in df.iterrows(): for _, r in df.iterrows():
if seen is not None:
key = biz_key(r)
if key in seen:
skip_dupe += 1; continue
seen.add(key)
cid = custmap.get(norm_id(r["cl"])) cid = custmap.get(norm_id(r["cl"]))
if not cid: if not cid:
skip_cust += 1; continue skip_cust += 1; continue
@@ -181,8 +209,14 @@ def main():
reference=s(r["recibo"]), message="IVA 2015", reference=s(r["recibo"]), message="IVA 2015",
src_db="UTILITIES", src_tbl="IVA 2015", legacy=str(int(r["_row_num"]))) src_db="UTILITIES", src_tbl="IVA 2015", legacy=str(int(r["_row_num"])))
efectivo_like("stg_utilities", "efectivo", "UTILITY", util_cust, "UTILITIES", "EFECTIVO") # Shared across both calls so BACKUP is de-duplicated against EFECTIVO —
efectivo_like("stg_utilities", "efectivo_backup", "UTILITY", util_cust, "UTILITIES", "EFECTIVO_BACKUP") # order matters: EFECTIVO is the live table and loads first, so a collision
# always resolves in its favour.
cash_seen: set = set()
efectivo_like("stg_utilities", "efectivo", "UTILITY", util_cust, "UTILITIES",
"EFECTIVO", seen=cash_seen)
efectivo_like("stg_utilities", "efectivo_backup", "UTILITY", util_cust, "UTILITIES",
"EFECTIVO_BACKUP", seen=cash_seen)
fm3("efectivo_fm3", "EFECTIVO FM3") fm3("efectivo_fm3", "EFECTIVO FM3")
fm3("cheque_fm3", "CHEQUE FM3", check_col="num_cheque") fm3("cheque_fm3", "CHEQUE FM3", check_col="num_cheque")
billing("datos2", "datos2") billing("datos2", "datos2")
@@ -217,6 +251,7 @@ def main():
print("=== Transactions load complete ===") print("=== Transactions load complete ===")
print(f" skipped (unresolved customer): {skip_cust}") print(f" skipped (unresolved customer): {skip_cust}")
print(f" skipped (unparseable date) : {skip_date}") print(f" skipped (unparseable date) : {skip_date}")
print(f" skipped (EFECTIVO_BACKUP dup): {skip_dupe}")
print(f" -> transactions : {count('transactions')}") print(f" -> transactions : {count('transactions')}")
print(f" by domain : {dict(by_dom)}") print(f" by domain : {dict(by_dom)}")
for src, n in by_src: for src, n in by_src: