feat(billing): receipt capture — outstanding workflow, batch by check, reconciliation
Implements docs/RECEIPT_CAPTURE_SPEC.md §1, the legacy "Editor" replacement, on top of the single-movement capture from plan step 6. No new abilities: batching and resolving are both capturing. - outstanding (legacy NOPAGO): capture flag, ?outstanding= filter, and POST /billing/:id/resolve-outstanding (gated ledger:create, not ledger:void — resolving completes a capture rather than reversing one). Outstanding rows are excluded from every balance aggregate, matching the legacy SALDOS ULTIMO 0 query's HAVING NOPAGO = 0, but still count in the movement browser's filtered totals. - POST /billing/batch: many customers' receipts against one check, in one $transaction. Deliberately not a persisted batch entity — checkNumber is already a column and grouping by it answers every legacy by-check query. - GET /billing/by-check + a cheque-count report, replacing REPORTE CHEQUE COUNT / REPORTE POR CHEQUE / EDITA CHEQUE ALF|COUNT|NUM. Print, PDF, CSV and XLSX come free from the existing /reportes/:slug machinery. - Web: /estado-cuenta/lote (the Editor screen, with live reconciliation against the physical check amount), an "Estado de pago" filter, a "sin fondos" row tag and a Resolver dialog, plus a top-level "Captura" nav entry. Integration seam for the OCR auto-capture module (spec §2), which is required to post through createBatch rather than writing Transaction rows itself: items[i] maps to lines[i] so postedTransactionId can be zipped back on; opts.refs[i] stamps captureRef with a duplicate-post guard that a voided row deliberately does not block; opts.source is service-level only, so an HTTP client cannot label hand-keyed rows as machine-captured. captureSource/captureRef are nullable so the 40,136 migrated rows stay NULL rather than being mislabelled. Fixes two pre-existing bugs found while building this: - statement() filtered legacySourceTable with `notIn`, which compiles to SQL NOT IN — and `NULL NOT IN (...)` is NULL, so every app-captured movement was invisible on the customer statement (438 rows in the movement browser vs 392 on the statement) while showing everywhere else. This would have made the whole capture feature look broken. - The balances count query omitted the void filter its own page query applied, so the total disagreed with the rows. Nav highlighting now resolves by longest match; the previous first-startsWith logic lit up both the parent and any nested entry. Verified end-to-end against the dev DB, API and browser; all test rows removed afterwards. Also corrects RESUME.md, which documented the dev ports as :3001/:3000 — they are :4501/:4500, from the env files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -131,12 +131,23 @@ Given the amount of near-duplicate/overlapping data across snapshot tables (mult
|
||||
9. Sync worker (push replicated tables' relevant subset, poll inbox tables for payment/propane submissions) — depends on step 8. **The separate Phase B Access additive sync is implemented:** `migration/run_all.py --sync` and the admin `SYNC` job upsert legacy-owned rows without truncating the database or touching manual rows. Portal write points confirmed present in `utility_dbo`: `peticion_gas` (propane requests), PayPal payment writes, `notifications_settings`, `verification_codes` — these define the VPS→internal inbox set.
|
||||
10. Reports/email campaigns/admin — parity with old app's `reports.php`/`emailCampaigns.php` intent, rebuilt properly.
|
||||
11. **Receipt capture ("Editor") completion + three net-new ops features — NOT STARTED, spec written.** Full design in [`docs/RECEIPT_CAPTURE_SPEC.md`](docs/RECEIPT_CAPTURE_SPEC.md), from the 2026-07-25/26 meeting with Jorge:
|
||||
- **Receipt capture module** — the legacy "Editor" replacement: wires up the already-existing but unused `Transaction.outstanding` field (NOPAGO workflow), adds batch-by-check capture, and a check-reconciliation view replacing `REPORTE CHEQUE COUNT`. Builds directly on the single-movement capture already shipped in `billing/` (step 6) — smallest piece, do first.
|
||||
- **Receipt capture module — DONE** (2026-07-27). The legacy "Editor" replacement, built on the single-movement capture from step 6. Wires up the previously-unused `Transaction.outstanding` (NOPAGO): capture flag on `POST /billing`, `?outstanding=` list filter, `POST /billing/:id/resolve-outstanding` (gated `ledger:create`, not `ledger:void` — resolving *completes* a capture), and exclusion from every balance aggregate exactly as the legacy `SALDOS ULTIMO 0`'s `HAVING NOPAGO = 0` did. Adds `POST /billing/batch` (one `$transaction`, check-level fields shared, per-line customer/amount) and `GET /billing/by-check`, plus the `cheque-count` report replacing `REPORTE CHEQUE COUNT` / `REPORTE POR CHEQUE` / `EDITA CHEQUE ALF|COUNT|NUM` — print/PDF/CSV/XLSX come free from the existing `/reportes/:slug` machinery. Web: `/estado-cuenta/lote` (the actual "Editor" screen, with live reconciliation against the physical check amount), plus an "Estado de pago" filter, a "sin fondos" row tag and a Resolver dialog on `/estado-cuenta`. No new abilities. Verified end-to-end against dev, API + browser.
|
||||
**Two pre-existing bugs found and fixed while building it:** (a) `statement()` filtered `legacySourceTable: { notIn: [...] }`, which compiles to SQL `NOT IN` — and `NULL NOT IN (…)` is NULL, so **every app-captured movement was invisible on the customer statement** (438 rows in the movement browser vs 392 on the statement) while still appearing everywhere else. This would have made the whole receipt-capture feature look broken to staff. Now NULL-safe. (b) The balances *count* query omitted the void filter its own page query applied, so the row count disagreed with the rows.
|
||||
**OCR seam:** `BillingService.createBatch(dto, opts)` is the single multi-row write path and carries three contract guarantees for the step-11 OCR module to post through — `items[i]` maps to `lines[i]` (so `StatementDocument.postedTransactionId` can be zipped back on), `opts.refs[i]` stamps `captureRef` with a duplicate-post guard that a *voided* row deliberately does not block, and `opts.source` is service-level only so an HTTP client cannot label hand-keyed rows as machine-captured. Backed by a new `TransactionCaptureSource` enum (MANUAL/BATCH/OCR) + `captureRef`, both nullable so the 40,136 migrated rows stay NULL rather than being mislabelled.
|
||||
- **PDF/OCR auto-capture** — ingest→split→OCR→match→review pipeline for the 300+/month/service-provider statements staff currently key in by hand. Posts through the capture module above. Matching logic was checked field-by-field against `migration/transform_properties.py`'s actual output and found three real gaps to close first: no `TELEPHONE` service kind exists yet, `PROPERTY_TAX.accountNumber` was migrated from `PREDIAL` not `CLAVE` (needs verification against a real predial statement), and `GAS.meterNumber` was never populated by the migration at all.
|
||||
- **Multi-bank chequera** — `Bank`/`BankAccount` models so Seguros (US bank) and Utilities (Mexican bank, currently SCOTHIA) can each have their own register; today's `bank_transactions` is hardcoded single-account/MXN-only by design (see step 7 above) and needs a required `bankAccountId` plus scoping added to every read path in `bank.service.ts`, including two raw-SQL queries in `summary()`.
|
||||
- **Customer-number recycling** — promotes the legacy `NUM id` (currently only inside `customer_legacy_refs`) into a first-class, reusable `Customer.customerNumber`, automates *finding* candidates for reuse (cancelled / 1-year-inactive), and auto-assigns the lowest free number at creation — the search is automated, the release/reuse decision stays a human action. Backfill needs care: ~140 utilities rows and all insurance-only customers have no real legacy number (synthetic `rownum_N`/`insrow_N` placeholders in `transform_customers.py`, not real `NUM id`s).
|
||||
|
||||
Several open questions block parts of this (OCR provider/budget, the Seguros bank's identity, the clave-catastral-vs-predial mismatch, exact recycling triggers, and whether "recycling" should ever mean true data purge vs. archive-and-reuse-the-number) — see the spec's collected open-questions section.
|
||||
12. **Insurance features — NOT STARTED, spec written.** Full design in [`docs/INSURANCE_FEATURES_SPEC.md`](docs/INSURANCE_FEATURES_SPEC.md), the insurance half of the same 2026-07-25/26 meeting with Jorge that produced step 11:
|
||||
- **Renewal notification emails** — a daily `@nestjs/schedule` sweep that mails the customer 30 days before expiry, 15 days before, and 7 days after, mapping onto `RenewalNotice.generation` 1/2/3 with **no schema change**. Sending is **Amazon SES** (`@aws-sdk/client-sesv2`, mirroring `StorageService`'s optional-client/degrade-don't-crash pattern) — the office already runs SES, so provider and budget are settled, not open. The letter body is the *existing* `aviso-renovacion` report (`reports.registry.ts:623-799`); `@@unique([policyId, generation])` is already-in-place idempotency, so a re-run cannot double-send. Volume ≈260 mails/month, and **815 of the 893 policyholders (91%) have an email**. Also adds the manual mark-as-sent mutation the report's own comment anticipates, so the report's permanently-zero `enviadas` total becomes real. Smallest useful piece — do first.
|
||||
- **Liquidación batch workflow** — ~70% already built (`liquidated`/`liquidationNumber`/`liquidationDate` are wired through DTOs, list filter, stats, form and detail page); only the *batch* print-and-mark step is missing, against a live pending set of 226 policies. Adds a ramo-parameterized pending report plus `POST /policies/liquidate-batch` under a new MANAGER `policy:liquidate` ability. Parameterized by ramo, not MULT-only — legacy `TABLA LIQUIDA MF` served `MULT`, `INCENDIO` and `M EMPR` alike.
|
||||
- **Certificate / "Solicitud Atlas"** — renders from the same `format: "letter"` machinery `aviso-renovacion` uses, then reaches customers as an extension of the step-8/9 replication (PDF generated here, pushed to MinIO, pointer replicated), **not** as a new public surface in this repo. Half-blocked: "Solicitud" has zero referent in the legacy system and normally means an *application form*, a different artifact from a certificate.
|
||||
- **Carrier API integration (ANA Seguros + GMX)** — shape only (`CarrierConnector` + an import-review queue rather than direct `Policy` writes, matching how step 11's OCR results are routed). Carrier research done 2026-07-27: **the two carriers are one company** — both belong to **Grupo Valore** (ANA writes autos, GMX writes daños, which is exactly this database's `AUTO`/`LICENCIAS` vs `MULT`/`INCENDIO`/`M_EMPR` split), so it is one commercial relationship, not two. **ANA has a real live SOAP service** (`server.anaseguros.com.mx/ananetws/service.asmx`, ASP.NET `.asmx`) with a published operation list — catalogs, `CalculaValor`/`CalculaMSI`, `ValidaSerie`, `RecuperaCotizacion`, `Transaccion`. **GMX publishes no machine interface at all**, only human agent portals. ⚠️ **Critical mismatch:** every ANA operation serves *new-business quoting/issuance*, not "list the policies where I am agent of record" — so if the ask is inbound portfolio sync, no evidence exists that either carrier sells it. Blocked on one phone call to Grupo Valore ((55) 5480-4000) for credentials + a direction answer, not on further research. ("GDMX" in the meeting notes was a typo for `GMX` — confirmed 2026-07-27.)
|
||||
|
||||
**Two pre-existing defects were found while verifying this spec and should be fixed as part of the liquidación work:** (a) `policy_types` is missing its `INCENDIO` and `M_EMPR` rows and, because `policies_policyTypeId_fkey` is `ON DELETE SET NULL`, 5 `m_empr` policies silently lost their ramo — 4 of them are pending liquidación and are invisible to every ramo-filtered query; (b) the legacy settlement slots don't match what the target model assumed — `MULT`/`INCENDIO` carry two and `M EMPR` carries four, while `Policy` collapses to one, so ≤41 MULT second settlements were dropped in migration. Spec recommends moving settlement onto `PolicyPaymentInstallment` rather than adding a second slot.
|
||||
|
||||
**One long-standing open question is closed by this spec:** `DATGRAL.[NUM UTIL]` is authoritative for Utilities↔Seguros reconciliation and **`UTILSEG` must not be used** — its numbers resolve to unrelated people under every reading tested (name match 58/1,024 vs. 298/563 for `NUM UTIL`), and where the two sources overlap they contradict each other on 170 of 218 shared ids. This matters to step 11's customer-number recycling, which touches the same identity space.
|
||||
|
||||
## Status
|
||||
|
||||
@@ -148,6 +159,8 @@ Repo scaffolded at `jorgecuadros-platform/`: npm workspaces, NestJS API with a r
|
||||
|
||||
**Step 11 spec written, not built.** `docs/RECEIPT_CAPTURE_SPEC.md` covers the receipt-capture ("Editor") completion plus the three net-new ops features (OCR auto-capture, multi-bank chequera, customer-number recycling) — see Build sequencing step 11 above for the summary. Written from the 2026-07-25/26 meeting notes and verified against the real migration scripts and current API code, not just designed from the meeting notes alone.
|
||||
|
||||
**Step 12 spec written, not built.** `docs/INSURANCE_FEATURES_SPEC.md` covers the insurance half of the same meeting (renewal emails, liquidación batch, certificate + portal delivery, carrier APIs) — see Build sequencing step 12 above. Verified the same way, plus a live query of the dev DB for the counts it quotes (email coverage, pending liquidación, installment fill rates) and of the staged Parquet for the legacy settlement-slot usage. Two of the four features are much smaller than they sound: the renewal-notice table, its idempotency key and the letter body already exist, and the per-policy liquidación fields are already wired end to end.
|
||||
|
||||
## Decisions (locked)
|
||||
|
||||
- **Stack:** Next.js + NestJS + Prisma + **MySQL** (locked earlier — see engine rationale above).
|
||||
@@ -164,9 +177,11 @@ Repo scaffolded at `jorgecuadros-platform/`: npm workspaces, NestJS API with a r
|
||||
- **VPS provisioning:** provider (Hetzner vs DigitalOcean), size, and Tailscale + MySQL replica setup on it — an ops task, still pending. Design is settled; only the box is missing.
|
||||
- **Old external-DB credential** (hardcoded plaintext MySQL password in the old repo's `dbConnection.php`, in git history) — rotate it regardless, since it's already exposed.
|
||||
|
||||
## Open design questions (step 11 — need Jorge before/while building)
|
||||
## Open design questions (steps 11 & 12 — need Jorge before/while building)
|
||||
|
||||
Unlike the ops items above, these block design decisions, not just infrastructure. Full detail in each section of `docs/RECEIPT_CAPTURE_SPEC.md`:
|
||||
Unlike the ops items above, these block design decisions, not just infrastructure. Full detail in each section of `docs/RECEIPT_CAPTURE_SPEC.md` (step 11) and `docs/INSURANCE_FEATURES_SPEC.md` (step 12):
|
||||
|
||||
**Step 11 — utilities/ops side:**
|
||||
|
||||
- OCR provider/budget for the statement auto-capture pipeline (self-hosted vs. a paid per-page API, given 300+ statements/month/service provider).
|
||||
- Whether `PROPERTY_TAX.accountNumber` (migrated from `DATMEX.PREDIAL`) is actually the same number as "Clave Catastral" (`DATMEX.CLAVE`) — blocks OCR matching for predial statements until confirmed against a real bill.
|
||||
@@ -174,6 +189,17 @@ Unlike the ops items above, these block design decisions, not just infrastructur
|
||||
- The exact "1 year inactivity" / "cancelled" triggers for customer-number recycling eligibility.
|
||||
- Whether customer-number recycling should ever include true PII purge (matching the office's paper-world habit) or archive-and-reuse-the-number is sufficient — recommended default is archive-only, consistent with this project's existing never-hard-delete convention.
|
||||
|
||||
**Step 12 — insurance side:**
|
||||
|
||||
- Which SES region + verified sending identity/configuration set the renewal mail goes out under, and whether it reuses the existing IAM credentials or gets its own scoped `ses:SendEmail` user. (Provider and budget are *not* open — SES is settled.)
|
||||
- What to do with the 78 policyholders who have no email on file: skip silently, or produce a print worklist? Recommended: the worklist, since `aviso-renovacion` already renders exactly those letters.
|
||||
- Whether renewal notices go out in Spanish or English — `Customer` carries no language preference.
|
||||
- What "garantías" refers to — it has zero referent in the legacy data, and it blocks the liquidación batch's exclusion filter.
|
||||
- Whether policy settlement should move onto `PolicyPaymentInstallment` (recommended) or gain a second slot on `Policy`, and whether to backfill the ≤41 MULT second settlements lost in migration.
|
||||
- Whether batch liquidación warrants a new MANAGER-level `policy:liquidate` ability (recommended) or should reuse the existing STAFF-level `policy:update`.
|
||||
- **What "Solicitud Atlas" actually is** — an application form or a certificate. These are different artifacts with different data and timing; this blocks the whole certificate feature.
|
||||
- **Carrier integration direction** — outbound quote/issue (which ANA's SOAP service supports today) or inbound sync of the office's existing book (which nothing found suggests either carrier offers)? This decides whether the feature is buildable at all. Bundle with the other three carrier questions into one call to Grupo Valore ((55) 5480-4000): WSDL + credentials for the ANA service, whether a cartera/portfolio download exists for an agent's own book, whether GMX daños has any machine interface, and whether one credential spans both carriers. ("GDMX" is resolved — it was a typo for `GMX`.)
|
||||
|
||||
## Verification
|
||||
|
||||
- Migration: automated row-count/sum reconciliation between `staging` and final schema per table group (see step 5 above), run as part of the migration script, not a manual spot-check.
|
||||
|
||||
Reference in New Issue
Block a user