Initial scaffold: unified customer/insurance/utilities platform
Next.js + NestJS + Prisma (MySQL) monorepo replacing the legacy PHP internal app. Includes a session-based auth module with Argon2 password hashing and global input validation (replacing the old app's SQL injection and plaintext password comparison), the full target Prisma schema for customers/insurance/utilities/shared ledger/bank register, Docker Compose + Dockerfiles, and an Access-to-staging migration pipeline (migration/) already run against the real source databases. See PLAN.md and RESUME.md for the full architecture and session history.
This commit is contained in:
@@ -0,0 +1,5 @@
|
||||
# Copy to .env and fill in for local development.
|
||||
DATABASE_URL=mysql://jorgecuadros:jorgecuadros@localhost:3306/jorgecuadros
|
||||
SESSION_SECRET=change-me-to-a-random-string
|
||||
WEB_ORIGIN=http://localhost:3000
|
||||
NEXT_PUBLIC_API_ORIGIN=http://localhost:3001
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
node_modules/
|
||||
.next/
|
||||
dist/
|
||||
build/
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
*.log
|
||||
migration/output/
|
||||
migration/.venv/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
packages/database/generated/
|
||||
.DS_Store
|
||||
@@ -0,0 +1,145 @@
|
||||
# Unified Customer / Insurance / Utilities Platform — Migration & Rebuild Plan
|
||||
|
||||
## Context
|
||||
|
||||
Jorge Cuadros & Assoc. runs two lines of business — property/utility management (`UTILITIES.accdb`) and insurance brokerage (`SEGUROS 16.mdb` + its linked backend `SEGUROS 16_be.mdb`) — out of separate, decades-old MS Access databases, plus a third file (`SCOTHIA.mdb`) that's the office's own Scotiabank checking-account register ("chequera"). The same people are customers of both business lines, but today there's no shared customer record: a person's utility account and their insurance policies live in unrelated systems with independent, inconsistent copies of their name/address/contact info. The bank register is a fourth, disconnected source of truth for the money actually moving through the office's own account.
|
||||
|
||||
An earlier attempt to modernize this (`jorgecuadros-intra-webapp`, PHP) got partway there — it correctly recognized that customers should be unified via a bridge table and began normalizing the flat Access tables into real relational tables (`properties`, `policies`, `home_policies`). But every query in its data layer builds SQL by string-concatenating `$_POST` directly (`src/core/db.php`), so it's SQL-injectable end to end, and `src/core/auth.php` compares passwords in plaintext with no hashing. Per your decision, this isn't worth patching — the app layer will be rebuilt from scratch, in a different stack, in a new repo. The Access data and the schema ideas from the old MySQL dump remain the reference for what the business actually needs.
|
||||
|
||||
Goal: one system with a single customer record, from which staff can see and manage that customer's utility services *and* insurance policies *and* shared billing/transaction history, replacing both Access files.
|
||||
|
||||
**Confirmed decisions:**
|
||||
- Stack: Next.js (React + TypeScript) frontend, TypeScript backend (NestJS), **MySQL**, Prisma ORM.
|
||||
- New repo, not built on top of `jorgecuadros-intra-webapp` (that repo is reference-only for business logic/field mappings).
|
||||
- Full historical data migrates — no cutoff. Source tables are small (largest is ~16k rows), so completeness costs little.
|
||||
|
||||
**Database engine — revised from PostgreSQL to MySQL.** The internal platform doesn't stand alone: a separate, pre-existing customer-facing PHP/MySQL portal (mobile-app-backed — see Infrastructure & Sync below) needs to keep reading live account data from it, and that portal's PHP code uses `mysqli` and is staying as-is (not being rewritten). Running the internal platform on Postgres would mean maintaining a cross-engine sync into MySQL just to feed the portal; using MySQL everywhere removes that translation layer entirely. Prisma supports MySQL natively, so this only changes the `datasource` provider in the schema — NestJS/Next.js/Prisma all stay.
|
||||
|
||||
## Infrastructure & sync architecture
|
||||
|
||||
The internal platform and the customer-facing portal are two genuinely separate deployments that need to stay in sync, not one app:
|
||||
|
||||
- **Internal server** — hosted on-prem, private IP (`192.168.1.xx`), **not exposed to the internet**. Runs `jorgecuadros-platform` (this rebuild) and the canonical MySQL database — source of truth for everything staff manage (customers, policies, properties, the full ledger).
|
||||
- **Customer-facing portal** — an existing PHP app (with a companion mobile app — its usage-tracking schema, `jorgecuadros_app` at `mysql.freakma.com`, logs sections like `LOGIN`, `STATEMENT_ALL_NOW`, `MAKE_PAYMENT`, `ORDER_PROPANE`, and a `devices` table of push-notification tokens) running on shared hosting. **Out of scope to rebuild** — stays exactly as it is. It reads/writes a separate live database, `utility_dbo` at `mysql.freakma.com`, which the old internal app already connects to externally (`src/core/dbConnection.php: getExternalDBConnection()`) — though only for `email_alert_log` in that codebase; the mobile app almost certainly talks to `utility_dbo` directly for statements/payments/propane orders. **Open item:** no schema dump of `utility_dbo` itself exists yet — only this reference to it — so the sync worker's exact target tables/columns can't be finalized until that's available (export or read-only credentials), the same way the Access files and the `jorgecuadros_app` dumps were provided.
|
||||
- **New VPS** (Hetzner or DigitalOcean, to be provisioned) — runs a MySQL instance and becomes what `mysql.freakma.com` resolves to. Because it's infrastructure you control (unlike the shared-hosting account), it can run **real native MySQL replication** — the shared-hosting limitation only ever applied to shared hosting itself, not to a VPS. The shared-hosting portal's PHP code needs no changes beyond the DNS target it already points at.
|
||||
- **Internal server ↔ VPS network path:** Tailscale (WireGuard mesh) — both machines join the same tailnet, giving the internal server a way to reach the VPS (and vice versa) without opening any inbound port on either the internal LAN or needing the internal server to be internet-facing.
|
||||
|
||||
**Sync mechanism:**
|
||||
- **Internal → VPS (statements, balances, customer profile changes):** one-way native MySQL replication (binlog/GTID-based) from the internal MySQL (source) to the VPS MySQL (replica), over the Tailscale link. Standard, well-supported, and only carries the subset of tables/columns the portal actually needs to read — internal-only data (staff notes, adjuster info, activity logs) should live in tables intentionally excluded from what's replicated, so the more internet-exposed side never receives more than the portal requires.
|
||||
- **VPS → Internal (payment submissions, propane order requests):** one-way replication can't carry writes back, and multi-master MySQL replication is fragile enough to avoid for a system this size. Instead: a small set of unreplicated **inbox tables** on the VPS (`payment_submissions`, `propane_order_requests`, matching whatever `utility_dbo` actually uses once its schema is available) that the portal writes to directly, polled every 1–5 minutes by a worker on the internal server (over Tailscale) that turns new rows into real `Transaction`/propane-request records and marks them processed.
|
||||
|
||||
## Source data inventory
|
||||
|
||||
Extracted live via `pyodbc` + the Windows Access ODBC driver (`dump_schema.py`, tables/columns/row counts/PKs for all four files). Access has **no real primary or foreign keys defined anywhere** — every relationship below is inferred from column-name conventions (`NUM id`, `NUMid`, `NUMER ID`, `IDISEG`, `IDIUT`), not enforced constraints, so the migration has to independently validate every join.
|
||||
|
||||
**UTILITIES.accdb** (52 tables, ~538MB, dominated by embedded document blobs):
|
||||
- `DATMEX` (1,520 rows) — one row per property: address, phones, and inline utility fields for water/electric/gas/cable/property-tax/federal-zone/trust, each with its own due-date/route/account-number columns, plus 6 `LONGBINARY` columns holding scanned utility bills/IDs.
|
||||
- `DATGRAL` / `COBRO3` (1,172 / 181 rows) — customer master data (name, MX + US address, phone, email, ID document, client-since date, fee, status). `COBRO3` looks like a filtered snapshot of `DATGRAL`, not a distinct entity.
|
||||
- `PROFILE` (1,520) — per-property service enrollment flags, joins 1:1 with `DATMEX` by `NUMERID`.
|
||||
- `EFECTIVO` (13,697), `EFECTIVO FM3` (627), `EFECTIVO_BACKUP` (12,387), `CHEQUE FM3` (157) — cash/check transaction ledgers, near-identical shape, apparent year/program snapshots rather than distinct data.
|
||||
- `FEE ANUAL` (1,082), `datos2` (16,000), `fee15` (1,030), `billing` (0) — recurring billing/fee transaction logs, again near-duplicate shapes across different periods/exports — needs de-duplication logic, not a straight union.
|
||||
- `TRUSTVENCE` (549) — bank trust account fee due-dates.
|
||||
- `TIPO HIST` (2,301) — exchange-rate history (date/hour/rate), referenced by the `MONEDAS` (currency) field used throughout.
|
||||
- `TYPE OF TRX` (79) — ES/EN transaction-type lookup (already mirrored as `type_transactions` in the old MySQL schema — reuse that mapping).
|
||||
- Scratch/working tables to **exclude** from migration: `BANCO EDITOR`, `Errores de pegado`, `TABLE1`, `PARA BILLING*`, `FALTANTES *`, `TELEFONOS FECHAS`, `TIT`, `BORRA`-equivalents. A few tables (`PROPANO`, `datosfreak`, `datosfreak2`, `LUZ TODOS`, etc.) have column names with characters pyodbc's UTF-16 path can't decode — need re-extraction with a Latin-1/CP1252 fallback before they can even be classified as real data vs. scratch.
|
||||
|
||||
**SEGUROS 16_be.mdb** (64 tables, ~882MB; `SEGUROS 16.mdb` is an empty Access "frontend" shell — all real data lives in `_be`):
|
||||
- `DATGRAL` (1,070 rows) — customer master, **same shape as the utilities `DATGRAL`**, and critically has a `NUM UTIL` column — a literal cross-reference to the utilities customer's `NUM id`. This is the join key for building the unified customer table; it's also what the old app's `customer_mapping` table was clearly reverse-engineered from.
|
||||
- One flat table per insurance line of business, all sharing the same repeated-column pattern (policy #, coverage dates, 4 hardcoded payment installments each with its own date/amount/check#/currency, premium breakdown, liquidation status, address, observations, embedded document blobs):
|
||||
- `INCENDIO` (fire), `MULT` (multi-risk/home — the most complete, matches old app's `home_policies`), `M EMPR` (commercial property)
|
||||
- Auto: `TABLA AUTOS`, `TABLA AUTOS AMPL`, `TABLA AUTOS AMPL R`, `TABLA AUTOS LIMIT`, `TABLA AUTOS LIMIT R`, `TABLA AUTOS RC R`, `MCA2` — seven variants of essentially one "auto policy" concept, differentiated by coverage tier/program. `MCA2` also hardcodes **3 driver+vehicle slots as repeated columns** (`MARCA1/2/3`, `PLACA1/2/3`, `NAME1/2/3`, `LIC1/2/3`...) that need to unpivot into child rows.
|
||||
- `LICENCIAS` (driver's-license insurance, MX-specific product) — also hardcodes 3 named insureds per policy.
|
||||
- `BENEF` (768) — policy beneficiaries, already a clean child table (policy#, name, address, phone, email).
|
||||
- `DATOS` (2 rows now, but structurally real) — claims/siniestros: claim date, description, adjuster, settlement amounts, checks, documents.
|
||||
- `AJUSTADORES` / `AJUSTADORESATLAS` — adjuster contact lists.
|
||||
- `EDOSEG` — per-policy premium/account statement summary.
|
||||
- `EFECTIVO` (295) — cash ledger, same shape as the utilities one.
|
||||
- `UTILSEG` (1,582) — appears to be the utilities↔insurance customer cross-reference table.
|
||||
- `TABLA LIQUIDA *` / `TABLE DATOS LIQUIDA *` — settlement/liquidation batches per policy line (matches `liquidation_number` already in the old MySQL `policies` table).
|
||||
- **Exclude from data migration:** the `* MENS` / `*MENSAJE` tables (`AMPL MENS`, `IN MENS`, `LIC MENS`, `MCA2 MENS`, `ME MENS`, `MF MENS`, `RC MENSAJE`, `RC R MENS`) are mail-merge document *templates* (letters, certificates) stored as blobs, not customer data — these get reimplemented as PDF templates in the new app, not migrated as rows. `TODOSJC`, `TODOS`, `vigenta casa y auto unicos` look like materialized Access query results (saved reports), not source-of-truth data — exclude, and if the report itself is still needed, rebuild it as a real query against the new schema.
|
||||
|
||||
**SCOTHIA.mdb** (7 tables, ~3MB — the office's own Scotiabank checking-account register, "chequera"). Much simpler than the other two files: this is the company's operating bank account, not customer-facing data.
|
||||
- `DATOS E` (15,406 rows) — expenses/outgoing (`EGRESO`): date, transaction type, check/reference `NUM`, `CONCEPTO` (payee/description), amount, `OPERADO` (cleared flag), notes, and a spelled-out amount-in-words field (`CANTIDAD EN LETRA`, standard Mexican check-writing convention).
|
||||
- `DATOS I` (6,948 rows) — income/incoming (`INGRESO`): same shape, plus a `TRANSFERIDO` (transferred) flag instead of the amount-in-words field.
|
||||
- `TABLA RAMODOS` (66 rows) — a category/business-line lookup ("ramo" = line of business in Mexican insurance terminology) — this is almost certainly what `CONCEPTO` entries get classified against, i.e. the link between a bank transaction and which part of the business (insurance line, utility service, trust, etc.) it belongs to.
|
||||
- `ban` (1 row) — just holds the bank's name; a config singleton, not data.
|
||||
- `INFORME` / `INFORME BA` (0 rows each) and `FECHAIF` (1 row) — Access report/query scratch tables (date-range parameters and a report shell), same pattern as the scratch tables in the other two files — **exclude** from migration.
|
||||
|
||||
## Target architecture
|
||||
|
||||
- **Frontend:** Next.js (App Router) + TypeScript + React. Server components for data-heavy list/detail views (customers, policies, statements); client components for interactive forms.
|
||||
- **Backend:** NestJS (TypeScript) REST API — modular by domain (customers, insurance, utilities, billing, auth, admin), matching the module boundaries below. Gives you DI, guards for auth/authorization, and a validation pipeline (`class-validator`) for free, which directly replaces the old app's biggest weakness (no input validation, no parameterization).
|
||||
- **Database:** MySQL, accessed via Prisma (schema-as-code, migrations, generates a typed client — eliminates the raw-SQL-injection class of bug entirely since Prisma parameterizes everything). See Infrastructure & Sync above for why MySQL rather than Postgres.
|
||||
- **Auth:** NestJS + Passport, sessions or JWT (pick one during build), `bcrypt`/`argon2` password hashing, role-based guards replacing the old `$_SESSION['level']/['role']` checks.
|
||||
- **File/document storage:** object storage (S3-compatible) for the scanned documents currently trapped as Access `LONGBINARY` blobs — extract once during migration, store as files, keep only the pointer + metadata in MySQL. The old repo already anticipated this (`src/objects/s3.php` exists but appears unused) — same idea, implemented for real this time.
|
||||
- **Deployment:** keep Docker as the packaging mechanism (already proven for this project) with a fresh `Dockerfile`/`docker-compose.yml` for the Node services + MySQL, running on the internal server described above; CI can stay on Jenkins if that's still the team's pipeline, or move to GitHub Actions if the new repo lives somewhere other than `git.freakma.com` — flagged as an open decision below since it wasn't part of the stack questions asked.
|
||||
|
||||
## Target data model (by domain)
|
||||
|
||||
All tables get a surrogate `id` (uuid or serial) plus, where the row came from a legacy table, provenance columns (`legacy_source_db`, `legacy_source_table`, `legacy_id`) so every migrated row can be traced back to its Access original for spot-checking and reconciliation — and so the ETL can be re-run idempotently (upsert on provenance key) as migration bugs get found and fixed.
|
||||
|
||||
**Identity (the actual point of this project):**
|
||||
- `customers` — one row per real person/entity, merged from utilities `DATGRAL`/`COBRO3` and insurance `DATGRAL`, matched via the `NUM UTIL` cross-reference plus name/address fuzzy-matching for anyone missing that link. Holds name, addresses (MX + US), phones, email, ID document info, status, currency preference.
|
||||
- `customer_legacy_refs` — generalizes the old `customer_mapping` table: `(customer_id, source_system, source_table, legacy_numeric_id)`, one row per legacy record folded into this customer. This is what makes "unified customer base" actually queryable and keeps the merge auditable.
|
||||
|
||||
**Insurance domain:**
|
||||
- `insurance_providers`, `policy_types` (carry over from old schema, already reasonable)
|
||||
- `policies` — generic header (policy #, type, provider, customer, dates, premiums, agent, liquidation status), consolidating `INCENDIO`/`MULT`/`M EMPR`/all six auto-table variants/`LICENCIAS` into one table with a `policy_type` discriminator, instead of one Access table per line of business.
|
||||
- `policy_payment_installments` — unpivots the 4 hardcoded payment-installment columns (`1ER PAGO`/`FECHA PAGO`/`NO CHEQUE`, `... 2`, `... 3`, `... 4`) into rows: due sequence, amount, currency, paid date, check/reference number.
|
||||
- `vehicles` — unpivots `MCA2`'s 3 hardcoded vehicle slots (and the single-vehicle auto tables) into one row per vehicle, FK'd to policy + customer.
|
||||
- `insured_drivers` — same unpivot for the repeated named-insured/license columns in `MCA2`/`LICENCIAS`.
|
||||
- `properties` (shared with utilities domain — see below), `policy_beneficiaries` (from `BENEF`, already clean), `claims` (from `DATOS`), `adjusters` (from `AJUSTADORES*`), `policy_documents` (extracted blobs, typed: ID, prior policy copy, damage photo, etc.).
|
||||
|
||||
**Utilities domain:**
|
||||
- `properties` — one row per property (from `DATMEX`), FK'd to `customers`, shared with insurance so a property can carry both a home-insurance policy and utility service enrollments — this is the second half of "unified."
|
||||
- `property_services` — one row per enrolled service per property (water/electric/gas/cable/trust/property-tax/federal-zone), unpivoting `DATMEX`'s inline service columns and `PROFILE`'s enrollment flags into real rows with account #, route, meter #, due day.
|
||||
- `service_documents` (extracted blobs), `trust_accounts` (from `TRUSTVENCE`).
|
||||
|
||||
**Shared financial ledger** (one office, one set of books — no reason to keep insurance and utility transactions in separate schemas):
|
||||
- `transactions` — unifies utilities' `EFECTIVO`/`EFECTIVO FM3`/`EFECTIVO_BACKUP`/`FEE ANUAL`/`datos2`/`fee15`/`billing`/`CHEQUE FM3`/`IVA 2015` and insurance's `EFECTIVO`, tagged by `domain` (utility/insurance/trust) and carrying the provenance columns so the de-duplication across those overlapping snapshot tables is traceable, not destructive.
|
||||
- `exchange_rates` (from `TIPO HIST`), `type_transactions` (carry over ES/EN lookup as-is).
|
||||
- `bank_transactions` — the company's own operating bank register, from SCOTHIA's `DATOS E`/`DATOS I` unified into one signed-amount table (income positive, expense negative) with a `category` FK to `business_line_categories` (from `TABLA RAMODOS`) and a `cleared`/`operado` flag. This is deliberately **separate** from customer-facing `transactions` — it's the office's own bank reconciliation book, not money owed by/to a customer — but sharing the `business_line_categories` lookup lets you eventually answer "how much of our actual bank activity ties back to insurance vs. utilities vs. trust," which is a natural reporting win from unifying these three sources.
|
||||
- `business_line_categories` (from `TABLA RAMODOS`).
|
||||
|
||||
**Admin/shared:** `users` (hashed passwords, roles), `activity_logs`, `email_templates`/`email_campaigns`/`email_log` (carry over the old schema's intent, rebuilt on the new stack).
|
||||
|
||||
## Migration strategy
|
||||
|
||||
Given the amount of near-duplicate/overlapping data across snapshot tables (multiple `EFECTIVO*` variants, multiple year-stamped billing tables, `COBRO3` vs `DATGRAL`), doing a direct Access → normalized-MySQL transform in one pass is risky — a bug loses the ability to check itself against the source.
|
||||
|
||||
1. **Raw staging load**: dump every non-scratch Access table 1:1 into a MySQL `staging` (per-source schema/database, e.g. `stg_utilities`/`stg_seguros`/`stg_scothia`) — same columns, minimal type coercion — via a Python script (`pyodbc` → `pandas`/SQLAlchemy, same connection approach already validated in this session), across all four source files. Already built and run against real data as `migration/load_staging.py` in the new repo — see Status below. This is the audit trail — nothing is transformed yet.
|
||||
2. **Reconciliation pass**: for each set of overlapping tables (the `EFECTIVO` variants, the billing-period tables, `DATGRAL` vs `COBRO3`), write SQL that diffs them and produces a report of exact duplicates vs. genuinely distinct records, before deciding the union/de-dupe rule. Don't guess the rule up front — the data decides it.
|
||||
3. **Transform + load**: SQL/TypeScript scripts (versioned in the new repo under `migration/`) that read `staging`, apply the customer-matching and unpivot logic described above, and upsert into the real Prisma-managed tables, writing `legacy_*` provenance on every row.
|
||||
4. **Document extraction**: separate one-off script pulls every `LONGBINARY` column out to files (named by provenance key), uploads to object storage, and inserts the corresponding `*_documents` metadata row.
|
||||
5. **Validation**: row-count and spot-check reconciliation between `staging` and final tables (e.g., every legacy customer has exactly one `customers` row via `customer_legacy_refs`; sum of migrated transaction amounts per customer matches sum in `staging`).
|
||||
|
||||
## Build sequencing
|
||||
|
||||
1. Repo scaffold (Next.js + NestJS + Prisma + MySQL, Docker Compose for local dev), CI pipeline, auth module with hashed passwords and role guards.
|
||||
2. Prisma schema for the full data model above; run migration steps 1–2 (staging load + reconciliation reports) against real data early, since that's where the biggest unknowns are (do `NUM UTIL` and name-matching actually cover everyone? how bad is the snapshot-table duplication?).
|
||||
3. Customer module (list/search/detail — the unified view is the core deliverable) backed by finished migration steps 3–5 for customers only.
|
||||
4. Insurance module (policies, vehicles, beneficiaries, claims) on top of the same customer records.
|
||||
5. Utilities module (properties, services, trust accounts) on top of the same customer records.
|
||||
6. Shared billing/statements module (the payoff: one statement per customer spanning both utility and insurance transactions).
|
||||
7. Bank register module (`bank_transactions`/`business_line_categories` from SCOTHIA) — small, self-contained, and has no customer FK, so it can slot in independently once the core migration pipeline exists; low risk, low priority relative to the customer-facing modules.
|
||||
8. VPS provisioning + Tailscale + MySQL replication setup, once `utility_dbo`'s schema is available to finalize exactly which tables/columns get replicated and what the inbox tables need to look like.
|
||||
9. Sync worker (push replicated tables' relevant subset, poll inbox tables for payment/propane submissions) — depends on step 8.
|
||||
10. Reports/email campaigns/admin — parity with old app's `reports.php`/`emailCampaigns.php` intent, rebuilt properly.
|
||||
|
||||
## Status (as of this session)
|
||||
|
||||
Repo scaffolded at `jorgecuadros-platform/` (sibling to the Access files): npm workspaces, NestJS API with a real hashed-password (Argon2) session-auth module replacing the old plaintext SQL comparison, Next.js web shell, Prisma schema covering the full data model above — both apps build clean under strict TypeScript. `migration/load_staging.py` (step 1 of Migration strategy) has been run end-to-end against all four real Access files, staging 82 tables to Parquet; it surfaced and fixed two real data issues: a `cursor.columns()` UTF-16 decode bug on several tables (worked around by reading metadata from `cursor.description` instead) and one Jet/ACE-level corrupted record in `MULT` (now skipped and logged rather than aborting the whole table). Schema/infra were built against Postgres first, then switched to MySQL after the shared-hosting/portal-sync constraint came up — the provider swap (Prisma schema, Docker Compose, `.env.example`, migration script's MySQL sink) has since been applied and re-verified (schema validates, client regenerates, API rebuilds clean against MySQL). A companion resume doc lives at `jorgecuadros-platform/RESUME.md` with exact file paths, environment notes, and a session-state summary — read both together.
|
||||
|
||||
## Open decisions (not yet locked down)
|
||||
|
||||
- **`utility_dbo` schema**: the customer-facing portal's live database — referenced from the old app (`getExternalDBConnection()`) but no dump/access provided yet. Blocks finalizing exactly which tables the sync replicates and what the inbox tables (`payment_submissions`, `propane_order_requests`) need to match.
|
||||
- **VPS provisioning**: which provider (Hetzner vs DigitalOcean), size, and who sets up Tailscale + MySQL on it — an ops task outside this session's ability to do directly.
|
||||
- **CI/hosting**: keep Jenkins + `git.freakma.com`, or move CI to GitHub Actions if the new repo goes elsewhere?
|
||||
- **i18n**: nearly all source data and, presumably, staff usage is in Spanish, while the old app's code/UI was English-labeled internally. Confirm whether the new UI should be Spanish-first, bilingual, or English (matching old app) before frontend work starts.
|
||||
|
||||
## Verification
|
||||
|
||||
- Migration: automated row-count/sum reconciliation between `staging` and final schema per table group (see step 5 above), run as part of the migration script, not a manual spot-check.
|
||||
- App: standard NestJS unit/integration tests per module (auth guards, Prisma queries), Playwright/Cypress e2e for the core "look up a customer, see their unified policies + services + statement" flow — the thing the whole project exists to deliver.
|
||||
- Sync: once the VPS replica and inbox tables exist, verify replication lag stays low (a few seconds to low minutes) and that a payment/propane submission on the portal reliably shows up in the internal app within one polling interval, before relying on it operationally.
|
||||
- Before cutover: run the new app against migrated data side-by-side with the live Access files for a period, comparing balances/statements for a sample of active customers to catch migration logic errors before the Access files are retired.
|
||||
@@ -0,0 +1,147 @@
|
||||
# Resume Notes — Jorge Cuadros & Assoc. Unified Platform
|
||||
|
||||
Comprehensive state-of-the-world doc for picking this project back up. Read this
|
||||
before doing anything else in a fresh session — it front-loads everything that
|
||||
took multiple rounds of investigation to establish.
|
||||
|
||||
**Companion doc:** the full architecture/migration plan is copied into this
|
||||
repo as [`PLAN.md`](PLAN.md) (source of truth is
|
||||
`C:\Users\ricar\.claude\plans\logical-yawning-tome.md` — copy here if that
|
||||
one gets updated further). This file is the "what happened and what's next"
|
||||
companion to that plan, not a replacement for it. Read both.
|
||||
|
||||
---
|
||||
|
||||
## 1. The goal
|
||||
|
||||
Jorge Cuadros & Assoc. runs two lines of business — property/utility
|
||||
management and insurance brokerage — out of separate, decades-old MS Access
|
||||
databases, plus a third Access file that's the office's own bank checking
|
||||
register. The same people are customers of both lines but there's no shared
|
||||
customer record between systems. Goal: **one platform with a single unified
|
||||
customer record**, from which staff see and manage that customer's utility
|
||||
services *and* insurance policies *and* shared billing/transaction history —
|
||||
replacing the Access files and the old, insecure PHP internal app.
|
||||
|
||||
There is also a **separate, pre-existing customer-facing portal** (PHP +
|
||||
MySQL, with a companion mobile app) that customers use to view statements,
|
||||
make payments, and order propane. That portal is **out of scope to rebuild**
|
||||
— it stays exactly as-is — but the new platform has to keep it supplied with
|
||||
live data. That constraint is what drove the database-engine and
|
||||
infrastructure decisions below.
|
||||
|
||||
## 2. Where everything lives (file paths)
|
||||
|
||||
**Source data (do not modify — read-only references):**
|
||||
- `C:\Users\ricar\Downloads\Jorge\UTILITIES.accdb` — utilities business, 52 tables, ~538MB
|
||||
- `C:\Users\ricar\Downloads\Jorge\SEGUROS 16.mdb` — insurance frontend shell, **empty**, all data is in `_be`
|
||||
- `C:\Users\ricar\Downloads\Jorge\SEGUROS 16_be.mdb` — insurance backend, 64 tables, ~882MB
|
||||
- `C:\Users\ricar\Downloads\Jorge\SCOTHIA.mdb` — office's own Scotiabank checking register ("chequera"), 7 tables, ~3MB
|
||||
- `C:\Users\ricar\Downloads\jorgecuadros_app.sql` and `jorgecuadros_app (1).sql` — MySQL dumps of the customer-portal's **tracking/analytics** sidecar DB (`browse_tracking`, `devices` push-tokens, `task_tracking`) from `mysql.freakma.com`. **Not** the portal's real data DB — see open item #1 below.
|
||||
|
||||
**Old internal app (reference-only, not being built on):**
|
||||
- `C:\Users\ricar\Downloads\Jorge\jorgecuadros-intra-webapp` — PHP, MySQL (`webapp_jorgecuadros`). Schema at `db\webapp_jorgecuadros.sql` is a useful reference for field mappings/business logic. Code itself is not being reused — see §4.
|
||||
|
||||
**New platform (the actual deliverable, in progress):**
|
||||
- `C:\Users\ricar\Downloads\Jorge\jorgecuadros-platform` — the new repo. Not yet a git repository (no commits made this session — user hasn't asked for any yet).
|
||||
|
||||
**The plan document:**
|
||||
- `C:\Users\ricar\.claude\plans\logical-yawning-tome.md` — full architecture, source-data inventory per table, target data model, migration strategy, infrastructure/sync design, open decisions, build sequencing. **This is the source of truth for the design** — this RESUME.md summarizes it plus session/environment state that isn't in the plan itself.
|
||||
|
||||
**Ephemeral / will NOT persist across sessions** (session-scoped temp directory):
|
||||
- `C:\Users\ricar\AppData\Local\Temp\claude\...\scratchpad\` — contained exploratory helper scripts (`dump_schema.py`, `summarize_schema.py`, `test_decode_fix.py`) and the schema JSON/txt dumps used during initial analysis, plus a Parquet staging output from one run of the migration script. **None of this needs to be recovered** — the real, permanent versions of the useful scripts are in `jorgecuadros-platform/migration/`, and the Parquet output can be regenerated in under 2 minutes by rerunning `load_staging.py` (see §6).
|
||||
|
||||
## 3. Key decisions made this session
|
||||
|
||||
| Decision | Answer | Why |
|
||||
|---|---|---|
|
||||
| Stack | Next.js (React/TS) + NestJS (TS) + Prisma | Type safety, parameterized queries by default (kills the SQL-injection class of bug the old app had everywhere) |
|
||||
| Database engine | **MySQL** (not Postgres — reversed mid-session) | The customer-facing portal's PHP code (`mysqli`) isn't being rewritten, and shared hosting can't run Postgres. Using MySQL everywhere avoids a cross-engine sync layer. |
|
||||
| Repo | New repo, not built on `jorgecuadros-intra-webapp` | That repo has SQL injection in every query (`src/core/db.php` string-concatenates `$_POST`) and plaintext password comparison (`src/core/auth.php`) — not worth patching |
|
||||
| Historical data | Migrate everything, no cutoff | Source tables are small (largest ~16k rows); completeness is cheap |
|
||||
| Customer portal | Stays as-is, not rebuilt | Explicit user decision |
|
||||
| Infrastructure | Internal server (private) + new VPS (Tailscale-linked) running a MySQL replica | Internal server has no inbound internet exposure; shared hosting can't be a replication target; a VPS you control can be both a real replication node and internet-reachable for the portal |
|
||||
| Auth mechanism | Session-based (Passport + `express-session`), Argon2 password hashing | Implemented already — see §5 |
|
||||
|
||||
## 4. What was actually built and verified this session
|
||||
|
||||
Everything below was **run and confirmed working**, not just written:
|
||||
|
||||
### 4.1 Repo scaffold
|
||||
- `jorgecuadros-platform/` — npm workspaces (`apps/*`, `packages/*`)
|
||||
- `apps/api` — NestJS. **Builds clean** under strict TypeScript (`npx nest build` in `apps/api`, zero errors).
|
||||
- `src/main.ts` — global `ValidationPipe` (whitelist + forbid unknown fields — structural replacement for the old app's total lack of input validation), session middleware, CORS.
|
||||
- `src/auth/` — `AuthService.validateUser()` verifies passwords with `argon2.verify()` (replaces `passwd = '$password'` plaintext SQL comparison in the old app), `LocalStrategy`, `SessionSerializer`, `AuthenticatedGuard` (replaces manually-called `validate_session()`), `AuthController` (`/auth/login`, `/auth/me`, `/auth/logout`).
|
||||
- `src/users/`, `src/prisma/` (global `PrismaModule`/`PrismaService`).
|
||||
- `apps/web` — Next.js App Router shell. **Builds clean** (`npx next build`).
|
||||
- `packages/database` — Prisma schema + generated client.
|
||||
|
||||
### 4.2 Prisma schema (`packages/database/prisma/schema.prisma`)
|
||||
Full target data model implementing the plan's design, **validated and generating a working client against MySQL**:
|
||||
- **Identity:** `Customer`, `CustomerLegacyRef` (generalizes the old `customer_mapping` bridge table — one row per legacy record folded into a unified customer, with provenance)
|
||||
- **Insurance:** `InsuranceProvider`, `PolicyType`, `Policy` (consolidates `INCENDIO`/`MULT`/`M EMPR`/6 auto-table variants/`LICENCIAS` into one table with a type discriminator), `PolicyPaymentInstallment` (unpivots the 4 hardcoded payment-installment columns found on every legacy policy table), `Vehicle` (unpivots `MCA2`'s 3 hardcoded vehicle slots), `InsuredDriver`, `PolicyBeneficiary`, `Claim`, `Adjuster`, `PolicyDocument`
|
||||
- **Utilities:** `Property` (shared with insurance — the actual unification point), `PropertyService`, `ServiceDocument`, `TrustAccount`
|
||||
- **Shared ledger:** `Transaction` (unifies all the `EFECTIVO*`/billing-period snapshot tables), `ExchangeRate`, `TypeTransaction`
|
||||
- **Bank register (SCOTHIA):** `BankTransaction`, `BusinessLineCategory`
|
||||
- **Admin:** `User` (hashed passwords, roles), `ActivityLog`, `EmailTemplate`, `EmailCampaign`, `EmailLog`
|
||||
|
||||
Every model sourced from a legacy Access table carries `legacySourceDb`/`legacySourceTable`/`legacyId` provenance columns for traceability and idempotent re-runs. Long-text fields (`notes`, `observations`, `description`, etc.) are explicitly `@db.Text` — MySQL's default `String` is `VARCHAR(191)` and would silently truncate them otherwise (this was caught and fixed during the Postgres→MySQL swap).
|
||||
|
||||
Regenerate the client any time with:
|
||||
```bash
|
||||
cd jorgecuadros-platform
|
||||
DATABASE_URL="mysql://user:pass@localhost:3306/placeholder" npx prisma generate --schema=packages/database/prisma/schema.prisma
|
||||
```
|
||||
(A real `DATABASE_URL` isn't needed for `generate`/`validate`, just a syntactically valid one — no live DB was available in this session, see §7.)
|
||||
|
||||
### 4.3 Docker Compose / Dockerfiles
|
||||
- `docker-compose.yml` — `mysql:8.4` + `api` + `web` services, healthchecked.
|
||||
- `docker/api.Dockerfile`, `docker/web.Dockerfile` — multi-stage builds.
|
||||
- `.env.example` — `DATABASE_URL`, `SESSION_SECRET`, `WEB_ORIGIN`, `NEXT_PUBLIC_API_ORIGIN`.
|
||||
- **Not run** — this environment has no Docker installed (`docker --version` fails). Untested beyond visual review; verify on a machine with Docker before relying on it.
|
||||
|
||||
### 4.4 Migration pipeline (`migration/`) — run end-to-end against real data
|
||||
- `config.py` — manifest of the 3 Access source files (paths + per-source exclude lists for confirmed-scratch tables, with reasoning in comments)
|
||||
- `extract.py` — connects via `pyodbc` + the Windows Access ODBC driver (`Microsoft Access Driver (*.mdb, *.accdb)`, 64-bit — confirmed installed on this machine). Two real bugs found and fixed here:
|
||||
1. `cursor.columns()` hits a UTF-16 decode bug on some tables (confirmed: `PROPANO`, `FALTANTES AGUA`, `TIT`) — fixed by reading column names from `cursor.description` after a `SELECT *` instead.
|
||||
2. `cursor.fetchall()` aborts an entire table on the first corrupted row — confirmed on `MULT` (Jet/ACE-level "Record is deleted" error, HY109). Fixed by fetching row-by-row in a try/except, skipping and logging just the bad row. Recovered 763 of 764 rows in `MULT`. **Verified the cursor advances correctly and doesn't infinite-loop on the bad row** before trusting this for a full run.
|
||||
- `load_staging.py` — dumps every non-excluded table into either Parquet (`--output-dir`, no DB needed) or MySQL (`--database-url`, one database per source: `stg_utilities`/`stg_seguros`/`stg_scothia`). **Actually run** in Parquet mode against all three Access files: **82 tables staged successfully, zero unhandled errors**, `MULT`'s corrupted row correctly skipped and logged.
|
||||
- `requirements.txt` — `pyodbc`, `pandas`, `pyarrow`, `sqlalchemy`, `pymysql`.
|
||||
|
||||
To rerun (from `jorgecuadros-platform/migration`, after `pip install -r requirements.txt`):
|
||||
```bash
|
||||
python load_staging.py --output-dir ./output # Parquet, no DB needed — always works
|
||||
python load_staging.py --database-url mysql+pymysql://user:pass@host:3306/ # loads into real MySQL once available
|
||||
```
|
||||
|
||||
## 5. Infrastructure & sync architecture (designed, not yet built)
|
||||
|
||||
- **Internal server** — on-prem, private IP `192.168.1.xx`, no inbound internet exposure. Runs the platform + canonical MySQL (source of truth).
|
||||
- **New VPS** (Hetzner or DigitalOcean — not yet provisioned) — becomes what `mysql.freakma.com` resolves to. Runs a MySQL replica. Because it's infrastructure the user controls (unlike shared hosting), it can be a real MySQL replication target.
|
||||
- **Tailscale** — mesh VPN joining the internal server and the VPS, so they can reach each other without opening inbound ports anywhere.
|
||||
- **Internal → VPS:** one-way native MySQL replication (binlog/GTID) for the subset of data the portal needs to read (statements, balances, customer profile). Internal-only tables (staff notes, adjuster info, activity logs) are deliberately excluded from what replicates.
|
||||
- **VPS → Internal:** the portal also *writes* (payment submissions, propane orders) — one-way replication can't carry that back, and multi-master MySQL replication was deliberately ruled out as too fragile for this system's size. Instead: unreplicated "inbox" tables on the VPS (`payment_submissions`, `propane_order_requests`) that the portal writes to directly, polled every 1–5 min by a worker on the internal server (over Tailscale) that turns new rows into real records.
|
||||
|
||||
## 6. Open items — need input/access before certain next steps can proceed
|
||||
|
||||
1. **`utility_dbo` schema is still unknown.** This is the customer portal's actual live data database (referenced in the old app via `getExternalDBConnection()` at `mysql.freakma.com`, used there only for `email_alert_log`, but the mobile app almost certainly reads/writes statements, payments, and propane orders directly against it). The two SQL dumps provided (`jorgecuadros_app.sql`, `jorgecuadros_app (1).sql`) turned out to be a **separate** analytics/tracking database, not this one. When asked, the user pointed back to `SEGUROS 16_be.mdb` — worth revisiting; it's possible the intent was "the source data ultimately comes from the Access files" rather than "here is utility_dbo's schema." **Without the real `utility_dbo` schema, the sync worker's exact target tables/columns for the inbox pattern can't be finalized.** Ask for an export or read-only credentials, same as how the Access files and tracking-DB dumps were provided.
|
||||
2. **VPS not yet provisioned** — provider (Hetzner vs DigitalOcean), size, and Tailscale/MySQL setup on it are pending. Ops task, not something done in this session.
|
||||
3. **CI/hosting** — keep Jenkins + `git.freakma.com`, or move to GitHub Actions if the new repo lives elsewhere?
|
||||
4. **i18n** — nearly all source data and, presumably, staff usage is in Spanish; old app's code/UI was English-labeled. Confirm Spanish-first / bilingual / English before frontend work goes deep.
|
||||
5. **Reconciliation pass not started** (migration plan step 2) — the near-duplicate snapshot tables (`EFECTIVO`/`EFECTIVO FM3`/`EFECTIVO_BACKUP`, `FEE ANUAL`/`datos2`/`fee15`/`billing`, `DATGRAL` vs `COBRO3`) need a diff/dedupe pass against the staged data before any transform-and-load into the real schema. This is explicitly *not* a "guess the rule up front" thing — the plan calls for writing SQL against the staged Parquet/MySQL data to see what's actually duplicate vs. distinct.
|
||||
|
||||
## 7. Environment notes (this machine, in case it matters for reproducing)
|
||||
|
||||
- Windows, PowerShell primary, Git Bash also available.
|
||||
- Node v25.2.0, npm 11.6.2 — no pnpm, yarn is present but npm workspaces were used throughout.
|
||||
- Python 3.9 (`C:\Python39`), `pip install`'d this session: `pyodbc`, `pandas`, `pyarrow`, `psycopg2` (installed but no longer used after the MySQL switch — harmless to leave or remove).
|
||||
- MS Access ODBC driver confirmed installed: **64-bit** `Microsoft Access Driver (*.mdb, *.accdb)` (matches 64-bit Python — this pairing matters, a 32/64-bit mismatch would break `pyodbc.connect`).
|
||||
- **No Docker, no local MySQL, no local Postgres** on this machine — `docker-compose.yml` and the MySQL-target mode of `load_staging.py` are written but unexecuted here. Test both on whatever machine ends up running this for real.
|
||||
- Old repo's `dbConnection.php` has a **hardcoded plaintext MySQL password** for the external DB connection, committed to git history. Not carried forward into the new platform, but worth rotating that credential regardless since it's already exposed in the old repo's history.
|
||||
|
||||
## 8. Suggested next session starting point
|
||||
|
||||
1. Chase down open item #1 (`utility_dbo` schema) — it blocks finalizing the sync design concretely.
|
||||
2. Start the reconciliation pass (plan step 2, open item #5) against the staged Parquet data — rerun `load_staging.py --output-dir` first since the original output didn't persist.
|
||||
3. Once reconciliation rules are known, write the transform-and-load scripts (plan step 3) that populate the real Prisma-managed MySQL tables from staging, starting with the `Customer`/`CustomerLegacyRef` module since every other module depends on it.
|
||||
4. In parallel or after: build out the Customer module in `apps/api`/`apps/web` (list/search/detail) — the first real feature, per Build Sequencing step 3 in the plan.
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/nest-cli",
|
||||
"collection": "@nestjs/schematics",
|
||||
"sourceRoot": "src",
|
||||
"compilerOptions": {
|
||||
"deleteOutDir": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"name": "@jorgecuadros/api",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"build": "nest build",
|
||||
"start": "nest start",
|
||||
"start:dev": "nest start --watch",
|
||||
"start:prod": "node dist/main",
|
||||
"lint": "eslint \"src/**/*.ts\"",
|
||||
"test": "jest"
|
||||
},
|
||||
"dependencies": {
|
||||
"@jorgecuadros/database": "0.1.0",
|
||||
"@nestjs/common": "^10.4.4",
|
||||
"@nestjs/config": "^3.3.0",
|
||||
"@nestjs/core": "^10.4.4",
|
||||
"@nestjs/passport": "^10.0.3",
|
||||
"@nestjs/platform-express": "^10.4.4",
|
||||
"argon2": "^0.41.1",
|
||||
"class-transformer": "^0.5.1",
|
||||
"class-validator": "^0.14.1",
|
||||
"express-session": "^1.18.0",
|
||||
"passport": "^0.7.0",
|
||||
"passport-local": "^1.0.0",
|
||||
"reflect-metadata": "^0.2.2",
|
||||
"rxjs": "^7.8.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@nestjs/cli": "^10.4.5",
|
||||
"@nestjs/testing": "^10.4.4",
|
||||
"@types/express": "^4.17.21",
|
||||
"@types/express-session": "^1.18.0",
|
||||
"@types/jest": "^29.5.13",
|
||||
"@types/node": "^20.16.11",
|
||||
"@types/passport-local": "^1.0.38",
|
||||
"jest": "^29.7.0",
|
||||
"ts-jest": "^29.2.5",
|
||||
"ts-node": "^10.9.2",
|
||||
"typescript": "^5.6.3"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
import { Controller, Get } from "@nestjs/common";
|
||||
|
||||
@Controller()
|
||||
export class AppController {
|
||||
@Get("health")
|
||||
health() {
|
||||
return { status: "ok" };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
import { Module } from "@nestjs/common";
|
||||
import { ConfigModule } from "@nestjs/config";
|
||||
import { PrismaModule } from "./prisma/prisma.module";
|
||||
import { UsersModule } from "./users/users.module";
|
||||
import { AuthModule } from "./auth/auth.module";
|
||||
import { AppController } from "./app.controller";
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
ConfigModule.forRoot({ isGlobal: true }),
|
||||
PrismaModule,
|
||||
UsersModule,
|
||||
AuthModule,
|
||||
],
|
||||
controllers: [AppController],
|
||||
})
|
||||
export class AppModule {}
|
||||
@@ -0,0 +1,38 @@
|
||||
import { Controller, Get, HttpCode, Post, Req, Res, UseGuards } from "@nestjs/common";
|
||||
import { Request, Response } from "express";
|
||||
import { LocalAuthGuard } from "./local-auth.guard";
|
||||
import { AuthenticatedGuard } from "./authenticated.guard";
|
||||
import { LoginDto } from "./login.dto";
|
||||
|
||||
@Controller("auth")
|
||||
export class AuthController {
|
||||
// LoginDto is only used for request-shape documentation/validation here —
|
||||
// the actual credential check happens inside LocalStrategy via Passport,
|
||||
// which populates req.user before this handler runs.
|
||||
@UseGuards(LocalAuthGuard)
|
||||
@Post("login")
|
||||
@HttpCode(200)
|
||||
login(@Req() req: Request, @Res({ passthrough: true }) _res: Response, _body?: LoginDto) {
|
||||
return req.user;
|
||||
}
|
||||
|
||||
@UseGuards(AuthenticatedGuard)
|
||||
@Get("me")
|
||||
me(@Req() req: Request) {
|
||||
return req.user;
|
||||
}
|
||||
|
||||
@Post("logout")
|
||||
@HttpCode(200)
|
||||
logout(@Req() req: Request) {
|
||||
return new Promise((resolve, reject) => {
|
||||
req.logout((err) => {
|
||||
if (err) {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
resolve({ success: true });
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
import { Module } from "@nestjs/common";
|
||||
import { PassportModule } from "@nestjs/passport";
|
||||
import { UsersModule } from "../users/users.module";
|
||||
import { AuthService } from "./auth.service";
|
||||
import { AuthController } from "./auth.controller";
|
||||
import { LocalStrategy } from "./local.strategy";
|
||||
import { SessionSerializer } from "./session.serializer";
|
||||
|
||||
@Module({
|
||||
imports: [UsersModule, PassportModule.register({ session: true })],
|
||||
providers: [AuthService, LocalStrategy, SessionSerializer],
|
||||
controllers: [AuthController],
|
||||
exports: [AuthService],
|
||||
})
|
||||
export class AuthModule {}
|
||||
@@ -0,0 +1,36 @@
|
||||
import { Injectable } from "@nestjs/common";
|
||||
import * as argon2 from "argon2";
|
||||
import { UsersService } from "../users/users.service";
|
||||
import type { User } from "@jorgecuadros/database";
|
||||
|
||||
export type SafeUser = Omit<User, "passwordHash">;
|
||||
|
||||
@Injectable()
|
||||
export class AuthService {
|
||||
constructor(private readonly usersService: UsersService) {}
|
||||
|
||||
/**
|
||||
* Replaces the old app's `passwd = '$password'` plaintext SQL comparison
|
||||
* (src/core/auth.php) with a constant-time hash verification. Returns
|
||||
* null on any failure — callers should not distinguish "no such user"
|
||||
* from "wrong password" in their response.
|
||||
*/
|
||||
async validateUser(email: string, password: string): Promise<SafeUser | null> {
|
||||
const user = await this.usersService.findByEmail(email);
|
||||
if (!user || !user.active) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const passwordMatches = await argon2.verify(user.passwordHash, password);
|
||||
if (!passwordMatches) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const { passwordHash: _passwordHash, ...safeUser } = user;
|
||||
return safeUser;
|
||||
}
|
||||
|
||||
static async hashPassword(plainTextPassword: string): Promise<string> {
|
||||
return argon2.hash(plainTextPassword);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import { CanActivate, ExecutionContext, Injectable } from "@nestjs/common";
|
||||
import { Request } from "express";
|
||||
|
||||
/**
|
||||
* Replaces the old app's validate_session() (src/core/auth.php), which every
|
||||
* page had to remember to call manually. Here it's a guard attached via
|
||||
* @UseGuards(AuthenticatedGuard) — a controller that forgets it simply has
|
||||
* no route, instead of silently serving unauthenticated data.
|
||||
*/
|
||||
@Injectable()
|
||||
export class AuthenticatedGuard implements CanActivate {
|
||||
canActivate(context: ExecutionContext): boolean {
|
||||
const request = context.switchToHttp().getRequest<Request>();
|
||||
return request.isAuthenticated();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
import { Injectable } from "@nestjs/common";
|
||||
import { AuthGuard } from "@nestjs/passport";
|
||||
|
||||
@Injectable()
|
||||
export class LocalAuthGuard extends AuthGuard("local") {}
|
||||
@@ -0,0 +1,19 @@
|
||||
import { Injectable, UnauthorizedException } from "@nestjs/common";
|
||||
import { PassportStrategy } from "@nestjs/passport";
|
||||
import { Strategy } from "passport-local";
|
||||
import { AuthService, SafeUser } from "./auth.service";
|
||||
|
||||
@Injectable()
|
||||
export class LocalStrategy extends PassportStrategy(Strategy) {
|
||||
constructor(private readonly authService: AuthService) {
|
||||
super({ usernameField: "email", passwordField: "password" });
|
||||
}
|
||||
|
||||
async validate(email: string, password: string): Promise<SafeUser> {
|
||||
const user = await this.authService.validateUser(email, password);
|
||||
if (!user) {
|
||||
throw new UnauthorizedException("Invalid email or password");
|
||||
}
|
||||
return user;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
import { IsEmail, IsString, MinLength } from "class-validator";
|
||||
|
||||
export class LoginDto {
|
||||
@IsEmail()
|
||||
email!: string;
|
||||
|
||||
@IsString()
|
||||
@MinLength(1)
|
||||
password!: string;
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import { Injectable } from "@nestjs/common";
|
||||
import { PassportSerializer } from "@nestjs/passport";
|
||||
import { UsersService } from "../users/users.service";
|
||||
import { SafeUser } from "./auth.service";
|
||||
|
||||
@Injectable()
|
||||
export class SessionSerializer extends PassportSerializer {
|
||||
constructor(private readonly usersService: UsersService) {
|
||||
super();
|
||||
}
|
||||
|
||||
serializeUser(user: SafeUser, done: (err: Error | null, id: string) => void) {
|
||||
done(null, user.id);
|
||||
}
|
||||
|
||||
async deserializeUser(id: string, done: (err: Error | null, user: SafeUser | null) => void) {
|
||||
const user = await this.usersService.findById(id);
|
||||
if (!user) {
|
||||
done(null, null);
|
||||
return;
|
||||
}
|
||||
const { passwordHash: _passwordHash, ...safeUser } = user;
|
||||
done(null, safeUser);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
import "reflect-metadata";
|
||||
import { NestFactory } from "@nestjs/core";
|
||||
import { ValidationPipe } from "@nestjs/common";
|
||||
import * as session from "express-session";
|
||||
import * as passport from "passport";
|
||||
import { AppModule } from "./app.module";
|
||||
|
||||
async function bootstrap() {
|
||||
const app = await NestFactory.create(AppModule);
|
||||
|
||||
// Every request body is validated and stripped of unknown fields before it
|
||||
// reaches a controller — this is the structural replacement for the old
|
||||
// app's complete lack of input validation (src/core/db.php took every
|
||||
// $_POST field straight into a SQL string).
|
||||
app.useGlobalPipes(
|
||||
new ValidationPipe({
|
||||
whitelist: true,
|
||||
forbidNonWhitelisted: true,
|
||||
transform: true,
|
||||
})
|
||||
);
|
||||
|
||||
const sessionSecret = process.env.SESSION_SECRET;
|
||||
if (!sessionSecret) {
|
||||
throw new Error("SESSION_SECRET must be set (see .env.example)");
|
||||
}
|
||||
|
||||
app.use(
|
||||
session({
|
||||
secret: sessionSecret,
|
||||
resave: false,
|
||||
saveUninitialized: false,
|
||||
cookie: {
|
||||
httpOnly: true,
|
||||
secure: process.env.NODE_ENV === "production",
|
||||
maxAge: 1000 * 60 * 60 * 8, // 8-hour session, matches a staff workday
|
||||
},
|
||||
})
|
||||
);
|
||||
app.use(passport.initialize());
|
||||
app.use(passport.session());
|
||||
|
||||
app.enableCors({ credentials: true, origin: process.env.WEB_ORIGIN ?? "http://localhost:3000" });
|
||||
|
||||
const port = process.env.PORT ? Number(process.env.PORT) : 3001;
|
||||
await app.listen(port);
|
||||
}
|
||||
|
||||
bootstrap();
|
||||
@@ -0,0 +1,9 @@
|
||||
import { Global, Module } from "@nestjs/common";
|
||||
import { PrismaService } from "./prisma.service";
|
||||
|
||||
@Global()
|
||||
@Module({
|
||||
providers: [PrismaService],
|
||||
exports: [PrismaService],
|
||||
})
|
||||
export class PrismaModule {}
|
||||
@@ -0,0 +1,13 @@
|
||||
import { Injectable, OnModuleDestroy, OnModuleInit } from "@nestjs/common";
|
||||
import { PrismaClient } from "@jorgecuadros/database";
|
||||
|
||||
@Injectable()
|
||||
export class PrismaService extends PrismaClient implements OnModuleInit, OnModuleDestroy {
|
||||
async onModuleInit() {
|
||||
await this.$connect();
|
||||
}
|
||||
|
||||
async onModuleDestroy() {
|
||||
await this.$disconnect();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
import { Module } from "@nestjs/common";
|
||||
import { UsersService } from "./users.service";
|
||||
|
||||
@Module({
|
||||
providers: [UsersService],
|
||||
exports: [UsersService],
|
||||
})
|
||||
export class UsersModule {}
|
||||
@@ -0,0 +1,16 @@
|
||||
import { Injectable } from "@nestjs/common";
|
||||
import { PrismaService } from "../prisma/prisma.service";
|
||||
import type { User } from "@jorgecuadros/database";
|
||||
|
||||
@Injectable()
|
||||
export class UsersService {
|
||||
constructor(private readonly prisma: PrismaService) {}
|
||||
|
||||
findByEmail(email: string): Promise<User | null> {
|
||||
return this.prisma.user.findUnique({ where: { email } });
|
||||
}
|
||||
|
||||
findById(id: string): Promise<User | null> {
|
||||
return this.prisma.user.findUnique({ where: { id } });
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "commonjs",
|
||||
"declaration": true,
|
||||
"removeComments": true,
|
||||
"emitDecoratorMetadata": true,
|
||||
"experimentalDecorators": true,
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"target": "ES2021",
|
||||
"sourceMap": true,
|
||||
"outDir": "./dist",
|
||||
"baseUrl": "./",
|
||||
"incremental": true,
|
||||
"skipLibCheck": true,
|
||||
"strict": true,
|
||||
"strictNullChecks": true,
|
||||
"noImplicitAny": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"moduleResolution": "node"
|
||||
}
|
||||
}
|
||||
Vendored
+5
@@ -0,0 +1,5 @@
|
||||
/// <reference types="next" />
|
||||
/// <reference types="next/image-types/global" />
|
||||
|
||||
// NOTE: This file should not be edited
|
||||
// see https://nextjs.org/docs/app/building-your-application/configuring/typescript for more information.
|
||||
@@ -0,0 +1,6 @@
|
||||
/** @type {import('next').NextConfig} */
|
||||
const nextConfig = {
|
||||
reactStrictMode: true,
|
||||
};
|
||||
|
||||
module.exports = nextConfig;
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"name": "@jorgecuadros/web",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"dev": "next dev",
|
||||
"build": "next build",
|
||||
"start": "next start",
|
||||
"lint": "next lint"
|
||||
},
|
||||
"dependencies": {
|
||||
"next": "^14.2.15",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.16.11",
|
||||
"@types/react": "^18.3.11",
|
||||
"@types/react-dom": "^18.3.0",
|
||||
"typescript": "^5.6.3"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
import type { ReactNode } from "react";
|
||||
|
||||
export const metadata = {
|
||||
title: "Jorge Cuadros & Assoc.",
|
||||
description: "Unified customer, insurance, and utilities platform",
|
||||
};
|
||||
|
||||
export default function RootLayout({ children }: { children: ReactNode }) {
|
||||
return (
|
||||
<html lang="en">
|
||||
<body>{children}</body>
|
||||
</html>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
export default function HomePage() {
|
||||
return (
|
||||
<main>
|
||||
<h1>Jorge Cuadros & Assoc.</h1>
|
||||
<p>Unified customer platform — scaffold in progress.</p>
|
||||
</main>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2020",
|
||||
"lib": ["dom", "dom.iterable", "esnext"],
|
||||
"allowJs": false,
|
||||
"skipLibCheck": true,
|
||||
"strict": true,
|
||||
"noEmit": true,
|
||||
"esModuleInterop": true,
|
||||
"module": "esnext",
|
||||
"moduleResolution": "bundler",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"jsx": "preserve",
|
||||
"incremental": true,
|
||||
"plugins": [{ "name": "next" }],
|
||||
"paths": {
|
||||
"@/*": ["./src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
|
||||
"exclude": ["node_modules"]
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
services:
|
||||
mysql:
|
||||
image: mysql:8.4
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MYSQL_USER: jorgecuadros
|
||||
MYSQL_PASSWORD: jorgecuadros
|
||||
MYSQL_DATABASE: jorgecuadros
|
||||
MYSQL_ALLOW_EMPTY_PASSWORD: "no"
|
||||
MYSQL_RANDOM_ROOT_PASSWORD: "yes"
|
||||
ports:
|
||||
- "3306:3306"
|
||||
volumes:
|
||||
- mysql_data:/var/lib/mysql
|
||||
healthcheck:
|
||||
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "jorgecuadros", "-pjorgecuadros"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
api:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: docker/api.Dockerfile
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
mysql:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
DATABASE_URL: mysql://jorgecuadros:jorgecuadros@mysql:3306/jorgecuadros
|
||||
SESSION_SECRET: ${SESSION_SECRET:?SESSION_SECRET must be set}
|
||||
WEB_ORIGIN: http://localhost:3000
|
||||
PORT: 3001
|
||||
ports:
|
||||
- "3001:3001"
|
||||
|
||||
web:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: docker/web.Dockerfile
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- api
|
||||
environment:
|
||||
NEXT_PUBLIC_API_ORIGIN: http://localhost:3001
|
||||
ports:
|
||||
- "3000:3000"
|
||||
|
||||
volumes:
|
||||
mysql_data:
|
||||
@@ -0,0 +1,24 @@
|
||||
FROM node:20-alpine AS base
|
||||
WORKDIR /repo
|
||||
|
||||
FROM base AS deps
|
||||
COPY package.json package-lock.json* ./
|
||||
COPY apps/api/package.json apps/api/package.json
|
||||
COPY packages/database/package.json packages/database/package.json
|
||||
RUN npm install --workspace=packages/database --workspace=apps/api --no-audit --no-fund
|
||||
|
||||
FROM deps AS build
|
||||
COPY packages/database packages/database
|
||||
COPY apps/api apps/api
|
||||
RUN npm run generate -w packages/database
|
||||
RUN npm run build -w apps/api
|
||||
|
||||
FROM node:20-alpine AS runtime
|
||||
WORKDIR /repo
|
||||
ENV NODE_ENV=production
|
||||
COPY --from=build /repo/node_modules node_modules
|
||||
COPY --from=build /repo/packages/database packages/database
|
||||
COPY --from=build /repo/apps/api/dist apps/api/dist
|
||||
COPY --from=build /repo/apps/api/package.json apps/api/package.json
|
||||
EXPOSE 3001
|
||||
CMD ["node", "apps/api/dist/main.js"]
|
||||
@@ -0,0 +1,20 @@
|
||||
FROM node:20-alpine AS base
|
||||
WORKDIR /repo
|
||||
|
||||
FROM base AS deps
|
||||
COPY package.json package-lock.json* ./
|
||||
COPY apps/web/package.json apps/web/package.json
|
||||
RUN npm install --workspace=apps/web --no-audit --no-fund
|
||||
|
||||
FROM deps AS build
|
||||
COPY apps/web apps/web
|
||||
RUN npm run build -w apps/web
|
||||
|
||||
FROM node:20-alpine AS runtime
|
||||
WORKDIR /repo
|
||||
ENV NODE_ENV=production
|
||||
COPY --from=build /repo/node_modules node_modules
|
||||
COPY --from=build /repo/apps/web apps/web
|
||||
EXPOSE 3000
|
||||
WORKDIR /repo/apps/web
|
||||
CMD ["npx", "next", "start"]
|
||||
@@ -0,0 +1,84 @@
|
||||
"""
|
||||
Source-database manifest for the staging load (migration plan step 1).
|
||||
|
||||
Exclusions here are deliberately conservative: only tables that are either
|
||||
(a) confirmed empty (0 rows — nothing is lost by skipping them) or (b) have
|
||||
rows but are structurally not customer/business data (mail-merge document
|
||||
templates, materialized Access query results) are excluded. Anything with
|
||||
real rows and an ambiguous purpose (e.g. PROPANO, datosfreak, pagos email)
|
||||
is loaded into staging anyway — the reconciliation pass decides what to do
|
||||
with it, per the migration plan's "don't guess the rule up front" principle.
|
||||
See C:\\Users\\ricar\\.claude\\plans\\logical-yawning-tome.md for the full
|
||||
rationale per table group.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
SOURCE_ROOT = Path(r"C:\Users\ricar\Downloads\Jorge")
|
||||
|
||||
SOURCES = {
|
||||
"utilities": {
|
||||
"path": SOURCE_ROOT / "UTILITIES.accdb",
|
||||
"schema": "stg_utilities",
|
||||
# Confirmed empty (0 rows) working/scratch tables.
|
||||
"exclude": {
|
||||
"BANCO EDITOR",
|
||||
"Errores de pegado",
|
||||
"TABLE1",
|
||||
"PARA BILLING SIN",
|
||||
"PARA BILLING1",
|
||||
"PARA BILLING2",
|
||||
"PARA EDO",
|
||||
"FALTANTES AGUA",
|
||||
"FALTANTES TEL",
|
||||
"LUZ TODOS",
|
||||
"TELEFONOS FECHAS",
|
||||
"TRUSTHFEE",
|
||||
"faltantes luz",
|
||||
"billing", # 0 rows; superseded by datos2/FEE ANUAL/fee15
|
||||
"TIT", # 1 row, default Access "Contacts" template shell — not real data
|
||||
},
|
||||
},
|
||||
"seguros": {
|
||||
# SEGUROS 16.mdb is an empty linked front-end; all data lives in _be.
|
||||
"path": SOURCE_ROOT / "SEGUROS 16_be.mdb",
|
||||
"schema": "stg_seguros",
|
||||
"exclude": {
|
||||
# Mail-merge document templates (letters/certificates), not data.
|
||||
"AMPL MENS",
|
||||
"AMPL R MENS",
|
||||
"IN MENS",
|
||||
"LIC MENS",
|
||||
"MCA2 MENS",
|
||||
"ME MENS",
|
||||
"MF MENS",
|
||||
"RC MENSAJE",
|
||||
"RC R MENS",
|
||||
# Materialized Access query results, not source-of-truth data.
|
||||
"TODOSJC",
|
||||
"TODOS",
|
||||
"vigenta casa y auto unicos",
|
||||
# Confirmed empty scratch tables.
|
||||
"ID TABLA",
|
||||
"ID TABLATLAS",
|
||||
"TABLA LIQUIDA MCA2",
|
||||
"TABLA LIQUIDA MF",
|
||||
"TABLA LIQUIDA RES",
|
||||
"TABLA LIQUIDA TUR",
|
||||
"TABLA LIQUIDA TUR ENDOSO",
|
||||
"TABLA AUTOS LIMIT R",
|
||||
"BORRA",
|
||||
"GENERICO_OLD",
|
||||
"TIT",
|
||||
},
|
||||
},
|
||||
"scothia": {
|
||||
"path": SOURCE_ROOT / "SCOTHIA.mdb",
|
||||
"schema": "stg_scothia",
|
||||
"exclude": {
|
||||
"INFORME",
|
||||
"INFORME BA",
|
||||
"FECHAIF", # date-range UI parameter table, not data
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
"""
|
||||
Reads tables out of an Access database via pyodbc.
|
||||
|
||||
Column metadata is read from cursor.description after a SELECT * rather than
|
||||
via cursor.columns() — the latter hits a UTF-16 decode bug in pyodbc/the
|
||||
Access ODBC driver on a subset of tables whose column names contain certain
|
||||
accented characters (confirmed against PROPANO, FALTANTES AGUA, TIT in this
|
||||
session). SELECT * + description does not hit that path.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import pyodbc
|
||||
import pandas as pd
|
||||
|
||||
ACCESS_DRIVER = "Microsoft Access Driver (*.mdb, *.accdb)"
|
||||
|
||||
|
||||
def connect(path) -> pyodbc.Connection:
|
||||
conn_str = f"DRIVER={{{ACCESS_DRIVER}}};DBQ={path};"
|
||||
return pyodbc.connect(conn_str, autocommit=True)
|
||||
|
||||
|
||||
def list_tables(cnxn: pyodbc.Connection) -> list[str]:
|
||||
cursor = cnxn.cursor()
|
||||
tables = []
|
||||
for row in cursor.tables(tableType="TABLE"):
|
||||
name = row.table_name
|
||||
if name.startswith("MSys") or name.startswith("~"):
|
||||
continue
|
||||
tables.append(name)
|
||||
return sorted(tables)
|
||||
|
||||
|
||||
def sanitize_column_name(name: str) -> str:
|
||||
"""Access column names are free-form ("NUM id", "A�O1", "TRUST NUM:");
|
||||
Postgres staging columns need to be predictable identifiers. Original
|
||||
name is preserved separately as metadata, this is only for the column
|
||||
identifier itself."""
|
||||
cleaned = re.sub(r"[^0-9a-zA-Z]+", "_", name).strip("_")
|
||||
cleaned = cleaned.lower()
|
||||
if not cleaned:
|
||||
cleaned = "col"
|
||||
if cleaned[0].isdigit():
|
||||
cleaned = f"c_{cleaned}"
|
||||
return cleaned
|
||||
|
||||
|
||||
def read_table(cnxn: pyodbc.Connection, table_name: str) -> pd.DataFrame:
|
||||
cursor = cnxn.cursor()
|
||||
cursor.execute(f"SELECT * FROM [{table_name}]")
|
||||
original_columns = [d[0] for d in cursor.description]
|
||||
|
||||
# fetchall() aborts the whole table on the first bad row. Some legacy
|
||||
# tables (confirmed: MULT) have Jet/ACE-level corruption — a record
|
||||
# marked deleted at the storage level that the driver still enumerates
|
||||
# but can't SQLGetData from ("Record is deleted", HY109). Fetch one row
|
||||
# at a time so a corrupted row is skipped and logged instead of losing
|
||||
# the entire table.
|
||||
data = []
|
||||
skipped = 0
|
||||
while True:
|
||||
try:
|
||||
row = cursor.fetchone()
|
||||
except pyodbc.Error as exc:
|
||||
skipped += 1
|
||||
print(f" [skip row] {table_name}: {exc}")
|
||||
continue
|
||||
if row is None:
|
||||
break
|
||||
data.append(list(row))
|
||||
if skipped:
|
||||
print(f" [{table_name}] skipped {skipped} corrupted row(s)")
|
||||
|
||||
df = pd.DataFrame(data, columns=original_columns)
|
||||
|
||||
# Track original -> sanitized name mapping for the loader; dedupe any
|
||||
# collisions that sanitization could introduce (e.g. "NUM id" and
|
||||
# "NUM_id" both -> "num_id").
|
||||
seen: dict[str, int] = {}
|
||||
sanitized = []
|
||||
for col in original_columns:
|
||||
base = sanitize_column_name(col)
|
||||
if base in seen:
|
||||
seen[base] += 1
|
||||
base = f"{base}_{seen[base]}"
|
||||
else:
|
||||
seen[base] = 0
|
||||
sanitized.append(base)
|
||||
df.columns = sanitized
|
||||
df.attrs["original_columns"] = original_columns
|
||||
|
||||
return df
|
||||
@@ -0,0 +1,120 @@
|
||||
"""
|
||||
Migration plan step 1: raw staging load.
|
||||
|
||||
Dumps every non-scratch table from all four source Access files 1:1 into
|
||||
either MySQL (one database per source system, per config.SOURCES — e.g.
|
||||
`stg_utilities`, `stg_seguros`, `stg_scothia`, matching a MySQL "schema" to
|
||||
a MySQL "database" 1:1) or, if --output-dir is given, to Parquet files —
|
||||
useful for environments (like this one) that have Access + pyodbc available
|
||||
but not a live MySQL instance reachable to load into yet.
|
||||
|
||||
Every staged table gets two extra columns: _legacy_source_table (the
|
||||
original Access table name) and _row_num (ordinal position in the source
|
||||
table) so rows are traceable even before any real key is identified.
|
||||
|
||||
Usage:
|
||||
python load_staging.py --output-dir ./output # Parquet, no DB needed
|
||||
python load_staging.py --database-url mysql+pymysql://user:pass@host/db # load into MySQL
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from config import SOURCES
|
||||
import extract
|
||||
|
||||
|
||||
def stage_source(source_name: str, source_cfg: dict, sink) -> None:
|
||||
path = source_cfg["path"]
|
||||
if not Path(path).exists():
|
||||
print(f" [skip] {source_name}: file not found at {path}", file=sys.stderr)
|
||||
return
|
||||
|
||||
print(f"=== {source_name} ({path}) ===")
|
||||
cnxn = extract.connect(path)
|
||||
tables = extract.list_tables(cnxn)
|
||||
excluded = source_cfg["exclude"]
|
||||
|
||||
for table_name in tables:
|
||||
if table_name in excluded:
|
||||
print(f" [exclude] {table_name}")
|
||||
continue
|
||||
|
||||
try:
|
||||
df = extract.read_table(cnxn, table_name)
|
||||
except Exception as exc: # noqa: BLE001 - report and keep going
|
||||
print(f" [ERROR] {table_name}: {exc}", file=sys.stderr)
|
||||
continue
|
||||
|
||||
df.insert(0, "_legacy_source_table", table_name)
|
||||
df.insert(1, "_row_num", range(len(df)))
|
||||
|
||||
sink(source_cfg["schema"], table_name, df)
|
||||
print(f" [ok] {table_name}: {len(df)} rows, {len(df.columns) - 2} columns")
|
||||
|
||||
|
||||
def make_parquet_sink(output_dir: Path):
|
||||
def sink(schema: str, table_name: str, df: pd.DataFrame) -> None:
|
||||
target_dir = output_dir / schema
|
||||
target_dir.mkdir(parents=True, exist_ok=True)
|
||||
safe_name = extract.sanitize_column_name(table_name)
|
||||
df.to_parquet(target_dir / f"{safe_name}.parquet", index=False)
|
||||
|
||||
return sink
|
||||
|
||||
|
||||
def make_mysql_sink(database_url: str):
|
||||
from sqlalchemy import create_engine, text
|
||||
|
||||
# A MySQL "schema" is a database - CREATE SCHEMA is a synonym for
|
||||
# CREATE DATABASE. Connect without a default database in the URL so
|
||||
# each staging area (stg_utilities/stg_seguros/stg_scothia) can be
|
||||
# created and written to independently by one engine.
|
||||
engine = create_engine(database_url)
|
||||
created_schemas: set[str] = set()
|
||||
|
||||
def sink(schema: str, table_name: str, df: pd.DataFrame) -> None:
|
||||
if schema not in created_schemas:
|
||||
with engine.begin() as conn:
|
||||
conn.execute(text(f"CREATE SCHEMA IF NOT EXISTS `{schema}`"))
|
||||
created_schemas.add(schema)
|
||||
|
||||
safe_name = extract.sanitize_column_name(table_name)
|
||||
df.to_sql(
|
||||
safe_name,
|
||||
engine,
|
||||
schema=schema,
|
||||
if_exists="replace",
|
||||
index=False,
|
||||
)
|
||||
|
||||
return sink
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--output-dir", type=Path, help="Write Parquet files here instead of a database")
|
||||
parser.add_argument("--database-url", type=str, help="MySQL connection string, e.g. mysql+pymysql://user:pass@host:3306/ (sqlalchemy format)")
|
||||
parser.add_argument("--only", type=str, help="Comma-separated subset of source names to run (utilities,seguros,scothia)")
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.output_dir and not args.database_url:
|
||||
parser.error("one of --output-dir or --database-url is required")
|
||||
|
||||
sink = make_parquet_sink(args.output_dir) if args.output_dir else make_mysql_sink(args.database_url)
|
||||
|
||||
only = set(args.only.split(",")) if args.only else None
|
||||
|
||||
for source_name, source_cfg in SOURCES.items():
|
||||
if only and source_name not in only:
|
||||
continue
|
||||
stage_source(source_name, source_cfg, sink)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,5 @@
|
||||
pyodbc>=5.0
|
||||
pandas>=2.2
|
||||
pyarrow>=15.0
|
||||
sqlalchemy>=2.0
|
||||
pymysql>=1.1
|
||||
Generated
+8655
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"name": "jorgecuadros-platform",
|
||||
"private": true,
|
||||
"workspaces": [
|
||||
"apps/*",
|
||||
"packages/*"
|
||||
],
|
||||
"scripts": {
|
||||
"dev:web": "npm run dev -w apps/web",
|
||||
"dev:api": "npm run start:dev -w apps/api",
|
||||
"build": "npm run build -ws --if-present",
|
||||
"prisma:generate": "npm run generate -w packages/database",
|
||||
"prisma:migrate": "npm run migrate:dev -w packages/database",
|
||||
"prisma:studio": "npm run studio -w packages/database"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"name": "@jorgecuadros/database",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"main": "generated/client/index.js",
|
||||
"types": "generated/client/index.d.ts",
|
||||
"scripts": {
|
||||
"generate": "prisma generate",
|
||||
"migrate:dev": "prisma migrate dev",
|
||||
"migrate:deploy": "prisma migrate deploy",
|
||||
"studio": "prisma studio",
|
||||
"validate": "prisma validate",
|
||||
"format": "prisma format"
|
||||
},
|
||||
"dependencies": {
|
||||
"@prisma/client": "^5.20.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"prisma": "^5.20.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,504 @@
|
||||
// Unified customer / insurance / utilities data model.
|
||||
// See C:\Users\ricar\.claude\plans\logical-yawning-tome.md for the migration
|
||||
// plan this schema implements (source: UTILITIES.accdb, SEGUROS 16_be.mdb,
|
||||
// SCOTHIA.mdb). Every model that originates from a legacy Access table
|
||||
// carries legacySource* provenance columns so migrated rows are traceable
|
||||
// back to their Access original and the ETL can be re-run idempotently.
|
||||
|
||||
generator client {
|
||||
provider = "prisma-client-js"
|
||||
output = "../generated/client"
|
||||
}
|
||||
|
||||
datasource db {
|
||||
provider = "mysql"
|
||||
url = env("DATABASE_URL")
|
||||
}
|
||||
|
||||
enum Currency {
|
||||
USD
|
||||
MXN
|
||||
}
|
||||
|
||||
enum TransactionDomain {
|
||||
UTILITY
|
||||
INSURANCE
|
||||
TRUST
|
||||
}
|
||||
|
||||
enum ServiceKind {
|
||||
WATER
|
||||
ELECTRIC
|
||||
GAS
|
||||
CABLE
|
||||
PROPERTY_TAX
|
||||
FEDERAL_ZONE
|
||||
ALARM
|
||||
OTHER
|
||||
}
|
||||
|
||||
enum UserRole {
|
||||
ADMIN
|
||||
STAFF
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Identity — the actual point of the project: one customer record shared by
|
||||
// both business lines.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
model Customer {
|
||||
id String @id @default(uuid())
|
||||
name String
|
||||
addressLine1 String?
|
||||
addressLine2 String?
|
||||
city String?
|
||||
state String?
|
||||
zipCode String?
|
||||
country String?
|
||||
phone String?
|
||||
mobile String?
|
||||
fax String?
|
||||
email String?
|
||||
notes String? @db.Text
|
||||
identificationType String?
|
||||
identificationNumber String?
|
||||
identificationExpiration DateTime?
|
||||
customerSince DateTime?
|
||||
status Boolean @default(true)
|
||||
minimumBalance Decimal? @db.Decimal(12, 2)
|
||||
feeAmount Decimal? @db.Decimal(12, 2)
|
||||
preferredCurrency Currency @default(USD)
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
legacyRefs CustomerLegacyRef[]
|
||||
properties Property[]
|
||||
policies Policy[]
|
||||
vehicles Vehicle[]
|
||||
transactions Transaction[]
|
||||
|
||||
@@map("customers")
|
||||
}
|
||||
|
||||
/// Generalizes the old app's customer_mapping table: one row per legacy
|
||||
/// record folded into this customer, from either source system.
|
||||
model CustomerLegacyRef {
|
||||
id String @id @default(uuid())
|
||||
customerId String
|
||||
customer Customer @relation(fields: [customerId], references: [id])
|
||||
sourceSystem String // "utilities" | "insurance"
|
||||
sourceTable String // e.g. "DATGRAL", "COBRO3"
|
||||
legacyId String // stringified legacy id (source columns are often DOUBLE)
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@unique([sourceSystem, sourceTable, legacyId])
|
||||
@@map("customer_legacy_refs")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Insurance domain
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
model InsuranceProvider {
|
||||
id String @id @default(uuid())
|
||||
name String @unique
|
||||
policies Policy[]
|
||||
|
||||
@@map("insurance_providers")
|
||||
}
|
||||
|
||||
model PolicyType {
|
||||
id String @id @default(uuid())
|
||||
name String @unique
|
||||
shortDescription String?
|
||||
policies Policy[]
|
||||
|
||||
@@map("policy_types")
|
||||
}
|
||||
|
||||
/// Consolidates INCENDIO/MULT/M EMPR/all auto-table variants/LICENCIAS into
|
||||
/// one table with a policyType discriminator, instead of one Access table
|
||||
/// per line of business.
|
||||
model Policy {
|
||||
id String @id @default(uuid())
|
||||
policyNumber String
|
||||
customerId String
|
||||
customer Customer @relation(fields: [customerId], references: [id])
|
||||
policyTypeId String?
|
||||
policyType PolicyType? @relation(fields: [policyTypeId], references: [id])
|
||||
insuranceProviderId String?
|
||||
insuranceProvider InsuranceProvider? @relation(fields: [insuranceProviderId], references: [id])
|
||||
agentName String?
|
||||
policyDate DateTime?
|
||||
policyFrom DateTime?
|
||||
policyTo DateTime?
|
||||
coveragePeriodDays Int? @default(365)
|
||||
netPremium Decimal? @db.Decimal(12, 2)
|
||||
policyFee Decimal? @db.Decimal(12, 2)
|
||||
brokerFee Decimal? @db.Decimal(12, 2)
|
||||
commission Decimal? @db.Decimal(12, 2)
|
||||
total Decimal? @db.Decimal(12, 2)
|
||||
currency Currency @default(MXN)
|
||||
observations String? @db.Text
|
||||
notes String? @db.Text
|
||||
coveragesJson Json?
|
||||
endorsement Boolean @default(false)
|
||||
liquidated Boolean @default(false)
|
||||
liquidationNumber String?
|
||||
liquidationDate DateTime?
|
||||
legacySourceDb String?
|
||||
legacySourceTable String?
|
||||
legacyId String?
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
installments PolicyPaymentInstallment[]
|
||||
vehicles Vehicle[]
|
||||
insuredDrivers InsuredDriver[]
|
||||
beneficiaries PolicyBeneficiary[]
|
||||
claims Claim[]
|
||||
documents PolicyDocument[]
|
||||
properties Property[]
|
||||
|
||||
@@unique([legacySourceDb, legacySourceTable, legacyId])
|
||||
@@index([policyNumber])
|
||||
@@map("policies")
|
||||
}
|
||||
|
||||
/// Unpivots the 4 hardcoded payment-installment columns found on every
|
||||
/// legacy policy table (1ER PAGO/FECHA PAGO/NO CHEQUE, ...2, ...3, ...4).
|
||||
model PolicyPaymentInstallment {
|
||||
id String @id @default(uuid())
|
||||
policyId String
|
||||
policy Policy @relation(fields: [policyId], references: [id])
|
||||
sequence Int
|
||||
amount Decimal? @db.Decimal(12, 2)
|
||||
currency Currency @default(MXN)
|
||||
dueDate DateTime?
|
||||
paidDate DateTime?
|
||||
checkNumber String?
|
||||
isCash Boolean @default(false)
|
||||
|
||||
@@map("policy_payment_installments")
|
||||
}
|
||||
|
||||
/// Unpivots MCA2's 3 hardcoded vehicle slots (and the single-vehicle auto
|
||||
/// tables) into one row per vehicle.
|
||||
model Vehicle {
|
||||
id String @id @default(uuid())
|
||||
customerId String?
|
||||
customer Customer? @relation(fields: [customerId], references: [id])
|
||||
policyId String?
|
||||
policy Policy? @relation(fields: [policyId], references: [id])
|
||||
make String?
|
||||
model String?
|
||||
modelYear String?
|
||||
bodyType String?
|
||||
engineNumber String?
|
||||
licensePlate String?
|
||||
vinNumber String?
|
||||
stateCode String?
|
||||
notes String? @db.Text
|
||||
legacySourceTable String?
|
||||
legacyId String?
|
||||
|
||||
@@map("vehicles")
|
||||
}
|
||||
|
||||
/// Unpivots the repeated named-insured/license columns in MCA2/LICENCIAS.
|
||||
model InsuredDriver {
|
||||
id String @id @default(uuid())
|
||||
policyId String
|
||||
policy Policy @relation(fields: [policyId], references: [id])
|
||||
fullName String?
|
||||
birthDate DateTime?
|
||||
sex String?
|
||||
occupation String?
|
||||
licenseNumber String?
|
||||
licenseState String?
|
||||
|
||||
@@map("insured_drivers")
|
||||
}
|
||||
|
||||
/// From BENEF — already a clean child table in the source data.
|
||||
model PolicyBeneficiary {
|
||||
id String @id @default(uuid())
|
||||
policyId String
|
||||
policy Policy @relation(fields: [policyId], references: [id])
|
||||
name String?
|
||||
address String?
|
||||
phone String?
|
||||
email String?
|
||||
|
||||
@@map("policy_beneficiaries")
|
||||
}
|
||||
|
||||
/// From DATOS (siniestros).
|
||||
model Claim {
|
||||
id String @id @default(uuid())
|
||||
policyId String
|
||||
policy Policy @relation(fields: [policyId], references: [id])
|
||||
claimType String?
|
||||
incidentDate DateTime?
|
||||
reportedDate DateTime?
|
||||
description String? @db.Text
|
||||
adjusterId String?
|
||||
adjuster Adjuster? @relation(fields: [adjusterId], references: [id])
|
||||
claimedAmount Decimal? @db.Decimal(12, 2)
|
||||
settledAmount Decimal? @db.Decimal(12, 2)
|
||||
settlementDate DateTime?
|
||||
checkNumber String?
|
||||
resolved Boolean @default(false)
|
||||
resolutionNotes String? @db.Text
|
||||
|
||||
@@map("claims")
|
||||
}
|
||||
|
||||
/// From AJUSTADORES / AJUSTADORESATLAS.
|
||||
model Adjuster {
|
||||
id String @id @default(uuid())
|
||||
company String?
|
||||
city String?
|
||||
name String?
|
||||
phone String?
|
||||
beeper String?
|
||||
claims Claim[]
|
||||
|
||||
@@map("adjusters")
|
||||
}
|
||||
|
||||
/// Extracted LONGBINARY blobs from the policy tables — file lives in object
|
||||
/// storage, only the pointer + type lives here.
|
||||
model PolicyDocument {
|
||||
id String @id @default(uuid())
|
||||
policyId String
|
||||
policy Policy @relation(fields: [policyId], references: [id])
|
||||
documentType String
|
||||
storageKey String
|
||||
originalColumn String?
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@map("policy_documents")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Utilities domain
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// From DATMEX — shared with the insurance domain so a property can carry
|
||||
/// both a home-insurance policy and utility service enrollments.
|
||||
model Property {
|
||||
id String @id @default(uuid())
|
||||
customerId String
|
||||
customer Customer @relation(fields: [customerId], references: [id])
|
||||
policyId String?
|
||||
policy Policy? @relation(fields: [policyId], references: [id])
|
||||
addressLine1 String?
|
||||
addressLine2 String?
|
||||
phone1 String?
|
||||
phone2 String?
|
||||
phone3 String?
|
||||
zone String?
|
||||
legacySourceTable String?
|
||||
legacyId String?
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
services PropertyService[]
|
||||
documents ServiceDocument[]
|
||||
trustAccount TrustAccount?
|
||||
|
||||
@@map("properties")
|
||||
}
|
||||
|
||||
/// Unpivots DATMEX's inline service columns and PROFILE's enrollment flags
|
||||
/// into one row per enrolled service per property.
|
||||
model PropertyService {
|
||||
id String @id @default(uuid())
|
||||
propertyId String
|
||||
property Property @relation(fields: [propertyId], references: [id])
|
||||
kind ServiceKind
|
||||
accountNumber String?
|
||||
meterNumber String?
|
||||
route String?
|
||||
dueDay String?
|
||||
active Boolean @default(true)
|
||||
notes String? @db.Text
|
||||
|
||||
@@map("property_services")
|
||||
}
|
||||
|
||||
model ServiceDocument {
|
||||
id String @id @default(uuid())
|
||||
propertyId String
|
||||
property Property @relation(fields: [propertyId], references: [id])
|
||||
documentType String
|
||||
storageKey String
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@map("service_documents")
|
||||
}
|
||||
|
||||
/// From TRUSTVENCE.
|
||||
model TrustAccount {
|
||||
id String @id @default(uuid())
|
||||
propertyId String @unique
|
||||
property Property @relation(fields: [propertyId], references: [id])
|
||||
bankName String?
|
||||
trustNumber String?
|
||||
bankFee Decimal? @db.Decimal(12, 2)
|
||||
dueDate1 DateTime?
|
||||
dueDate2 DateTime?
|
||||
|
||||
@@map("trust_accounts")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Shared financial ledger — one office, one set of books.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// ES/EN transaction-type lookup, carried over from the old schema.
|
||||
model TypeTransaction {
|
||||
id String @id @default(uuid())
|
||||
nameEn String
|
||||
nameEs String?
|
||||
isService Boolean @default(false)
|
||||
transactions Transaction[]
|
||||
|
||||
@@map("type_transactions")
|
||||
}
|
||||
|
||||
/// Unifies utilities' EFECTIVO/EFECTIVO FM3/EFECTIVO_BACKUP/FEE ANUAL/
|
||||
/// datos2/fee15/billing/CHEQUE FM3/IVA 2015 and insurance's EFECTIVO.
|
||||
model Transaction {
|
||||
id String @id @default(uuid())
|
||||
customerId String
|
||||
customer Customer @relation(fields: [customerId], references: [id])
|
||||
domain TransactionDomain
|
||||
typeId String?
|
||||
type TypeTransaction? @relation(fields: [typeId], references: [id])
|
||||
transactionDate DateTime
|
||||
period String?
|
||||
reference String?
|
||||
amount Decimal @db.Decimal(12, 2)
|
||||
currency Currency @default(MXN)
|
||||
exchangeRate Decimal? @db.Decimal(10, 4)
|
||||
checkNumber String?
|
||||
message String? @db.Text
|
||||
outstanding Boolean @default(false)
|
||||
legacySourceDb String?
|
||||
legacySourceTable String?
|
||||
legacyId String?
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@index([customerId, transactionDate])
|
||||
@@map("transactions")
|
||||
}
|
||||
|
||||
/// From TIPO HIST.
|
||||
model ExchangeRate {
|
||||
id String @id @default(uuid())
|
||||
rate Decimal @db.Decimal(10, 4)
|
||||
effectiveDate DateTime
|
||||
effectiveHour DateTime?
|
||||
|
||||
@@map("exchange_rates")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Company bank register (SCOTHIA.mdb) — the office's own operating account,
|
||||
// deliberately separate from customer-facing Transaction records.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// From TABLA RAMODOS ("ramo" = line of business).
|
||||
model BusinessLineCategory {
|
||||
id String @id @default(uuid())
|
||||
name String @unique
|
||||
bankTransactions BankTransaction[]
|
||||
|
||||
@@map("business_line_categories")
|
||||
}
|
||||
|
||||
/// Unifies SCOTHIA's DATOS E (egresos) / DATOS I (ingresos) into one
|
||||
/// signed-amount table: income positive, expense negative.
|
||||
model BankTransaction {
|
||||
id String @id @default(uuid())
|
||||
transactionDate DateTime
|
||||
transactionType String?
|
||||
reference String?
|
||||
concept String?
|
||||
amount Decimal @db.Decimal(12, 2)
|
||||
categoryId String?
|
||||
category BusinessLineCategory? @relation(fields: [categoryId], references: [id])
|
||||
cleared Boolean @default(false)
|
||||
transferred Boolean @default(false)
|
||||
notes String? @db.Text
|
||||
amountInWords String?
|
||||
legacySourceTable String?
|
||||
legacyId String?
|
||||
|
||||
@@map("bank_transactions")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Admin / shared
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
model User {
|
||||
id String @id @default(uuid())
|
||||
name String
|
||||
email String @unique
|
||||
passwordHash String
|
||||
role UserRole @default(STAFF)
|
||||
active Boolean @default(true)
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
activityLogs ActivityLog[]
|
||||
|
||||
@@map("users")
|
||||
}
|
||||
|
||||
model ActivityLog {
|
||||
id String @id @default(uuid())
|
||||
userId String?
|
||||
user User? @relation(fields: [userId], references: [id])
|
||||
event String
|
||||
level String
|
||||
message Json?
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@map("activity_logs")
|
||||
}
|
||||
|
||||
model EmailTemplate {
|
||||
id String @id @default(uuid())
|
||||
name String
|
||||
subject String
|
||||
templateSource String @db.Text
|
||||
|
||||
@@map("email_templates")
|
||||
}
|
||||
|
||||
model EmailCampaign {
|
||||
id String @id @default(uuid())
|
||||
campaignName String
|
||||
subject String?
|
||||
body String? @db.Text
|
||||
status String @default("in_progress")
|
||||
emailSentCount Int @default(0)
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@map("email_campaigns")
|
||||
}
|
||||
|
||||
model EmailLog {
|
||||
id String @id @default(uuid())
|
||||
customerId String?
|
||||
emailAddress String?
|
||||
emailType String?
|
||||
requestBody String? @db.Text
|
||||
responseBody String? @db.Text
|
||||
sentAt DateTime @default(now())
|
||||
|
||||
@@map("email_log")
|
||||
}
|
||||
Reference in New Issue
Block a user