master
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  rmancinasandClaude Opus 5
|
ec139737be |
docs: as-built reference for the statement OCR capture
Gives receipt capture the same treatment policy OCR just got: a doc that records what is in the code, separate from the spec that records what was designed. RECEIPT_CAPTURE_SPEC.md §2 had accumulated three BUILT notes totalling ~120 lines of findings, which is the right place for the evidence but the wrong place to look up how the matcher picks a column. docs/STATEMENT_OCR.md covers the pipeline, the OCR seam and its text-layer-first rule, all eight parsers and the ordering constraints between them, the matcher's two governing rules and the scopedRefField table, confirm-through-BillingService, the learning write-back, and the API surface. Weight goes to the things that are load-bearing and invisible from the code shape: brand detection must run to completion before layout because Tijuana bills predial and zona federal off the same treasury header; scopedRefField is exported because three call sites must agree or a reference gets learned into a column nothing searches; FEDERAL_ZONE's accountNumber holds a peso amount, so it fails the null-guards as well as the lookup; a misread `$` is the dangerous failure, not a missing one. Also records that CFE/CESPT/Telnor have no unit suite — they predate the gas/predial extension and were only verified end to end. Cross-linked from the spec, POLICY_OCR.md, PLAN.md, README and RESUME.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
872a661051 |
docs: document policy OCR capture, the feature no spec proposed
Policy OCR shipped 2026-08-01 (
|
||
|
rmancinasandClaude Opus 5
|
5bce0e4c94 |
feat(recibos): OCR capture for zona federal (ZOFEMAT Tijuana)
Adds the ZONA FEDERAL TIJUANA parser to the statement intake, measured against 8 pages of real "Zona Federal Marítimo Terrestre" receipts — the federal maritime-zone occupancy fee the municipality bills on beachfront lots. Provider read on 8/8, amount on 8/8 (each verified against the paper), concession clave on 6/8, period on 8/8, deadline on 2/8. Four things the corpus forced: - Tijuana bills predial and zona federal from the same treasury: same header, same Paseo del Centenario address, same ATB-541201 RFC. Every predial discriminator matches a zona federal page too, so whichever rule is asked first wins it. The only words exclusive to this layout are "Marítimo Terrestre", so its brand rule is asked ahead of all three predial ones — and its structural rule, anchored on the stub's "Derechos de ocupación", ahead of theirs. - FEDERAL_ZONE.accountNumber is an amount, not a reference. It holds DATMEX.zfed, whose 77 values include 246.06, 2369.09, 22653.94 and a negative -1679, while the concession claves these receipts are keyed by appear nowhere in the database. Matching on that column could never hit — and because every row already has a value, the `[field]: null` guards on learnAccountRefs and on the review blank-service fill would never fire either, so every page would return to the queue every bimester forever. The clave moves to meterNumber, joining gas and Tijuana predial, and the first confirm teaches the match. - The payable figure is not the printed subtotal. The municipality rounds to whole pesos and prints the difference on its own "Ajuste Ley Hacienda Mpal" line (-$0.05 against a 591.05 subtotal, $0.21 against 2,872.79). The "Total a pagar" box carrying the rounded figure sits on a grey fill and OCR'd on 1 of 8 pages; the SubTotal row read on 8 of 8. So the amount is the rounded subtotal, cross-checked against the printed box wherever it survives — where it did, it agreed. - The clave is 2 digits, a letter and 3 digits (12-T -012), not the cadastral shape, and the letter is kept as printed: toDigits maps D to 0, which turns a real 14-D -014 into 140014. It is printed twice, which rescued a page whose heading was struck through by the office's own highlighter — the failure mode behind both missing claves. Deriving the deadline from the bimester is deliberately not attempted: it is the 17th of the month after the bimester closes on a current bill, but four of these eight are late (a $1,000 Multa) and print a recalculated date, so a derived date would be wrong on exactly the pages a human most wants to see. Re-ran the earlier corpora (25 pages: predial Tijuana/Rosarito/Ensenada, CFE, CESPT, Telnor) through detection to confirm the new rules steal nothing — all 25 still read as their original provider, including the five Tijuana predial pages that share the RFC. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
d6501f1d74 |
feat(recibos): OCR capture for gas butano and municipal predial
Adds four parsers to the statement intake — GAS TIJUANA plus one per municipality, because Tijuana, Rosarito and Ensenada issue three completely different predial documents — and a text-layer fast path for the born-digital invoices the gas company sends. Measured against a new corpus of 14 documents / 29 pages: provider read on 29/29, amount on 26/29, and 21/29 auto-matched against the dev database (22/29 identified). The eight review cases are all legitimate. Five things the corpus forced: - Not every statement is a scan. The gas invoices are born-digital CFDIs whose text layer is exact; rasterising them only loses information (one sample turned `MEDIDOR: VM01014426` into `ar (LTR): 014420`). The new `OcrProvider.textPages` reads the embedded layer via `pdftotext -bbox-layout` — same poppler package as `pdftoppm`, so no new dependency — and OCR stays the fallback for real scans. Poppler's own `<line>` grouping follows text flow rather than the page, so words are regrouped by vertical position; without that, a two-column header leaves every label separated from the value printed beside it. - The clave catastral is not two letters and six digits. Position three is a letter in 15 of the 932 stored claves, and digitising the whole tail mapped a real `MMB01041` to a nonexistent `MM801041`. - Tijuana predial prints no clave at all. Its only identifier is an 8-digit municipal account carried in a 32-digit payment barcode, which the legacy database never held, so it goes in `meterNumber` alongside gas — `accountNumber` holds `DATMEX.predial`, which is not a per-property key and must not be overwritten. Those pages start cold and are taught by the first confirm. - On Rosarito and Ensenada the clave is the primary key, not a fallback: those receipts print nothing else, so a unique hit auto-matches. On a utility bill that merely happens to print one it stays a review hint. - A misread `$` is the dangerous failure. An Ensenada receipt for $2,203.00 OCR'd as `82,203.00`, which would post a charge 37x too large and look ordinary in the ledger. Predial amounts now require a literal `$` and a page that cannot produce one goes to review. The scoped match field is now one exported function rather than three copies of `kind === "GAS" ? ... : ...`, since the lookup, the blank-service fill and the confirm write-back have to agree or a reference gets learned into a column nothing searches. First tests in this package: 23 specs over the parsers and the text-layer reader, every fixture a verbatim OCR excerpt from a real receipt. Adds the jest config they need and a build tsconfig so they stay out of dist. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
b59abda895 |
feat(captura): fold recibo OCR into Captura as an automatic mode
Scanning a stack of bills and keying them in are the same daily job, ending in the same ledger path, so OCR intake becomes a mode of the capture screen instead of a second menu entry: - components/Captura.tsx holds the mode switch; the manual check form moves verbatim to components/ManualCheckCapture.tsx and the OCR intake to components/StatementIntake.tsx. - /estado-cuenta/lote opens on manual, /recibos on automatic — both render Captura, so batch-review links and old bookmarks still land right. - Nav drops "Recibos (OCR)"; "Captura" covers both, with a NavLink.aliases field so /recibos still highlights it. Also fixes the "El almacenamiento de documentos no está configurado" failure staff hit on upload. Uploading with no object storage configured used to succeed, then die on the first put minutes later, leaving a FAILED batch whose only explanation was that string. createBatch now refuses up front, GET /statements/status reports storageAvailable alongside ocrAvailable, and the intake tab explains the situation instead of offering an upload that cannot work. S3_* documented in .env.example (deploy stacks already set it). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
4d5008b545 |
feat(statements): OCR intake for scanned utility bills
Staff key 300+ utility statements per company per month by hand. This adds the ingest -> split -> OCR -> match -> review pipeline that proposes customer and amount per page instead (RECEIPT_CAPTURE_SPEC §2), posting through the existing BillingService.createBatch seam with source=OCR and a per-document captureRef so machine and hand capture share one write path and audit trail. Everything was designed against 10 real scanned statements (46 pages of CFE, CESPT and Telnor bills) rather than from the sample-free spec. The scans have no text layer at all — they are camera images — so OCR is mandatory, and they arrive bundled one customer per page. Measured on those pages the parser identifies the provider 46/46 and reads an account reference 43/46; against the dev database that is 39/46 (85%) exact auto-match, 40/46 identified, with the rest genuine review cases. That closes the OCR-provider question in favour of self-hosted Tesseract: it clears the bar for a queue where a human confirms every row, and OcrProvider keeps a managed API a one-line swap. The samples corrected three things the spec had wrong or unknown: - Clave catastral is NOT predial. DATMEX.clave (934 rows) is what CESPT and predial bills print; DATMEX.predial, which PROPERTY_TAX.accountNumber holds, has 663 distinct values across 1135 rows and appears on no statement. The clave now lives on Property.cadastralKey as the matcher's secondary key; predial is left untouched. This had been blocking predial matching. - Gas was recoverable: 160 of 334 DATMEX.gas values are real account numbers (the rest are ESTACIONARIO/CILINDRO descriptors), now in GAS.meterNumber. - Phone is one billed line per property (534/18/1 across phone1/2/3), so the new TELEPHONE ServiceKind backfills from phone1 only, not three rows. Matching is scoped to one column per service kind and never reads the customer name — a CESPT receipt prints ARNAIZ ROSAS ELSA AURORA for an account this office holds under CATT, RANDY, because the printed name is the registrant, not the current owner. Where a provider prints a payment barcode it beats the printed label (one CFE label OCR'd a digit too many while its barcode was correct) and the two cross-check, with disagreement forcing review. Confirming a document whose service had no reference writes it back, so gas and any other cold start is a one-time cost rather than a permanent queue. Verified end to end against the live dev API and MinIO: real scans uploaded over HTTP, matched, confirmed against a check, and the resulting rows checked in MySQL (negative amounts, captureSource=OCR, concept derived from the batch kind, captureRef linking back to each page). Re-confirming a posted batch is refused. Test data was removed afterwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
9ba5d2d09a |
feat(bank): multi-bank chequera — required bankAccountId, per-account scoping
The office keeps more than one operating account (Utilities banks in MXN, Seguros in USD), but bank_transactions was a single implicit MXN register by design. Adds Bank/BankAccount and makes every read and write in the module scoped to exactly one account. Schema: - Bank / BankAccount. Currency is fixed per account and BankTransaction has no currency column of its own — a movement inherits its account's, the way a real bank account doesn't mix currencies. - BankTransaction.bankAccountId, required. A movement with no known account isn't reconcilable against a statement. - @@index([bankAccountId, transactionDate]): every read now filters by account and orders/groups by date. Migration: - backfill_bank_accounts.py seeds Scotiabank + "Utilities — Scotiabank (MXN)" and backfills all 22,669 existing rows onto it, then promotes the column to NOT NULL and attaches the FK. Standalone because prisma db push cannot add a required column to a populated table. Idempotent; re-running once a second account exists does not re-point rows. - run_all.py runs it (both modes) before transform_bank.py, which now resolves the account by label and fails fast if it is missing. API: - ?bankAccountId= required on list/stats/facets/summary — not optional with an "all accounts" default, since summing an MXN and a USD register repeats the currency-collapsing mistake the billing module exists to prevent. Missing is 400, unknown is 404. - facets() had no account clause at all and summary() has two raw-SQL rollups; all three are now parameterised. Scoping only one of summary's queries would leave the year list and its drill-down describing different books. - New bank/accounts + bank/banks sub-resource under a MANAGER bank:manage-accounts ability. currency is absent from the update DTO: booked movements are denominated in it, so editing would re-denominate history. Capture into a closed account is rejected. Web: - /banco gains an account picker (remembered per browser) and reads every figure in the selected account's currency; the "single currency (MXN)" doc-comment and the hardcoded MXN formatting are gone. - New /banco/cuentas for banks and accounts. Accounts are closed, never deleted — the FK is required, so deleting one would destroy its register. - /inicio's chequera card names the account it is reading instead of implying a single register. Verified against dev + browser: a second USD account showed full read/write isolation from the MXN register, whose totals were unchanged (22,669 movements, net 1,014,266.97). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Sonnet 5
|
9b9ee201c9 |
docs: add receipt capture, OCR, multi-bank & customer-recycling spec
Forward implementation spec covering the legacy "Editor" receipt-capture workflow plus three net-new requests from the 2026-07-25/26 meeting with Jorge: PDF/OCR auto-capture, multi-bank chequera support, and customer-number recycling. Matching logic and data-model gaps for each were verified against the actual migration scripts and API code, not just the schema comments. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |