344319f45606a8233188e2f13d94b03000637bb6
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  rmancinasandClaude Opus 5.5
|
fd5df240bc |
feat(recibos): split a receipt, sin fondos per receipt, keep a batch open
Three asks from the 2026-10-08 meeting, all on the OCR receipt queue. - Split: one receipt paid by several customers (411 + 523 sharing water 3014057) posts one charge per part. statement_document_splits holds free amounts that must sum to the receipt in cents, checked on save and again at confirm; "partes iguales" fills them in. Each part's row carries captureRef <docId>#<n>, so the double-post guard still holds. - Sin fondos per receipt: StatementDocument.outstanding, toggled in review. At confirm a line is outstanding if its receipt or the whole check is. - Open batch: POST /statements/batches/:id/files appends scans, numbered after the existing pages; a COMPLETED batch reopens. The confirm card prefills the check the batch was last posted against. Verified on dev against a real batch: 1,892.00 split 946/946 across two customers, a sibling receipt posted outstanding, second confirm refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5.5
|
031b13f8c8 |
fix(reports): balances the statement agrees with, NUMid on every report
From the 2026-10-08 meeting. - Clientes en rojo summed every row a customer ever had, so the opening balance's history counted twice, cash receipts twice and the 2025 archive on top: 195 customers in the red against 22 real ones. It now takes the balance scope balances() uses (extracted as balanceScopeSql), sorts by most recent real movement, and filters by ramo, date, minimum owed and inactive customers (status = 0, the internal accounts). - Pagos no efectuados asked "no charge of any kind in N days"; legacy asks "no row of THAT service's concept in the period". Rewritten to legacy's question, per customer, defaulting to one billing cycle back. It also passed ELECTRICITY/PHONE, which are not ServiceKind values, so Luz and Telefono never returned a row. Sin fondos rows show as their own status. - Faltantes demanded account, meter AND route of every kind; water and phone have no route. Now the legacy fields per kind. - Reporte de efectivo counted every credit BALANCE FORWARD as cash (958 of 1093 rows on dev). - Every report carries the legacy NUMid; policy reports add the insurance file number. The statement header shows it too. - Printed headers showed placeholder company details: prod sets no COMPANY_* variables. Defaults are now the office's real locator, plus a US mailing address line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
19864f16f2 |
fix(ocr): keep the printed layout when rebuilding text from word boxes
The parsers are written against `pdftotext -layout`, and every policy-ocr
fixture is a verbatim excerpt of it. The runtime does not use it: it reads
`-bbox-layout` and rebuilds the page from word boxes, and that rebuild
collapsed all white space — no blank lines between blocks, one space
between columns. White space is the only thing marking a cell boundary on
these borderless forms, so the fixtures could not see any of it.
What it cost, on the GMX PVL especificación and the ANA driver's policy:
- `espectBlock` walks a wrapped cell until a blank line. With no blank
line it ran to the end of the page, so the insured's name came back as
the entire first page of the specification.
- `INSURED\s{2,}` and its siblings matched nothing, and the phone that
shares the name cell rode along with it ("PAMELA DENISE WAGONER
Ph.3102001538"), which matches no customer.
- `parseAnaDriverCoverages` splits SUM INSURED from PREMIUM by the
header's own column offsets. Without offsets, every premium was filed
as a sum insured.
So `toVisualRows` now emits a blank line where the reader sees one (a
vertical gap over 1.6 line heights — the two populations measure 0.3-1.1
and 2.1+, so the threshold sits in empty space) and pads each word to its
own column, using one space wherever words merely follow each other so
rounding drift cannot sprinkle false cell boundaries through prose.
Two independent guards, so neither failure can come back silently: the
ANA phone splits on a single space, and the especificación's cell walk is
capped at the one wrap the longest cell on that document actually uses.
Verified against the real PDFs: the especificación reads "EMMER .
KATHLEEN" with all 18 coverages named (they were "(sin nombre)"), and the
ten born-digital gas invoices parse byte-identically to before. The
scanned statements are untouched — they come through tesseract, not this
path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
rmancinas
|
3125b52057 |
feat(ocr): discard abandoned capture batches
A bad scan, the wrong PDFs or a duplicate upload used to leave a batch sitting in READY_FOR_REVIEW forever, because the only exits were confirm (posts to the books) or rejecting every page one at a time. Add a DISCARDED terminal status to both OCR domains and a single endpoint per domain that rejects every page still pending in one shot. Discarding is refused once anything has landed: statements once a page is POSTED, policies once a page is APPLIED. Those batches did real work and have to be settled page by page. - POST /statements/batches/:id/discard - POST /policy-ocr/batches/:id/discard - shared DiscardBatchCard on both review screens, gated the same way |
||
|
rmancinasandClaude Opus 5
|
5e9cb12fba |
feat(polizas): OCR capture for insurance policy PDFs
Mirrors the utility statement intake on the insurance side: a policy_ocr batch/document pair of tables, a GMX parser, a matcher keyed on Policy.policyNumber, and a "Captura" screen under /polizas that proposes policy -> customer for staff to confirm. Lifts the OCR seam out of StatementsModule into its own OcrModule so PolicyOcrModule can inject OCR_PROVIDER without taking on the rest of the statement pipeline; StatementsModule now imports it and binds nothing itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
5bce0e4c94 |
feat(recibos): OCR capture for zona federal (ZOFEMAT Tijuana)
Adds the ZONA FEDERAL TIJUANA parser to the statement intake, measured against 8 pages of real "Zona Federal Marítimo Terrestre" receipts — the federal maritime-zone occupancy fee the municipality bills on beachfront lots. Provider read on 8/8, amount on 8/8 (each verified against the paper), concession clave on 6/8, period on 8/8, deadline on 2/8. Four things the corpus forced: - Tijuana bills predial and zona federal from the same treasury: same header, same Paseo del Centenario address, same ATB-541201 RFC. Every predial discriminator matches a zona federal page too, so whichever rule is asked first wins it. The only words exclusive to this layout are "Marítimo Terrestre", so its brand rule is asked ahead of all three predial ones — and its structural rule, anchored on the stub's "Derechos de ocupación", ahead of theirs. - FEDERAL_ZONE.accountNumber is an amount, not a reference. It holds DATMEX.zfed, whose 77 values include 246.06, 2369.09, 22653.94 and a negative -1679, while the concession claves these receipts are keyed by appear nowhere in the database. Matching on that column could never hit — and because every row already has a value, the `[field]: null` guards on learnAccountRefs and on the review blank-service fill would never fire either, so every page would return to the queue every bimester forever. The clave moves to meterNumber, joining gas and Tijuana predial, and the first confirm teaches the match. - The payable figure is not the printed subtotal. The municipality rounds to whole pesos and prints the difference on its own "Ajuste Ley Hacienda Mpal" line (-$0.05 against a 591.05 subtotal, $0.21 against 2,872.79). The "Total a pagar" box carrying the rounded figure sits on a grey fill and OCR'd on 1 of 8 pages; the SubTotal row read on 8 of 8. So the amount is the rounded subtotal, cross-checked against the printed box wherever it survives — where it did, it agreed. - The clave is 2 digits, a letter and 3 digits (12-T -012), not the cadastral shape, and the letter is kept as printed: toDigits maps D to 0, which turns a real 14-D -014 into 140014. It is printed twice, which rescued a page whose heading was struck through by the office's own highlighter — the failure mode behind both missing claves. Deriving the deadline from the bimester is deliberately not attempted: it is the 17th of the month after the bimester closes on a current bill, but four of these eight are late (a $1,000 Multa) and print a recalculated date, so a derived date would be wrong on exactly the pages a human most wants to see. Re-ran the earlier corpora (25 pages: predial Tijuana/Rosarito/Ensenada, CFE, CESPT, Telnor) through detection to confirm the new rules steal nothing — all 25 still read as their original provider, including the five Tijuana predial pages that share the RFC. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
d6501f1d74 |
feat(recibos): OCR capture for gas butano and municipal predial
Adds four parsers to the statement intake — GAS TIJUANA plus one per municipality, because Tijuana, Rosarito and Ensenada issue three completely different predial documents — and a text-layer fast path for the born-digital invoices the gas company sends. Measured against a new corpus of 14 documents / 29 pages: provider read on 29/29, amount on 26/29, and 21/29 auto-matched against the dev database (22/29 identified). The eight review cases are all legitimate. Five things the corpus forced: - Not every statement is a scan. The gas invoices are born-digital CFDIs whose text layer is exact; rasterising them only loses information (one sample turned `MEDIDOR: VM01014426` into `ar (LTR): 014420`). The new `OcrProvider.textPages` reads the embedded layer via `pdftotext -bbox-layout` — same poppler package as `pdftoppm`, so no new dependency — and OCR stays the fallback for real scans. Poppler's own `<line>` grouping follows text flow rather than the page, so words are regrouped by vertical position; without that, a two-column header leaves every label separated from the value printed beside it. - The clave catastral is not two letters and six digits. Position three is a letter in 15 of the 932 stored claves, and digitising the whole tail mapped a real `MMB01041` to a nonexistent `MM801041`. - Tijuana predial prints no clave at all. Its only identifier is an 8-digit municipal account carried in a 32-digit payment barcode, which the legacy database never held, so it goes in `meterNumber` alongside gas — `accountNumber` holds `DATMEX.predial`, which is not a per-property key and must not be overwritten. Those pages start cold and are taught by the first confirm. - On Rosarito and Ensenada the clave is the primary key, not a fallback: those receipts print nothing else, so a unique hit auto-matches. On a utility bill that merely happens to print one it stays a review hint. - A misread `$` is the dangerous failure. An Ensenada receipt for $2,203.00 OCR'd as `82,203.00`, which would post a charge 37x too large and look ordinary in the ledger. Predial amounts now require a literal `$` and a page that cannot produce one goes to review. The scoped match field is now one exported function rather than three copies of `kind === "GAS" ? ... : ...`, since the lookup, the blank-service fill and the confirm write-back have to agree or a reference gets learned into a column nothing searches. First tests in this package: 23 specs over the parsers and the text-layer reader, every fixture a verbatim OCR excerpt from a real receipt. Adds the jest config they need and a build tsconfig so they stay out of dist. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
b59abda895 |
feat(captura): fold recibo OCR into Captura as an automatic mode
Scanning a stack of bills and keying them in are the same daily job, ending in the same ledger path, so OCR intake becomes a mode of the capture screen instead of a second menu entry: - components/Captura.tsx holds the mode switch; the manual check form moves verbatim to components/ManualCheckCapture.tsx and the OCR intake to components/StatementIntake.tsx. - /estado-cuenta/lote opens on manual, /recibos on automatic — both render Captura, so batch-review links and old bookmarks still land right. - Nav drops "Recibos (OCR)"; "Captura" covers both, with a NavLink.aliases field so /recibos still highlights it. Also fixes the "El almacenamiento de documentos no está configurado" failure staff hit on upload. Uploading with no object storage configured used to succeed, then die on the first put minutes later, leaving a FAILED batch whose only explanation was that string. createBatch now refuses up front, GET /statements/status reports storageAvailable alongside ocrAvailable, and the intake tab explains the situation instead of offering an upload that cannot work. S3_* documented in .env.example (deploy stacks already set it). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
rmancinasandClaude Opus 5
|
4d5008b545 |
feat(statements): OCR intake for scanned utility bills
Staff key 300+ utility statements per company per month by hand. This adds the ingest -> split -> OCR -> match -> review pipeline that proposes customer and amount per page instead (RECEIPT_CAPTURE_SPEC §2), posting through the existing BillingService.createBatch seam with source=OCR and a per-document captureRef so machine and hand capture share one write path and audit trail. Everything was designed against 10 real scanned statements (46 pages of CFE, CESPT and Telnor bills) rather than from the sample-free spec. The scans have no text layer at all — they are camera images — so OCR is mandatory, and they arrive bundled one customer per page. Measured on those pages the parser identifies the provider 46/46 and reads an account reference 43/46; against the dev database that is 39/46 (85%) exact auto-match, 40/46 identified, with the rest genuine review cases. That closes the OCR-provider question in favour of self-hosted Tesseract: it clears the bar for a queue where a human confirms every row, and OcrProvider keeps a managed API a one-line swap. The samples corrected three things the spec had wrong or unknown: - Clave catastral is NOT predial. DATMEX.clave (934 rows) is what CESPT and predial bills print; DATMEX.predial, which PROPERTY_TAX.accountNumber holds, has 663 distinct values across 1135 rows and appears on no statement. The clave now lives on Property.cadastralKey as the matcher's secondary key; predial is left untouched. This had been blocking predial matching. - Gas was recoverable: 160 of 334 DATMEX.gas values are real account numbers (the rest are ESTACIONARIO/CILINDRO descriptors), now in GAS.meterNumber. - Phone is one billed line per property (534/18/1 across phone1/2/3), so the new TELEPHONE ServiceKind backfills from phone1 only, not three rows. Matching is scoped to one column per service kind and never reads the customer name — a CESPT receipt prints ARNAIZ ROSAS ELSA AURORA for an account this office holds under CATT, RANDY, because the printed name is the registrant, not the current owner. Where a provider prints a payment barcode it beats the printed label (one CFE label OCR'd a digit too many while its barcode was correct) and the two cross-check, with disagreement forcing review. Confirming a document whose service had no reference writes it back, so gas and any other cold start is a one-time cost rather than a permanent queue. Verified end to end against the live dev API and MinIO: real scans uploaded over HTTP, matched, confirmed against a check, and the resulting rows checked in MySQL (negative amounts, captureSource=OCR, concept derived from the batch kind, captureRef linking back to each page). Re-confirming a posted batch is refused. Test data was removed afterwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |