Port extraction layer from pyodbc to mdbtools for macOS

macOS has no Access ODBC driver, so the pyodbc-based extract.py could not
run. Rewrite it to shell out to mdbtools (mdb-tables/mdb-export) while
keeping the public interface (connect/list_tables/read_table/
sanitize_column_name) unchanged, so load_staging.py and config.py are
untouched. connect() now returns the file path as the handle (mdbtools has
no persistent connection).

Behavior details:
- -b strip drops LONGBINARY/OLE bytes so blobs never corrupt the CSV
  (documents are extracted separately in migration step 4).
- ISO date/datetime output (-D/-T); staging read as text (dtype=str),
  only empty fields treated as null (keep_default_na=False) so literal
  "NA"/"NULL" data strings survive.
- mdbtools reads deleted/corrupted records mdbtools omits rather than
  aborting, so the old per-row skip loop is no longer needed.

Verified end-to-end: full staging load reproduces the original Windows
run (82 tables, 0 errors) with exact row counts (datos2 16000, EFECTIVO
13697, DATGRAL 1172/1070, DATMEX 1520) and recovers all 764 MULT rows
(the pyodbc path lost 1 to HY109 corruption). Accented-column tables
(PROPANO) read cleanly.

requirements.txt: drop pyodbc/pywin32 (Windows-only), keep pandas/pyarrow/
sqlalchemy/pymysql; document the Windows-only DAO catalog as historical.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-22 17:15:02 -07:00
co-authored by Claude Opus 4.8
parent 22a53bc562
commit dc9866c24a
2 changed files with 124 additions and 48 deletions
+8 -2
View File
@@ -1,6 +1,12 @@
pyodbc>=5.0
# Extraction on macOS uses mdbtools (brew install mdbtools) via subprocess,
# not pyodbc — no Access ODBC driver exists on macOS. See extract.py.
pandas>=2.2
pyarrow>=15.0
sqlalchemy>=2.0
pymysql>=1.1
pywin32>=306 # Windows only - needed for catalog_objects.py (DAO COM automation)
# Windows-only, historical — the DAO/COM object catalog (catalog_objects.py)
# was already run on Windows and its output is committed (objects.json). Not
# needed on macOS; left documented for provenance.
# pyodbc>=5.0
# pywin32>=306