Port extraction layer from pyodbc to mdbtools for macOS
macOS has no Access ODBC driver, so the pyodbc-based extract.py could not run. Rewrite it to shell out to mdbtools (mdb-tables/mdb-export) while keeping the public interface (connect/list_tables/read_table/ sanitize_column_name) unchanged, so load_staging.py and config.py are untouched. connect() now returns the file path as the handle (mdbtools has no persistent connection). Behavior details: - -b strip drops LONGBINARY/OLE bytes so blobs never corrupt the CSV (documents are extracted separately in migration step 4). - ISO date/datetime output (-D/-T); staging read as text (dtype=str), only empty fields treated as null (keep_default_na=False) so literal "NA"/"NULL" data strings survive. - mdbtools reads deleted/corrupted records mdbtools omits rather than aborting, so the old per-row skip loop is no longer needed. Verified end-to-end: full staging load reproduces the original Windows run (82 tables, 0 errors) with exact row counts (datos2 16000, EFECTIVO 13697, DATGRAL 1172/1070, DATMEX 1520) and recovers all 764 MULT rows (the pyodbc path lost 1 to HY109 corruption). Accented-column tables (PROPANO) read cleanly. requirements.txt: drop pyodbc/pywin32 (Windows-only), keep pandas/pyarrow/ sqlalchemy/pymysql; document the Windows-only DAO catalog as historical. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,12 @@
|
||||
pyodbc>=5.0
|
||||
# Extraction on macOS uses mdbtools (brew install mdbtools) via subprocess,
|
||||
# not pyodbc — no Access ODBC driver exists on macOS. See extract.py.
|
||||
pandas>=2.2
|
||||
pyarrow>=15.0
|
||||
sqlalchemy>=2.0
|
||||
pymysql>=1.1
|
||||
pywin32>=306 # Windows only - needed for catalog_objects.py (DAO COM automation)
|
||||
|
||||
# Windows-only, historical — the DAO/COM object catalog (catalog_objects.py)
|
||||
# was already run on Windows and its output is committed (objects.json). Not
|
||||
# needed on macOS; left documented for provenance.
|
||||
# pyodbc>=5.0
|
||||
# pywin32>=306
|
||||
|
||||
Reference in New Issue
Block a user