rmancinasandClaude Opus 4.8
dc9866c24a
macOS has no Access ODBC driver, so the pyodbc-based extract.py could not run. Rewrite it to shell out to mdbtools (mdb-tables/mdb-export) while keeping the public interface (connect/list_tables/read_table/ sanitize_column_name) unchanged, so load_staging.py and config.py are untouched. connect() now returns the file path as the handle (mdbtools has no persistent connection). Behavior details: - -b strip drops LONGBINARY/OLE bytes so blobs never corrupt the CSV (documents are extracted separately in migration step 4). - ISO date/datetime output (-D/-T); staging read as text (dtype=str), only empty fields treated as null (keep_default_na=False) so literal "NA"/"NULL" data strings survive. - mdbtools reads deleted/corrupted records mdbtools omits rather than aborting, so the old per-row skip loop is no longer needed. Verified end-to-end: full staging load reproduces the original Windows run (82 tables, 0 errors) with exact row counts (datos2 16000, EFECTIVO 13697, DATGRAL 1172/1070, DATMEX 1520) and recovers all 764 MULT rows (the pyodbc path lost 1 to HY109 corruption). Accented-column tables (PROPANO) read cleanly. requirements.txt: drop pyodbc/pywin32 (Windows-only), keep pandas/pyarrow/ sqlalchemy/pymysql; document the Windows-only DAO catalog as historical. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
13 lines
444 B
Plaintext
13 lines
444 B
Plaintext
# Extraction on macOS uses mdbtools (brew install mdbtools) via subprocess,
|
|
# not pyodbc — no Access ODBC driver exists on macOS. See extract.py.
|
|
pandas>=2.2
|
|
pyarrow>=15.0
|
|
sqlalchemy>=2.0
|
|
pymysql>=1.1
|
|
|
|
# Windows-only, historical — the DAO/COM object catalog (catalog_objects.py)
|
|
# was already run on Windows and its output is committed (objects.json). Not
|
|
# needed on macOS; left documented for provenance.
|
|
# pyodbc>=5.0
|
|
# pywin32>=306
|