Add MinIO object storage + LONGBINARY blob extractor (migration step 4)
deploy/jorgecuadros-minio.stack.yml: S3-compatible object storage (MinIO) for the platform's document blobs, deployed to the cubex Swarm with the same statefulness rules as the DB stack (named volume, pinned to the labeled node). Parametrized for dev/prod as two stacks (dev API 9100/console 9101, prod 9000/9001). Dev deployed + bucket jorgecuadros-documents created. migration/blob_extract.py: re-reads the LONGBINARY columns via mdb-export -b hex (staging used -b strip), carves the embedded file out of the Access OLE wrapper by locating its magic bytes (JPEG/PNG/PDF/GIF/TIFF) and trimming trailing OLE junk, uploads to MinIO, and writes service_documents / policy_documents pointer rows. Row->parent alignment uses mdb-export's deterministic order (== staged _row_num) for policies and numer_id for properties. Idempotent (truncate doc tables + overwrite by deterministic key); --limit/--tables for test passes. Validated on a limited pass: carved blobs are valid JPEGs (ffd8ff..ffd9) correctly linked to their policies. requirements.txt: add boto3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -4,6 +4,7 @@ pandas>=2.2
|
||||
pyarrow>=15.0
|
||||
sqlalchemy>=2.0
|
||||
pymysql>=1.1
|
||||
boto3>=1.34 # S3/MinIO client for blob_extract.py (migration step 4)
|
||||
|
||||
# Windows-only, historical — the DAO/COM object catalog (catalog_objects.py)
|
||||
# was already run on Windows and its output is committed (objects.json). Not
|
||||
|
||||
Reference in New Issue
Block a user