Add MinIO object storage + LONGBINARY blob extractor (migration step 4)
deploy/jorgecuadros-minio.stack.yml: S3-compatible object storage (MinIO) for the platform's document blobs, deployed to the cubex Swarm with the same statefulness rules as the DB stack (named volume, pinned to the labeled node). Parametrized for dev/prod as two stacks (dev API 9100/console 9101, prod 9000/9001). Dev deployed + bucket jorgecuadros-documents created. migration/blob_extract.py: re-reads the LONGBINARY columns via mdb-export -b hex (staging used -b strip), carves the embedded file out of the Access OLE wrapper by locating its magic bytes (JPEG/PNG/PDF/GIF/TIFF) and trimming trailing OLE junk, uploads to MinIO, and writes service_documents / policy_documents pointer rows. Row->parent alignment uses mdb-export's deterministic order (== staged _row_num) for policies and numer_id for properties. Idempotent (truncate doc tables + overwrite by deterministic key); --limit/--tables for test passes. Validated on a limited pass: carved blobs are valid JPEGs (ffd8ff..ffd9) correctly linked to their policies. requirements.txt: add boto3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,56 @@
|
||||
# S3-compatible object storage (MinIO) for the platform's document blobs.
|
||||
#
|
||||
# Holds the scanned utility bills / IDs / policy docs extracted from the
|
||||
# Access LONGBINARY columns (migration step 4). MySQL keeps only the pointer
|
||||
# (storageKey) + metadata; the bytes live here.
|
||||
#
|
||||
# Target: Portainer local endpoint on cubex (3-node Swarm). Same statefulness
|
||||
# rules as the MySQL stack (deploy/jorgecuadros-db.stack.yml): named volume +
|
||||
# pinned to one node so the data volume is stable. Reuses the same node label.
|
||||
#
|
||||
# DEV / PROD as two stacks from this one file:
|
||||
# dev : stack jorgecuadros-dev-minio API 9100 / console 9101
|
||||
# prod: stack jorgecuadros-prod-minio API 9000 / console 9001
|
||||
# Swarm namespaces the volume per stack name -> isolated data per environment.
|
||||
#
|
||||
# Secrets (MINIO_ROOT_USER / MINIO_ROOT_PASSWORD) injected via Portainer stack
|
||||
# env at deploy time, not committed.
|
||||
|
||||
version: "3.8"
|
||||
|
||||
services:
|
||||
minio:
|
||||
image: minio/minio:RELEASE.2024-10-13T13-34-11Z
|
||||
command: server /data --console-address ":9001"
|
||||
environment:
|
||||
MINIO_ROOT_USER: ${MINIO_ROOT_USER:?MINIO_ROOT_USER must be set}
|
||||
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:?MINIO_ROOT_PASSWORD must be set}
|
||||
ports:
|
||||
- target: 9000
|
||||
published: ${MINIO_API_PORT:-9000}
|
||||
protocol: tcp
|
||||
mode: ingress
|
||||
- target: 9001
|
||||
published: ${MINIO_CONSOLE_PORT:-9001}
|
||||
protocol: tcp
|
||||
mode: ingress
|
||||
volumes:
|
||||
- minio_data:/data
|
||||
deploy:
|
||||
replicas: 1
|
||||
placement:
|
||||
constraints:
|
||||
- node.labels.jorgecuadros_db == true
|
||||
restart_policy:
|
||||
condition: any
|
||||
update_config:
|
||||
order: stop-first
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "mc ready local || curl -f http://localhost:9000/minio/health/live || exit 1"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 12
|
||||
start_period: 20s
|
||||
|
||||
volumes:
|
||||
minio_data:
|
||||
Reference in New Issue
Block a user