fix(deploy): dump from a dedicated container as root, and prove the dump is real

The pre-migrate backup ran INSIDE the API container, which made it depend on
that image's toolchain — and deadlocked: the running image shipped a MySQL
client that could not authenticate, so the backup failed, which blocked the
very deploy that would have replaced the broken image. A backup must not depend
on the thing being deployed.

The dump now runs in a throwaway container built from mysql:8.4 with the API's
backup volume mounted. The volume name is discovered from the API container's
mounts, so the file still lands where the Operaciones restore screen looks. As
a container rather than an exec, its logs can simply be read — no more failures
reported as a bare exit code. The image is pulled if the host lacks it, since a
scope:app deploy never touches the db stack.

Three further defects found while verifying, none of which would have surfaced
without dumping against the real database:

- The dump now runs as root. mysqldump --single-transaction issues FLUSH
  TABLES, needing the global RELOAD privilege; the MySQL image grants the
  application user only ALL ON `<db>`.*, and --skip-lock-tables does not avoid
  it. Elevating the app's own runtime user would have been the worse trade.

- --set-gtid-purged=OFF. galactus is the replication SOURCE with GTID on, so a
  default dump embeds SET @@GLOBAL.GTID_PURGED and is unrestorable onto the
  server it came from. Verified: 0 GTID_PURGED lines in the output.

- Verification was too weak to be worth having. `test -s` plus `gzip -t` passes
  on a 372-byte gzip containing no tables, which is exactly what a dump that
  died on its first statement produces. It now asserts a CREATE TABLE count and
  logs it. A failed attempt also deletes its own output, so a truncated file
  never appears in the restore list.

Verified against live prod, both paths: success writes a 31-table dump the API
container can see; a wrong password fails with mysqldump's own error quoted and
leaves the volume empty.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-30 16:03:40 -07:00
co-authored by Claude Opus 5
parent 19f03198d6
commit d5ebb86cae
3 changed files with 188 additions and 120 deletions
+6 -2
View File
@@ -126,10 +126,10 @@ jobs:
REQUIRED="PORTAINER_URL PORTAINER_API_KEY PORTAINER_ENDPOINT_ID
PORTAINER_APP_STACK_NAME DATABASE_URL SESSION_SECRET
APP_API_ORIGIN APP_WEB_ORIGIN APP_S3_ENDPOINT
MINIO_ROOT_USER MINIO_ROOT_PASSWORD"
MINIO_ROOT_USER MINIO_ROOT_PASSWORD MYSQL_ROOT_PASSWORD"
if [ "$SCOPE" = "full" ]; then
REQUIRED="$REQUIRED PORTAINER_DB_STACK_NAME PORTAINER_MINIO_STACK_NAME
MYSQL_PASSWORD MYSQL_ROOT_PASSWORD"
MYSQL_PASSWORD"
fi
missing=""
for name in $REQUIRED; do
@@ -193,6 +193,10 @@ jobs:
PORTAINER_API_KEY: ${{ secrets.PORTAINER_API_KEY }}
PORTAINER_ENDPOINT_ID: ${{ secrets.PORTAINER_ENDPOINT_ID }}
DATABASE_URL: ${{ secrets.DATABASE_URL }}
# The dump runs as root: --single-transaction issues FLUSH TABLES,
# which needs the global RELOAD privilege the application user
# deliberately does not have.
MYSQL_ROOT_PASSWORD: ${{ secrets.MYSQL_ROOT_PASSWORD }}
BACKUP_TAG: ${{ github.event.inputs.tag }}
ALLOW_MISSING_CONTAINER: ${{ github.event.inputs.bootstrap }}
# Portainer serves a self-signed certificate. Scoped to this step