fix(ops): run backups as an admin login, and stop recording failed dumps as good

The Operaciones panel (backup, restore, sync, re-import) shelled out to
mysqldump as the application user, parsed straight out of DATABASE_URL.
`--single-transaction` issues FLUSH TABLES, which needs the global RELOAD
privilege, and the app user is granted only ALL ON jorgecuadros.* plus
USAGE ON *.*. BACKUP failed outright; SYNC and REIMPORT failed with it,
since both take a safety backup first.

An admin credential is now supplied out of band via OPS_DB_ADMIN_USER /
OPS_DB_ADMIN_PASSWORD, mirroring what deploy/scripts/pre-migrate-backup.mjs
already does, rather than permanently elevating the user the API serves
requests as. Host, port and database still come from DATABASE_URL, so the
override can only change who logs in, never which server. Unset, it falls
back to the DATABASE_URL credentials and warns — local development is
unaffected.

Two defects in the dumps themselves, both shared with the deploy backup
before it was rewritten:

- No --set-gtid-purged=OFF. The production server is the replication source
  with GTID on, so every dump embedded SET @@GLOBAL.GTID_PURGED and was
  unrestorable onto the server it came from — the one thing the restore
  screen is for.

- The pipeline's exit status was gzip's, and gzip succeeded. A mysqldump
  that died on its first statement left a small, perfectly valid archive
  that the job recorded as SUCCESS and the restore screen listed as an
  ordinary restore point. Dumps now run under `set -o pipefail`, assert a
  CREATE TABLE count, and delete their own output on failure. Verified with
  a stubbed mysqldump: a failing dump exits 1, surfaces the real error,
  removes the partial file, and — critically — stops SYNC/REIMPORT before
  the ETL touches anything.

Restores gained pipefail too: a corrupt archive made gunzip fail while
mysql, fed a truncated stream, could still exit 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-30 16:21:00 -07:00
co-authored by Claude Opus 5
parent d5ebb86cae
commit 30dfc7dc3e
7 changed files with 154 additions and 7 deletions
+37
View File
@@ -253,6 +253,43 @@ had only ever been exercised with the API running on a developer machine, where
the Oracle client is installed, which is why this went unnoticed until the
first containerised deploy.
## The Operaciones panel needs its own database login
The panel's four jobs all shell out to `mysqldump`/`mysql`, and they cannot do
so as the application user. `mysqldump --single-transaction` issues
`FLUSH TABLES`, which requires the **global** `RELOAD` privilege; the MySQL
image grants the app user only `ALL PRIVILEGES ON jorgecuadros.*` plus
`USAGE ON *.*`. `--skip-lock-tables` does not avoid it. BACKUP therefore failed
outright, and SYNC and RE-IMPORT with it, because both take a safety backup
first.
The API is given an admin login out of band rather than permanently elevating
the user it serves requests as:
```
OPS_DB_ADMIN_USER=root
OPS_DB_ADMIN_PASSWORD=<MYSQL_ROOT_PASSWORD>
```
Both deploy workflows pass these into the app stack from the existing
`MYSQL_ROOT_PASSWORD` secret. Host, port and database still come from
`DATABASE_URL` — the override changes *who logs in*, never *which server*. With
the pair unset the service falls back to the `DATABASE_URL` credentials and logs
a warning, which is what local development wants.
Two more things the panel's dumps now do, for the same reasons the pre-migrate
backup does them (see `deploy/scripts/pre-migrate-backup.mjs`):
- **`--set-gtid-purged=OFF`.** galactus is the replication *source* with GTID
on, so without this every dump embeds `SET @@GLOBAL.GTID_PURGED` and cannot be
restored onto the server it came from — which is precisely what the restore
screen exists to do.
- **`set -o pipefail` and a `CREATE TABLE` count.** `mysqldump | gzip` reports
gzip's exit status, and a `mysqldump` that dies on its first statement still
produces a ~372-byte perfectly valid archive that passes `gzip -t`. Without
both checks a failed backup was recorded as a successful one and listed as an
ordinary restore point. A dump that fails now deletes its own output.
## Known caveats in the deploy path
- The pre-migrate backup step sets `NODE_TLS_REJECT_UNAUTHORIZED=0` because