jorgecuadros-platform/docs/DEPLOY_AND_MIGRATIONS.md

Releasing, deploying, and changing the schema

How a version gets from this repo onto a server, and the one rule that keeps rollbacks possible.

The short version

Dispatch Cut release from the Actions tab and pick patch, minor or major (or explicit plus a number). It stamps every package.json, commits chore(release): vX.Y.Z, tags, and pushes both refs in one go. It refuses a version that already exists as a tag, and refuses a no-op bump.

The equivalent by hand, if you would rather cut it locally:

pnpm version:set 1.2.0                       # stamp every package.json
git commit -am "chore(release): v1.2.0"
git tag v1.2.0 && git push origin master v1.2.0

Either way that push triggers .gitea/workflows/build.yml, which builds both images in one matrix run and publishes:

tag pushed	image tags produced
v1.2.0	1.2.0, 1.2, sha-<short>
push to master	master, sha-<short>, latest

Then dispatch a deploy from the Actions tab:

• galactus (office server, standalone Docker) — Deploy to galactus
• cubex (3-node Swarm) — Deploy to Portainer

The v is not part of the image tag. docker/metadata-action's {{version}} strips it. Git tag v1.2.0, dispatch 1.2.0. Dispatching v1.2.0 deploys nothing that exists.

Cut release needs a RELEASE_TOKEN secret — a Gitea PAT with write:repository. It does not use the built-in Actions token on purpose: whether a push made with that token re-triggers build.yml depends on the Gitea version, and a release that quietly publishes no images is worse than one that fails outright. If the build somehow does not start, build.yml has workflow_dispatch — run it against the new tag by hand.

Because api and web are built from one matrix run, they cannot drift at build time. They can drift at deploy time if a stack is applied with only one image moved — the web footer shows both versions and flags a mismatch, and the deploy workflow's last step fails if the API does not report the tag you dispatched.

What a deploy actually does

1. db + minio — scope: full only. Idempotent; data lives on named volumes.
2. Pre-migrate backup — deploy/scripts/pre-migrate-backup.mjs runs mysqldump inside the still-running old API container, via Portainer's Docker API. The file lands in that container's BACKUP_DIR volume as pre-migrate-<tag>-<timestamp>.sql.gz, which is exactly what the Operaciones admin screen lists and can restore. A dump taken on the CI runner would be unreachable by the only restore path the platform has.
3. prisma migrate deploy — as a workflow step, never the container CMD. If it were the CMD, N replicas would race each other applying the same migration.
4. app — the new api + web images.
5. Verify — GET /version on the running API must report the dispatched tag.

Rollback is tag: 1.1.9 re-dispatched. That rolls back code only. The schema stays where it is. Which brings us to the rule.

The rule: expand / contract

Prisma has no down-migrations. There is no prisma migrate down, and there never will be. So a schema change that the previous release cannot tolerate turns a 30-second rollback into a restore-from-backup outage.

Every schema change must leave the previous release working. Split anything destructive across two releases:

	Release N (expand)	Release N+1 (contract)
Rename a column	add the new column, write to both, read the old	drop the old column
Drop a column	stop reading and writing it in code	drop it
Add a required column	add it nullable (or with a default), backfill	make it NOT NULL
Split a table	create the new table, dual-write	stop writing the old, drop it
Add an enum value	add the value; old code must not choke on unknowns	start emitting it

Ship N, let it soak, then ship N+1. If N has to be rolled back you just re-dispatch the old tag — the expanded schema still satisfies it.

Restoring from the pre-migrate dump is the emergency lever, not the routine path, and on galactus it is worse than it sounds: galactus is the replication master, DDL replicates through the binlog, and restoring the master from a dump diverges every replica. GTIDs will not line up and each replica needs a full re-seed. Assume a restore is a multi-hour, whole-topology event.

Migration history

packages/database/prisma/migrations/0000_init/ is a baseline. It is the full schema as it stood on 2026-07-30, generated with:

prisma migrate diff --from-empty \
  --to-schema-datamodel packages/database/prisma/schema.prisma --script

Until then the schema had only ever been applied with prisma db push, so no history existed and the schema state was disconnected from the app version.

One-time, on every database that already exists

0000_init describes tables those databases already have, so migrate deploy would fail with P3005 "the database schema is not empty". Mark it applied instead of applying it — this writes a _prisma_migrations row and changes no data:

DATABASE_URL=<the database> npx prisma@5 migrate resolve \
  --applied 0000_init --schema packages/database/prisma/schema.prisma

Do this once per database (prod, dev, any local copy). Verify first that the live schema really does match the baseline — this should print an empty migration:

prisma migrate diff --from-url "$DATABASE_URL" \
  --to-schema-datamodel packages/database/prisma/schema.prisma --script

If it prints actual statements, the live database has drifted from schema.prisma. Reconcile before baselining, or the first real migration will fail against a schema Prisma believes it already knows.

From here on

# edit schema.prisma, then:
pnpm --filter @jorgecuadros/database exec prisma migrate dev --name add_foo

Commit the generated migrations/<timestamp>_add_foo/ directory. db push is now a local-scratch tool only — using it against a database with history desynchronises it from _prisma_migrations.

galactus vs cubex

galactus is standalone Docker (Portainer endpoint 3), cubex is a 3-node Swarm (endpoint 2). They need different compose files because plain compose silently ignores Swarm's deploy: keys rather than erroring:

	Swarm (deploy/*.stack.yml)	standalone (deploy/galactus/*.compose.yml)
restart	deploy.restart_policy	restart: unless-stopped — without this nothing comes back after a host reboot
placement	node.labels.jorgecuadros_db == true	dropped, one host
ports	{mode: ingress} long syntax	"3306:3306"
depends_on	ignored by Swarm	honoured, with condition: service_healthy
volumes	named	named (unchanged — the pinning hazard was a Swarm problem)

Keep the two sets in sync when either changes.

On both hosts, cross-stack traffic goes over the host address, not compose service DNS: db, minio and app are three separate stacks, so three separate networks. DATABASE_URL and S3_ENDPOINT name the host and its published port. Do not "simplify" them to mysql:3306.

galactus is addressed by MagicDNS, and containers need help resolving it

galactus is Tailscale-only once it is installed in the office, so every URL names galactus.tail01aa2.ts.net. Its LAN IP is a DHCP lease and has already drifted once — never put a 192.168.4.x address in a secret.

Containers on galactus cannot resolve that name by default. The host runs systemd-resolved, whose 127.0.0.53 stub is unreachable from inside a container, so Docker falls back to the upstream resolver listed in /run/systemd/resolve/resolv.conf — the LAN router, which knows nothing about the tailnet. Routing to 100.x works fine; only the lookup fails, and the symptom is Prisma P1001 "can't reach database server" on a container that otherwise started cleanly.

deploy/galactus/jorgecuadros-app.compose.yml therefore pins the resolver:

dns:        [100.100.100.100]        # Tailscale's fixed anycast MagicDNS address
dns_search: [tail01aa2.ts.net]       # this tailnet's suffix

Both are overridable (TAILSCALE_DNS, TAILNET_SUFFIX) if the tailnet changes. Browser-facing origins need none of this — those names resolve on the client.

Replication

galactus's MySQL is the master; every other MySQL in the estate is a replica. Consequences that bite:

• server-id must be unique across the whole topology (prod 1, cubex dev 11). A duplicate breaks replication silently.
• GTID is on from first boot, so replicas attach with SOURCE_AUTO_POSITION=1.
• binlog_expire_logs_seconds is raised to 60 days in the galactus compose file (MYSQL_BINLOG_EXPIRE_SECONDS). MySQL 8.4 defaults to 30 days; a replica offline longer than the retention needs a full re-seed.

Still open, and not handled by anything in this repo:

• No replication user with REPLICATION SLAVE granted exists yet.
• Nothing sets read_only / super_read_only on the replicas, so a stray write to a replica will diverge it.
• The channel to the VPS crosses the public internet. It needs a tunnel or TLS — do not publish raw 3306.

Seeding the first sign-in account

A freshly migrated database has a schema and no users, so nobody can log in. prisma migrate deploy creates tables, never rows; nothing in the deploy path seeds an account, by design — creating an administrator should be a deliberate act, not a side effect of shipping code.

apps/api/scripts/seed-user.mjs ships inside the API image. On the target host:

docker exec -e SEED_PASSWORD='<a strong password>' \
  <api-container> node apps/api/scripts/seed-user.mjs

Defaults are admin@jorgecuadros.local / ChangeMe!2026 / role ADMIN, overridable with SEED_EMAIL, SEED_PASSWORD, SEED_NAME. Do not accept the default password on anything but a dev database — it is published in this repo's README. The script upserts by email, so re-running is safe, but it also resets the password of an existing account.

The session cookie and TLS

SESSION_COOKIE_SECURE controls the Secure flag on the session cookie. It defaults to on in production, and it must be explicitly "false" for a deployment served over plain HTTP.

This is not cosmetic. express-session silently declines to emit a Secure cookie over an unencrypted connection: no Set-Cookie header is sent at all, POST /auth/login still answers 200 with the user object, no session is established, every subsequent request gets 403, and the UI bounces back to /login in a loop. It looks like an auth bug and is really a transport mismatch.

galactus runs with SESSION_COOKIE_SECURE=false, which is acceptable only because it is reachable exclusively over Tailscale — WireGuard already encrypts the wire, so the cookie never crosses an untrusted network. Turn it back on the moment the app is served over TLS or reachable off-tailnet. Behind a TLS-terminating reverse proxy, set trust proxy on the Nest app instead of disabling the flag.

The MySQL client inside the API image

Alpine's mysql-client package is MariaDB's client, and it installs an empty /usr/lib/mariadb/plugin. It therefore cannot speak caching_sha2_password, which is MySQL 8.4's default and effectively only auth method, and every mysqldump/mysql call from the container fails with:

ERROR 1045: Plugin caching_sha2_password could not be loaded:
  ... /usr/lib/mariadb/plugin/caching_sha2_password.so: No such file or directory

mariadb-connector-c supplies that plugin and is installed in docker/api.Dockerfile for exactly this reason — do not drop it as an unused dependency. It affects far more than the deploy backup: the entire Operaciones panel (backup, restore, sync, re-import) shells out to these binaries, so without it none of those work in a container either. The feature had only ever been exercised with the API running on a developer machine, where the Oracle client is installed, which is why this went unnoticed until the first containerised deploy.

The Operaciones panel needs its own database login

The panel's four jobs all shell out to mysqldump/mysql, and they cannot do so as the application user. mysqldump --single-transaction issues FLUSH TABLES, which requires the global RELOAD privilege; the MySQL image grants the app user only ALL PRIVILEGES ON jorgecuadros.* plus USAGE ON *.*. --skip-lock-tables does not avoid it. BACKUP therefore failed outright, and SYNC and RE-IMPORT with it, because both take a safety backup first.

The API is given an admin login out of band rather than permanently elevating the user it serves requests as:

OPS_DB_ADMIN_USER=root
OPS_DB_ADMIN_PASSWORD=<MYSQL_ROOT_PASSWORD>

Both deploy workflows pass these into the app stack from the existing MYSQL_ROOT_PASSWORD secret. Host, port and database still come from DATABASE_URL — the override changes who logs in, never which server. With the pair unset the service falls back to the DATABASE_URL credentials and logs a warning, which is what local development wants.

Two more things the panel's dumps now do, for the same reasons the pre-migrate backup does them (see deploy/scripts/pre-migrate-backup.mjs):

• --set-gtid-purged=OFF, but only when the dumper supports it. galactus is the replication source with GTID on, so on a MySQL client this flag is what keeps every dump from embedding SET @@GLOBAL.GTID_PURGED and becoming unrestorable onto the server it came from — which is precisely what the restore screen exists to do. The panel, however, dumps from inside the API container, where Alpine's mysql-client is MariaDB's: there mysqldump is a shim over mariadb-dump, the flag does not exist, and passing it failed every backup with mysqldump: unknown variable 'set-gtid-purged=OFF'. So the panel probes mysqldump --help and passes the flag only if it is advertised, invoking mariadb-dump directly otherwise (MariaDB writes no GTID state unless asked with --gtid, so there is nothing to suppress). The pre-migrate backup keeps the flag unconditionally — it runs in a real mysql:8.4 image, not in the API container.
• set -o pipefail and a CREATE TABLE count. mysqldump | gzip reports gzip's exit status, and a mysqldump that dies on its first statement still produces a ~372-byte perfectly valid archive that passes gzip -t. Without both checks a failed backup was recorded as a successful one and listed as an ordinary restore point. A dump that fails now deletes its own output.

Known caveats in the deploy path

• The pre-migrate backup step sets NODE_TLS_REJECT_UNAUTHORIZED=0 because Portainer serves a self-signed certificate. It is scoped to that one step, which talks to nothing but Portainer. Replacing the certificate and dropping the flag is the real fix.
• The runner lives on cubex and must reach the target host's Portainer (9443) and MySQL (3306). If it cannot reach 3306, run the migration by hand from a host that can and dispatch with skip_migrate: true.
• bootstrap: true lets the pre-migrate backup be skipped when no API container exists yet. Use it for a first-ever deploy only — it is the one switch that lets a migration run with no restore point.
• The API container sends mail on a timer. Two sweeps run inside it (renewal avisos, on by default at 06:00 America/Tijuana; the servicios run-all, off by default) — see MASS_EMAIL_NOTIFICATIONS.md. Two consequences for deploys: the cadence lives in app_settings, so it survives a redeploy and is not restored by rolling back an image, and running more than one API replica would double-fire the servicios sweep, which has no DB lock (the pólizas one does). Keep it single-replica.
• SES_* is optional to deploy — the preflight only warns — but the production image sets NODE_ENV=production, which disables the stdout dev fallback. A blank SES config therefore makes every send fail and log FAILED. As of 2026-08-02 these secrets are still unset in Gitea.