feat(deploy): prisma migration history, /version, galactus standalone deploy
Build and Push Images / Build jorgecuadros-web (push) Successful in 1m49s
Build and Push Images / Build jorgecuadros-api (push) Successful in 2m2s

Closes the gap between "what tag did I deploy" and "what is actually running",
and gives the schema a history that can be reasoned about across releases.

Migrations
- Baseline the existing schema as 0000_init (migrate diff --from-empty). The
  schema had only ever been applied with `prisma db push`, so no history
  existed and schema state was disconnected from app version. Existing
  databases must be baselined once with `migrate resolve --applied 0000_init`;
  the workflows print this remedy on P3005.
- Run `prisma migrate deploy` as a deploy STEP, not the container CMD — as a
  CMD, N replicas would race each other applying the same migration.

Version reporting
- GET /version on the API reports the APP_VERSION / GIT_SHA / BUILD_DATE that
  build.yml already baked into both images but nothing ever read.
- The web footer shows the web build and flags an api/web mismatch. The two
  cannot drift at build time (one matrix run) but can at deploy time.
- Both deploy workflows now fail if the running API does not report the tag
  that was dispatched — a stack naming a tag is not proof of what is running.
- scripts/set-version.mjs stamps every package.json, which had all sat at
  0.1.0 while real releases shipped as v1.x.

Pre-migrate backup
- deploy/scripts/pre-migrate-backup.mjs dumps the database from INSIDE the
  still-running old API container over Portainer's Docker API, so the file
  lands in the volume the Operaciones restore screen reads. A dump taken on
  the CI runner would be unreachable by the only restore path we have.
  Verifies the artefact with `gzip -t` before letting the migration proceed.

galactus
- deploy/galactus/*.compose.yml: standalone-Docker ports of the Swarm stacks.
  Plain compose silently ignores `deploy:`, so restart_policy becomes
  `restart: unless-stopped` — without it nothing returns after a host reboot.
- .gitea/workflows/deploy-galactus.yml drives endpoint 3 with its own secrets.

Fixes
- deploy.yml passed `endpoint_id` and `pull_image` to
  cssnr/portainer-stack-deploy-action, which has no such inputs (they are
  `endpoint` and `pull`). The endpoint was silently never set.

docs/DEPLOY_AND_MIGRATIONS.md documents expand/contract as the rule for schema
changes: Prisma has no down-migrations, so a code rollback never rolls the
schema back, and restoring the replication master from a dump diverges every
replica.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-30 11:41:12 -07:00
co-authored by Claude Opus 5
parent 9ba5d2d09a
commit 4ee7ec71f0
18 changed files with 1677 additions and 8 deletions
@@ -0,0 +1,81 @@
# NestJS API + Next.js web on galactus (standalone Docker, Portainer endpoint 3).
#
# Standalone port of deploy/jorgecuadros-app.stack.yml — see the header of
# deploy/galactus/jorgecuadros-db.compose.yml for the Swarm keys plain compose
# silently ignores. The one that matters most here: without
# `restart: unless-stopped` neither service returns after a host reboot.
#
# Cross-stack traffic still goes over the HOST, not service DNS. db and minio
# are separate Portainer stacks, so they are on separate compose networks and
# their service names do not resolve from here. DATABASE_URL / S3_ENDPOINT must
# name galactus's own address and the published port — exactly as on cubex
# today. Do not "simplify" them to `mysql:3306`.
#
# The web image is NOT URL-baked: the browser's API origin is injected at
# runtime from API_ORIGIN (apps/web/src/app/layout.tsx), so the same image works
# for any deployment. APP_VERSION / GIT_SHA / BUILD_DATE come baked in from
# build.yml and are surfaced at GET /version (api) and in the web footer.
#
# Keep in sync with deploy/jorgecuadros-app.stack.yml when either changes.
services:
api:
image: git.mancinas.io/rmancinas/jorgecuadros-api:${APP_TAG:-latest}
restart: unless-stopped
# Stable handle for deploy/scripts/pre-migrate-backup.sh, which finds this
# container by label to run mysqldump into the backup volume. A label
# survives stack renames; the compose service name does not.
labels:
io.jorgecuadros.role: "api"
environment:
DATABASE_URL: ${DATABASE_URL:?DATABASE_URL must be set}
SESSION_SECRET: ${SESSION_SECRET:?SESSION_SECRET must be set}
WEB_ORIGIN: ${WEB_ORIGIN:?WEB_ORIGIN must be set}
PORT: "3001"
INGEST_DIR: /data/ingest
BACKUP_DIR: /data/backups
MIGRATION_ENV: prod
S3_ENDPOINT: ${S3_ENDPOINT:?S3_ENDPOINT must be set}
S3_BUCKET: ${S3_BUCKET:-jorgecuadros-documents}
MINIO_ROOT_USER: ${MINIO_ROOT_USER:?MINIO_ROOT_USER must be set}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:?MINIO_ROOT_PASSWORD must be set}
ports:
- "${API_PORT:-3001}:3001"
volumes:
# Uploaded Access files and DB backups. Named, so they survive every
# redeploy — and so the pre-migrate dump the deploy takes is the same
# file the "Operaciones" restore screen lists.
- ingest_data:/data/ingest
- backup_data:/data/backups
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:3001/health || exit 1"]
interval: 15s
timeout: 5s
retries: 10
start_period: 30s
web:
image: git.mancinas.io/rmancinas/jorgecuadros-web:${APP_TAG:-latest}
restart: unless-stopped
labels:
io.jorgecuadros.role: "web"
environment:
# Public API URL the browser calls (injected at runtime, see layout.tsx).
API_ORIGIN: ${API_ORIGIN:?API_ORIGIN must be set}
ports:
- "${WEB_PORT:-3000}:3000"
depends_on:
# Unlike Swarm — which ignores depends_on entirely — plain compose honours
# this, so web waits for the API to pass its healthcheck.
api:
condition: service_healthy
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:3000/ >/dev/null 2>&1 || exit 1"]
interval: 15s
timeout: 5s
retries: 10
start_period: 30s
volumes:
ingest_data:
backup_data:
@@ -0,0 +1,64 @@
# MySQL for the Jorge Cuadros platform on galactus — the PROD source of truth.
#
# galactus is STANDALONE Docker (Portainer endpoint 3, `swarm: inactive`), not
# the 3-node Swarm on cubex. deploy/jorgecuadros-db.stack.yml is the Swarm
# version of this file; the deltas are called out below because plain compose
# SILENTLY IGNORES the Swarm keys rather than erroring on them:
#
# 1. `deploy.restart_policy` is ignored -> `restart: unless-stopped` instead.
# Without this MySQL does not come back after a host reboot. This is the
# single highest-risk difference.
# 2. `deploy.placement.constraints` is meaningless on one host — dropped,
# along with its `docker node update --label-add jorgecuadros_db=true`
# prerequisite.
# 3. `deploy.replicas` / `update_config` are ignored — dropped.
# 4. `ports: {mode: ingress}` long syntax is Swarm-only -> short syntax.
# 5. Named volumes stay exactly as they were: the node-pinning hazard that
# motivated them was purely a Swarm problem, and Portainer still namespaces
# the volume by stack name.
#
# This node is the REPLICATION MASTER for the whole topology. Every other MySQL
# is a replica of it. server-id must be unique across the topology (prod=1,
# cubex dev=11); a duplicate silently breaks replication. binlog + GTID are on
# from first boot so a replica can attach with SOURCE_AUTO_POSITION=1 and no
# file/position bookkeeping.
#
# Keep in sync with deploy/jorgecuadros-db.stack.yml when either changes.
services:
mysql:
image: mysql:8.4
restart: unless-stopped
command:
# (caching_sha2_password is already the default in 8.4; the old
# --default-authentication-plugin flag was REMOVED in 8.4 and aborts boot.)
- --server-id=${MYSQL_SERVER_ID:-1}
- --log-bin=mysql-bin
- --binlog-format=ROW
- --gtid-mode=ON
- --enforce-gtid-consistency=ON
# A replica offline longer than this needs a full re-seed, because the
# binlogs it still needs are gone. The 8.4 default is 30 days; raise it
# here rather than discovering the gap during an outage.
- --binlog-expire-logs-seconds=${MYSQL_BINLOG_EXPIRE_SECONDS:-5184000}
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE:-jorgecuadros}
MYSQL_USER: ${MYSQL_USER:-jorgecuadros}
MYSQL_PASSWORD: ${MYSQL_PASSWORD:?MYSQL_PASSWORD must be set}
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?MYSQL_ROOT_PASSWORD must be set}
ports:
# Standalone: binds directly on the host. Reachable at
# <galactus>:${MYSQL_PORT}. Replicas connect here — see
# docs/DEPLOY_AND_MIGRATIONS.md on NOT exposing raw 3306 to the internet.
- "${MYSQL_PORT:-3306}:3306"
volumes:
- mysql_data:/var/lib/mysql
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-p$$MYSQL_ROOT_PASSWORD"]
interval: 10s
timeout: 5s
retries: 12
start_period: 40s
volumes:
mysql_data:
@@ -0,0 +1,32 @@
# MinIO object storage on galactus (standalone Docker, Portainer endpoint 3).
#
# Holds the document blobs extracted from the Access LONGBINARY columns; MySQL
# keeps only the storageKey pointer. Standalone port of
# deploy/jorgecuadros-minio.stack.yml — see the header of
# deploy/galactus/jorgecuadros-db.compose.yml for the full list of Swarm keys
# that plain compose silently ignores.
#
# Keep in sync with deploy/jorgecuadros-minio.stack.yml when either changes.
services:
minio:
image: minio/minio:RELEASE.2024-10-13T13-34-11Z
restart: unless-stopped
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER:?MINIO_ROOT_USER must be set}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:?MINIO_ROOT_PASSWORD must be set}
ports:
- "${MINIO_API_PORT:-9000}:9000"
- "${MINIO_CONSOLE_PORT:-9001}:9001"
volumes:
- minio_data:/data
healthcheck:
test: ["CMD-SHELL", "mc ready local || curl -f http://localhost:9000/minio/health/live || exit 1"]
interval: 10s
timeout: 5s
retries: 12
start_period: 20s
volumes:
minio_data:
+7
View File
@@ -27,6 +27,11 @@ version: "3.8"
services:
api:
image: git.mancinas.io/rmancinas/jorgecuadros-api:${APP_TAG:-latest}
# Container label (not `deploy.labels`, which labels the swarm SERVICE).
# deploy/scripts/pre-migrate-backup.mjs finds the container by this label to
# run its pre-migrate mysqldump into the backup volume.
labels:
io.jorgecuadros.role: "api"
environment:
DATABASE_URL: ${DATABASE_URL:?DATABASE_URL must be set}
SESSION_SECRET: ${SESSION_SECRET:?SESSION_SECRET must be set}
@@ -68,6 +73,8 @@ services:
web:
image: git.mancinas.io/rmancinas/jorgecuadros-web:${APP_TAG:-latest}
labels:
io.jorgecuadros.role: "web"
environment:
# Public API URL the browser calls (injected at runtime, see layout.tsx).
API_ORIGIN: ${API_ORIGIN:?API_ORIGIN must be set}
+206
View File
@@ -0,0 +1,206 @@
#!/usr/bin/env node
/**
* Take a mysqldump immediately before a deploy runs `prisma migrate deploy`.
*
* Why it is done THIS way and not with a plain `mysqldump` on the CI runner:
* a dump is only useful if a human can restore it, and the only restore path
* this platform has is the "Operaciones" admin screen, which lists and replays
* whatever `*.sql.gz` files sit in the API container's BACKUP_DIR volume
* (apps/api/src/ops/ops.service.ts — listBackups / RESTORE). A dump written on
* the runner would land nowhere anybody can reach. So we drive the dump INSIDE
* the still-running old API container, via Portainer's Docker API proxy: the
* container already has mysql-client baked in (docker/api.Dockerfile), and the
* file lands in the exact directory the restore UI reads.
*
* It must therefore run BEFORE the app stack is re-applied, while the previous
* container is still up.
*
* Required env:
* PORTAINER_URL https://<host>:9443
* PORTAINER_API_KEY Portainer access token
* PORTAINER_ENDPOINT_ID numeric endpoint id (galactus = 3)
* DATABASE_URL mysql://user:pass@host:port/db
* BACKUP_TAG label for the filename, e.g. the deployed tag
* Optional env:
* ALLOW_MISSING_CONTAINER=true exit 0 when no API container exists yet
* (first-ever deploy — nothing to back up)
* API_CONTAINER_LABEL=io.jorgecuadros.role=api
* EXEC_TIMEOUT_SECONDS=1800
*
* TLS: Portainer here uses a self-signed certificate. The caller is expected to
* set NODE_TLS_REJECT_UNAUTHORIZED=0 for this step; see the workflow. That
* disables verification for the whole process, so nothing else should run in it.
*/
const PORTAINER_URL = required("PORTAINER_URL").replace(/\/+$/, "");
const API_KEY = required("PORTAINER_API_KEY");
const ENDPOINT_ID = required("PORTAINER_ENDPOINT_ID");
const DATABASE_URL = required("DATABASE_URL");
const BACKUP_TAG = required("BACKUP_TAG");
const CONTAINER_LABEL =
process.env.API_CONTAINER_LABEL ?? "io.jorgecuadros.role=api";
const ALLOW_MISSING = process.env.ALLOW_MISSING_CONTAINER === "true";
const TIMEOUT_MS =
Number(process.env.EXEC_TIMEOUT_SECONDS ?? 1800) * 1000;
function required(name) {
const v = process.env[name];
if (!v) {
console.error(`missing required env: ${name}`);
process.exit(1);
}
return v;
}
const DOCKER = `${PORTAINER_URL}/api/endpoints/${ENDPOINT_ID}/docker`;
async function docker(path, init = {}) {
const res = await fetch(`${DOCKER}${path}`, {
...init,
headers: {
"X-API-Key": API_KEY,
...(init.body ? { "Content-Type": "application/json" } : {}),
...(init.headers ?? {}),
},
});
const text = await res.text();
if (!res.ok) {
throw new Error(`docker ${path} -> ${res.status} ${text.slice(0, 400)}`);
}
return text ? JSON.parse(text) : null;
}
/** Single-quote for `sh -c`, the same discipline ops.service.ts uses. */
function shq(value) {
return `'${String(value).replace(/'/g, `'\\''`)}'`;
}
function parseDbUrl(raw) {
const u = new URL(raw);
return {
host: u.hostname,
port: u.port || "3306",
user: decodeURIComponent(u.username),
password: decodeURIComponent(u.password),
database: u.pathname.replace(/^\//, ""),
};
}
/** Matches ops.service.ts's own naming: ISO, colons and dots flattened. */
function timestamp() {
return new Date()
.toISOString()
.replace(/[:.]/g, "-")
.replace("T", "_")
.slice(0, 19);
}
/**
* ops.service.ts refuses to restore any name outside this character set, so a
* file we write with, say, a `+` in the tag would be permanently unrestorable
* through the UI. Sanitise before writing, not after.
*/
function safeTag(tag) {
return tag.replace(/[^A-Za-z0-9._-]/g, "-");
}
async function findApiContainer() {
const [key, value] = CONTAINER_LABEL.split("=");
const filters = encodeURIComponent(
JSON.stringify({ label: [`${key}=${value}`], status: ["running"] }),
);
const list = await docker(`/containers/json?filters=${filters}`);
return list.length ? list[0] : null;
}
/**
* Run a command in the container and return its exit code. Detach:true keeps
* this to plain HTTP — a non-detached exec start hijacks the connection into a
* raw stream, which fetch cannot read. The cost is that we get no stdout, so
* every check below has to be expressed as an exit code.
*/
async function execInContainer(containerId, cmd, env = []) {
const created = await docker(`/containers/${containerId}/exec`, {
method: "POST",
body: JSON.stringify({
AttachStdout: false,
AttachStderr: false,
Tty: false,
Env: env,
Cmd: ["sh", "-c", cmd],
}),
});
await docker(`/exec/${created.Id}/start`, {
method: "POST",
body: JSON.stringify({ Detach: true, Tty: false }),
});
const deadline = Date.now() + TIMEOUT_MS;
for (;;) {
const info = await docker(`/exec/${created.Id}/json`);
if (!info.Running) return info.ExitCode ?? 1;
if (Date.now() > deadline) {
throw new Error(`exec timed out after ${TIMEOUT_MS / 1000}s`);
}
await new Promise((r) => setTimeout(r, 3000));
}
}
async function main() {
const container = await findApiContainer();
if (!container) {
const message = `no running container matching label ${CONTAINER_LABEL}`;
if (ALLOW_MISSING) {
console.warn(`skipping pre-migrate backup: ${message}`);
return;
}
throw new Error(
`${message} — pass bootstrap=true only if this is the first deploy and ` +
`there is genuinely no data to lose`,
);
}
const conn = parseDbUrl(DATABASE_URL);
const file = `pre-migrate-${safeTag(BACKUP_TAG)}-${timestamp()}.sql.gz`;
const out = `/data/backups/${file}`;
console.log(`container : ${container.Id.slice(0, 12)}`);
console.log(`database : ${conn.user}@${conn.host}:${conn.port}/${conn.database}`);
console.log(`writing : ${out}`);
// Password via MYSQL_PWD in the exec env, never on the command line — argv is
// world-readable through `ps` inside the container.
const flags = `--host=${conn.host} --port=${conn.port} --user=${shq(conn.user)}`;
const dump =
`set -o pipefail; mysqldump ${flags} --single-transaction --routines ` +
`--triggers --no-tablespaces ${shq(conn.database)} | gzip -c > ${shq(out)}`;
const code = await execInContainer(container.Id, dump, [
`MYSQL_PWD=${conn.password}`,
]);
if (code !== 0) {
// Leave the truncated file behind for inspection but never let the deploy
// proceed believing it has a restore point.
throw new Error(`mysqldump exited ${code} — refusing to migrate`);
}
// Detached exec gives no stdout, so prove the artefact separately: non-empty
// file, and gzip that actually decompresses. A dump that fails mid-stream can
// still leave a plausible-looking file.
const verify = await execInContainer(
container.Id,
`test -s ${shq(out)} && gzip -t ${shq(out)}`,
);
if (verify !== 0) {
throw new Error(`backup ${file} is empty or corrupt (check exited ${verify})`);
}
console.log(`ok: ${file} written and verified in the API backup volume`);
}
main().catch((err) => {
console.error(`pre-migrate backup FAILED: ${err.message}`);
process.exit(1);
});