Closes the gap between "what tag did I deploy" and "what is actually running", and gives the schema a history that can be reasoned about across releases. Migrations - Baseline the existing schema as 0000_init (migrate diff --from-empty). The schema had only ever been applied with `prisma db push`, so no history existed and schema state was disconnected from app version. Existing databases must be baselined once with `migrate resolve --applied 0000_init`; the workflows print this remedy on P3005. - Run `prisma migrate deploy` as a deploy STEP, not the container CMD — as a CMD, N replicas would race each other applying the same migration. Version reporting - GET /version on the API reports the APP_VERSION / GIT_SHA / BUILD_DATE that build.yml already baked into both images but nothing ever read. - The web footer shows the web build and flags an api/web mismatch. The two cannot drift at build time (one matrix run) but can at deploy time. - Both deploy workflows now fail if the running API does not report the tag that was dispatched — a stack naming a tag is not proof of what is running. - scripts/set-version.mjs stamps every package.json, which had all sat at 0.1.0 while real releases shipped as v1.x. Pre-migrate backup - deploy/scripts/pre-migrate-backup.mjs dumps the database from INSIDE the still-running old API container over Portainer's Docker API, so the file lands in the volume the Operaciones restore screen reads. A dump taken on the CI runner would be unreachable by the only restore path we have. Verifies the artefact with `gzip -t` before letting the migration proceed. galactus - deploy/galactus/*.compose.yml: standalone-Docker ports of the Swarm stacks. Plain compose silently ignores `deploy:`, so restart_policy becomes `restart: unless-stopped` — without it nothing returns after a host reboot. - .gitea/workflows/deploy-galactus.yml drives endpoint 3 with its own secrets. Fixes - deploy.yml passed `endpoint_id` and `pull_image` to cssnr/portainer-stack-deploy-action, which has no such inputs (they are `endpoint` and `pull`). The endpoint was silently never set. docs/DEPLOY_AND_MIGRATIONS.md documents expand/contract as the rule for schema changes: Prisma has no down-migrations, so a code rollback never rolls the schema back, and restoring the replication master from a dump diverges every replica. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
207 lines
7.1 KiB
JavaScript
207 lines
7.1 KiB
JavaScript
#!/usr/bin/env node
|
|
/**
|
|
* Take a mysqldump immediately before a deploy runs `prisma migrate deploy`.
|
|
*
|
|
* Why it is done THIS way and not with a plain `mysqldump` on the CI runner:
|
|
* a dump is only useful if a human can restore it, and the only restore path
|
|
* this platform has is the "Operaciones" admin screen, which lists and replays
|
|
* whatever `*.sql.gz` files sit in the API container's BACKUP_DIR volume
|
|
* (apps/api/src/ops/ops.service.ts — listBackups / RESTORE). A dump written on
|
|
* the runner would land nowhere anybody can reach. So we drive the dump INSIDE
|
|
* the still-running old API container, via Portainer's Docker API proxy: the
|
|
* container already has mysql-client baked in (docker/api.Dockerfile), and the
|
|
* file lands in the exact directory the restore UI reads.
|
|
*
|
|
* It must therefore run BEFORE the app stack is re-applied, while the previous
|
|
* container is still up.
|
|
*
|
|
* Required env:
|
|
* PORTAINER_URL https://<host>:9443
|
|
* PORTAINER_API_KEY Portainer access token
|
|
* PORTAINER_ENDPOINT_ID numeric endpoint id (galactus = 3)
|
|
* DATABASE_URL mysql://user:pass@host:port/db
|
|
* BACKUP_TAG label for the filename, e.g. the deployed tag
|
|
* Optional env:
|
|
* ALLOW_MISSING_CONTAINER=true exit 0 when no API container exists yet
|
|
* (first-ever deploy — nothing to back up)
|
|
* API_CONTAINER_LABEL=io.jorgecuadros.role=api
|
|
* EXEC_TIMEOUT_SECONDS=1800
|
|
*
|
|
* TLS: Portainer here uses a self-signed certificate. The caller is expected to
|
|
* set NODE_TLS_REJECT_UNAUTHORIZED=0 for this step; see the workflow. That
|
|
* disables verification for the whole process, so nothing else should run in it.
|
|
*/
|
|
|
|
const PORTAINER_URL = required("PORTAINER_URL").replace(/\/+$/, "");
|
|
const API_KEY = required("PORTAINER_API_KEY");
|
|
const ENDPOINT_ID = required("PORTAINER_ENDPOINT_ID");
|
|
const DATABASE_URL = required("DATABASE_URL");
|
|
const BACKUP_TAG = required("BACKUP_TAG");
|
|
|
|
const CONTAINER_LABEL =
|
|
process.env.API_CONTAINER_LABEL ?? "io.jorgecuadros.role=api";
|
|
const ALLOW_MISSING = process.env.ALLOW_MISSING_CONTAINER === "true";
|
|
const TIMEOUT_MS =
|
|
Number(process.env.EXEC_TIMEOUT_SECONDS ?? 1800) * 1000;
|
|
|
|
function required(name) {
|
|
const v = process.env[name];
|
|
if (!v) {
|
|
console.error(`missing required env: ${name}`);
|
|
process.exit(1);
|
|
}
|
|
return v;
|
|
}
|
|
|
|
const DOCKER = `${PORTAINER_URL}/api/endpoints/${ENDPOINT_ID}/docker`;
|
|
|
|
async function docker(path, init = {}) {
|
|
const res = await fetch(`${DOCKER}${path}`, {
|
|
...init,
|
|
headers: {
|
|
"X-API-Key": API_KEY,
|
|
...(init.body ? { "Content-Type": "application/json" } : {}),
|
|
...(init.headers ?? {}),
|
|
},
|
|
});
|
|
const text = await res.text();
|
|
if (!res.ok) {
|
|
throw new Error(`docker ${path} -> ${res.status} ${text.slice(0, 400)}`);
|
|
}
|
|
return text ? JSON.parse(text) : null;
|
|
}
|
|
|
|
/** Single-quote for `sh -c`, the same discipline ops.service.ts uses. */
|
|
function shq(value) {
|
|
return `'${String(value).replace(/'/g, `'\\''`)}'`;
|
|
}
|
|
|
|
function parseDbUrl(raw) {
|
|
const u = new URL(raw);
|
|
return {
|
|
host: u.hostname,
|
|
port: u.port || "3306",
|
|
user: decodeURIComponent(u.username),
|
|
password: decodeURIComponent(u.password),
|
|
database: u.pathname.replace(/^\//, ""),
|
|
};
|
|
}
|
|
|
|
/** Matches ops.service.ts's own naming: ISO, colons and dots flattened. */
|
|
function timestamp() {
|
|
return new Date()
|
|
.toISOString()
|
|
.replace(/[:.]/g, "-")
|
|
.replace("T", "_")
|
|
.slice(0, 19);
|
|
}
|
|
|
|
/**
|
|
* ops.service.ts refuses to restore any name outside this character set, so a
|
|
* file we write with, say, a `+` in the tag would be permanently unrestorable
|
|
* through the UI. Sanitise before writing, not after.
|
|
*/
|
|
function safeTag(tag) {
|
|
return tag.replace(/[^A-Za-z0-9._-]/g, "-");
|
|
}
|
|
|
|
async function findApiContainer() {
|
|
const [key, value] = CONTAINER_LABEL.split("=");
|
|
const filters = encodeURIComponent(
|
|
JSON.stringify({ label: [`${key}=${value}`], status: ["running"] }),
|
|
);
|
|
const list = await docker(`/containers/json?filters=${filters}`);
|
|
return list.length ? list[0] : null;
|
|
}
|
|
|
|
/**
|
|
* Run a command in the container and return its exit code. Detach:true keeps
|
|
* this to plain HTTP — a non-detached exec start hijacks the connection into a
|
|
* raw stream, which fetch cannot read. The cost is that we get no stdout, so
|
|
* every check below has to be expressed as an exit code.
|
|
*/
|
|
async function execInContainer(containerId, cmd, env = []) {
|
|
const created = await docker(`/containers/${containerId}/exec`, {
|
|
method: "POST",
|
|
body: JSON.stringify({
|
|
AttachStdout: false,
|
|
AttachStderr: false,
|
|
Tty: false,
|
|
Env: env,
|
|
Cmd: ["sh", "-c", cmd],
|
|
}),
|
|
});
|
|
|
|
await docker(`/exec/${created.Id}/start`, {
|
|
method: "POST",
|
|
body: JSON.stringify({ Detach: true, Tty: false }),
|
|
});
|
|
|
|
const deadline = Date.now() + TIMEOUT_MS;
|
|
for (;;) {
|
|
const info = await docker(`/exec/${created.Id}/json`);
|
|
if (!info.Running) return info.ExitCode ?? 1;
|
|
if (Date.now() > deadline) {
|
|
throw new Error(`exec timed out after ${TIMEOUT_MS / 1000}s`);
|
|
}
|
|
await new Promise((r) => setTimeout(r, 3000));
|
|
}
|
|
}
|
|
|
|
async function main() {
|
|
const container = await findApiContainer();
|
|
if (!container) {
|
|
const message = `no running container matching label ${CONTAINER_LABEL}`;
|
|
if (ALLOW_MISSING) {
|
|
console.warn(`skipping pre-migrate backup: ${message}`);
|
|
return;
|
|
}
|
|
throw new Error(
|
|
`${message} — pass bootstrap=true only if this is the first deploy and ` +
|
|
`there is genuinely no data to lose`,
|
|
);
|
|
}
|
|
|
|
const conn = parseDbUrl(DATABASE_URL);
|
|
const file = `pre-migrate-${safeTag(BACKUP_TAG)}-${timestamp()}.sql.gz`;
|
|
const out = `/data/backups/${file}`;
|
|
|
|
console.log(`container : ${container.Id.slice(0, 12)}`);
|
|
console.log(`database : ${conn.user}@${conn.host}:${conn.port}/${conn.database}`);
|
|
console.log(`writing : ${out}`);
|
|
|
|
// Password via MYSQL_PWD in the exec env, never on the command line — argv is
|
|
// world-readable through `ps` inside the container.
|
|
const flags = `--host=${conn.host} --port=${conn.port} --user=${shq(conn.user)}`;
|
|
const dump =
|
|
`set -o pipefail; mysqldump ${flags} --single-transaction --routines ` +
|
|
`--triggers --no-tablespaces ${shq(conn.database)} | gzip -c > ${shq(out)}`;
|
|
|
|
const code = await execInContainer(container.Id, dump, [
|
|
`MYSQL_PWD=${conn.password}`,
|
|
]);
|
|
if (code !== 0) {
|
|
// Leave the truncated file behind for inspection but never let the deploy
|
|
// proceed believing it has a restore point.
|
|
throw new Error(`mysqldump exited ${code} — refusing to migrate`);
|
|
}
|
|
|
|
// Detached exec gives no stdout, so prove the artefact separately: non-empty
|
|
// file, and gzip that actually decompresses. A dump that fails mid-stream can
|
|
// still leave a plausible-looking file.
|
|
const verify = await execInContainer(
|
|
container.Id,
|
|
`test -s ${shq(out)} && gzip -t ${shq(out)}`,
|
|
);
|
|
if (verify !== 0) {
|
|
throw new Error(`backup ${file} is empty or corrupt (check exited ${verify})`);
|
|
}
|
|
|
|
console.log(`ok: ${file} written and verified in the API backup volume`);
|
|
}
|
|
|
|
main().catch((err) => {
|
|
console.error(`pre-migrate backup FAILED: ${err.message}`);
|
|
process.exit(1);
|
|
});
|