feat(deploy): prisma migration history, /version, galactus standalone deploy
Closes the gap between "what tag did I deploy" and "what is actually running", and gives the schema a history that can be reasoned about across releases. Migrations - Baseline the existing schema as 0000_init (migrate diff --from-empty). The schema had only ever been applied with `prisma db push`, so no history existed and schema state was disconnected from app version. Existing databases must be baselined once with `migrate resolve --applied 0000_init`; the workflows print this remedy on P3005. - Run `prisma migrate deploy` as a deploy STEP, not the container CMD — as a CMD, N replicas would race each other applying the same migration. Version reporting - GET /version on the API reports the APP_VERSION / GIT_SHA / BUILD_DATE that build.yml already baked into both images but nothing ever read. - The web footer shows the web build and flags an api/web mismatch. The two cannot drift at build time (one matrix run) but can at deploy time. - Both deploy workflows now fail if the running API does not report the tag that was dispatched — a stack naming a tag is not proof of what is running. - scripts/set-version.mjs stamps every package.json, which had all sat at 0.1.0 while real releases shipped as v1.x. Pre-migrate backup - deploy/scripts/pre-migrate-backup.mjs dumps the database from INSIDE the still-running old API container over Portainer's Docker API, so the file lands in the volume the Operaciones restore screen reads. A dump taken on the CI runner would be unreachable by the only restore path we have. Verifies the artefact with `gzip -t` before letting the migration proceed. galactus - deploy/galactus/*.compose.yml: standalone-Docker ports of the Swarm stacks. Plain compose silently ignores `deploy:`, so restart_policy becomes `restart: unless-stopped` — without it nothing returns after a host reboot. - .gitea/workflows/deploy-galactus.yml drives endpoint 3 with its own secrets. Fixes - deploy.yml passed `endpoint_id` and `pull_image` to cssnr/portainer-stack-deploy-action, which has no such inputs (they are `endpoint` and `pull`). The endpoint was silently never set. docs/DEPLOY_AND_MIGRATIONS.md documents expand/contract as the rule for schema changes: Prisma has no down-migrations, so a code rollback never rolls the schema back, and restoring the replication master from a dump diverges every replica. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,182 @@
|
||||
# Releasing, deploying, and changing the schema
|
||||
|
||||
How a version gets from this repo onto a server, and the one rule that keeps
|
||||
rollbacks possible.
|
||||
|
||||
## The short version
|
||||
|
||||
```bash
|
||||
pnpm version:set 1.2.0 # stamp every package.json
|
||||
git commit -am "chore(release): v1.2.0"
|
||||
git tag v1.2.0 && git push origin master v1.2.0
|
||||
```
|
||||
|
||||
That push triggers `.gitea/workflows/build.yml`, which builds **both** images in
|
||||
one matrix run and publishes:
|
||||
|
||||
| tag pushed | image tags produced |
|
||||
| --- | --- |
|
||||
| `v1.2.0` | `1.2.0`, `1.2`, `sha-<short>` |
|
||||
| push to `master` | `master`, `sha-<short>`, `latest` |
|
||||
|
||||
Then dispatch a deploy from the Actions tab:
|
||||
|
||||
- **galactus** (office server, standalone Docker) — *Deploy to galactus*
|
||||
- **cubex** (3-node Swarm) — *Deploy to Portainer*
|
||||
|
||||
> **The `v` is not part of the image tag.** `docker/metadata-action`'s
|
||||
> `{{version}}` strips it. Git tag `v1.2.0`, dispatch `1.2.0`. Dispatching
|
||||
> `v1.2.0` deploys nothing that exists.
|
||||
|
||||
Because api and web are built from one matrix run, they cannot drift at build
|
||||
time. They *can* drift at deploy time if a stack is applied with only one image
|
||||
moved — the web footer shows both versions and flags a mismatch, and the deploy
|
||||
workflow's last step fails if the API does not report the tag you dispatched.
|
||||
|
||||
## What a deploy actually does
|
||||
|
||||
1. **db + minio** — `scope: full` only. Idempotent; data lives on named volumes.
|
||||
2. **Pre-migrate backup** — `deploy/scripts/pre-migrate-backup.mjs` runs
|
||||
`mysqldump` *inside the still-running old API container*, via Portainer's
|
||||
Docker API. The file lands in that container's `BACKUP_DIR` volume as
|
||||
`pre-migrate-<tag>-<timestamp>.sql.gz`, which is exactly what the
|
||||
**Operaciones** admin screen lists and can restore. A dump taken on the CI
|
||||
runner would be unreachable by the only restore path the platform has.
|
||||
3. **`prisma migrate deploy`** — as a workflow *step*, never the container
|
||||
`CMD`. If it were the CMD, N replicas would race each other applying the
|
||||
same migration.
|
||||
4. **app** — the new api + web images.
|
||||
5. **Verify** — `GET /version` on the running API must report the dispatched
|
||||
tag.
|
||||
|
||||
Rollback is `tag: 1.1.9` re-dispatched. **That rolls back code only.** The
|
||||
schema stays where it is. Which brings us to the rule.
|
||||
|
||||
## The rule: expand / contract
|
||||
|
||||
Prisma has no down-migrations. There is no `prisma migrate down`, and there
|
||||
never will be. So a schema change that the *previous* release cannot tolerate
|
||||
turns a 30-second rollback into a restore-from-backup outage.
|
||||
|
||||
**Every schema change must leave the previous release working.** Split anything
|
||||
destructive across two releases:
|
||||
|
||||
| | Release N (expand) | Release N+1 (contract) |
|
||||
| --- | --- | --- |
|
||||
| Rename a column | add the new column, write to both, read the old | drop the old column |
|
||||
| Drop a column | stop reading and writing it in code | drop it |
|
||||
| Add a required column | add it nullable (or with a default), backfill | make it `NOT NULL` |
|
||||
| Split a table | create the new table, dual-write | stop writing the old, drop it |
|
||||
| Add an enum value | add the value; old code must not choke on unknowns | start emitting it |
|
||||
|
||||
Ship N, let it soak, *then* ship N+1. If N has to be rolled back you just
|
||||
re-dispatch the old tag — the expanded schema still satisfies it.
|
||||
|
||||
Restoring from the pre-migrate dump is the **emergency lever, not the routine
|
||||
path**, and on galactus it is worse than it sounds: galactus is the replication
|
||||
master, DDL replicates through the binlog, and restoring the master from a dump
|
||||
diverges every replica. GTIDs will not line up and each replica needs a full
|
||||
re-seed. Assume a restore is a multi-hour, whole-topology event.
|
||||
|
||||
## Migration history
|
||||
|
||||
`packages/database/prisma/migrations/0000_init/` is a **baseline**. It is the
|
||||
full schema as it stood on 2026-07-30, generated with:
|
||||
|
||||
```bash
|
||||
prisma migrate diff --from-empty \
|
||||
--to-schema-datamodel packages/database/prisma/schema.prisma --script
|
||||
```
|
||||
|
||||
Until then the schema had only ever been applied with `prisma db push`, so no
|
||||
history existed and the schema state was disconnected from the app version.
|
||||
|
||||
### One-time, on every database that already exists
|
||||
|
||||
`0000_init` describes tables those databases already have, so `migrate deploy`
|
||||
would fail with **P3005 "the database schema is not empty"**. Mark it applied
|
||||
instead of applying it — this writes a `_prisma_migrations` row and changes no
|
||||
data:
|
||||
|
||||
```bash
|
||||
DATABASE_URL=<the database> npx prisma@5 migrate resolve \
|
||||
--applied 0000_init --schema packages/database/prisma/schema.prisma
|
||||
```
|
||||
|
||||
Do this once per database (prod, dev, any local copy). Verify first that the
|
||||
live schema really does match the baseline — this should print an empty
|
||||
migration:
|
||||
|
||||
```bash
|
||||
prisma migrate diff --from-url "$DATABASE_URL" \
|
||||
--to-schema-datamodel packages/database/prisma/schema.prisma --script
|
||||
```
|
||||
|
||||
If it prints actual statements, the live database has drifted from
|
||||
`schema.prisma`. Reconcile *before* baselining, or the first real migration
|
||||
will fail against a schema Prisma believes it already knows.
|
||||
|
||||
### From here on
|
||||
|
||||
```bash
|
||||
# edit schema.prisma, then:
|
||||
pnpm --filter @jorgecuadros/database exec prisma migrate dev --name add_foo
|
||||
```
|
||||
|
||||
Commit the generated `migrations/<timestamp>_add_foo/` directory. `db push` is
|
||||
now a local-scratch tool only — using it against a database with history
|
||||
desynchronises it from `_prisma_migrations`.
|
||||
|
||||
## galactus vs cubex
|
||||
|
||||
`galactus` is standalone Docker (Portainer endpoint **3**), `cubex` is a 3-node
|
||||
Swarm (endpoint **2**). They need different compose files because **plain
|
||||
compose silently ignores Swarm's `deploy:` keys** rather than erroring:
|
||||
|
||||
| | Swarm (`deploy/*.stack.yml`) | standalone (`deploy/galactus/*.compose.yml`) |
|
||||
| --- | --- | --- |
|
||||
| restart | `deploy.restart_policy` | `restart: unless-stopped` — **without this nothing comes back after a host reboot** |
|
||||
| placement | `node.labels.jorgecuadros_db == true` | dropped, one host |
|
||||
| ports | `{mode: ingress}` long syntax | `"3306:3306"` |
|
||||
| `depends_on` | ignored by Swarm | honoured, with `condition: service_healthy` |
|
||||
| volumes | named | named (unchanged — the pinning hazard was a Swarm problem) |
|
||||
|
||||
Keep the two sets in sync when either changes.
|
||||
|
||||
On both hosts, cross-stack traffic goes over the **host address**, not compose
|
||||
service DNS: db, minio and app are three separate stacks, so three separate
|
||||
networks. `DATABASE_URL` and `S3_ENDPOINT` name the host and its published
|
||||
port. Do not "simplify" them to `mysql:3306`.
|
||||
|
||||
## Replication
|
||||
|
||||
galactus's MySQL is the **master**; every other MySQL in the estate is a
|
||||
replica. Consequences that bite:
|
||||
|
||||
- `server-id` must be unique across the whole topology (prod `1`, cubex dev
|
||||
`11`). A duplicate breaks replication silently.
|
||||
- GTID is on from first boot, so replicas attach with `SOURCE_AUTO_POSITION=1`.
|
||||
- `binlog_expire_logs_seconds` is raised to 60 days in the galactus compose file
|
||||
(`MYSQL_BINLOG_EXPIRE_SECONDS`). MySQL 8.4 defaults to 30 days; a replica
|
||||
offline longer than the retention needs a full re-seed.
|
||||
|
||||
Still open, and **not** handled by anything in this repo:
|
||||
|
||||
- No replication user with `REPLICATION SLAVE` granted exists yet.
|
||||
- Nothing sets `read_only` / `super_read_only` on the replicas, so a stray write
|
||||
to a replica will diverge it.
|
||||
- The channel to the VPS crosses the public internet. It needs a tunnel or TLS —
|
||||
do not publish raw 3306.
|
||||
|
||||
## Known caveats in the deploy path
|
||||
|
||||
- The pre-migrate backup step sets `NODE_TLS_REJECT_UNAUTHORIZED=0` because
|
||||
Portainer serves a self-signed certificate. It is scoped to that one step,
|
||||
which talks to nothing but Portainer. Replacing the certificate and dropping
|
||||
the flag is the real fix.
|
||||
- The runner lives on cubex and must reach the target host's Portainer (9443)
|
||||
**and** MySQL (3306). If it cannot reach 3306, run the migration by hand from
|
||||
a host that can and dispatch with `skip_migrate: true`.
|
||||
- `bootstrap: true` lets the pre-migrate backup be skipped when no API container
|
||||
exists yet. Use it for a first-ever deploy only — it is the one switch that
|
||||
lets a migration run with no restore point.
|
||||
Reference in New Issue
Block a user