jorgecuadros-platform/docs/DEPLOY_AND_MIGRATIONS.md

# Releasing, deploying, and changing the schema

How a version gets from this repo onto a server, and the one rule that keeps
rollbacks possible.

## The short version

```bash
pnpm version:set 1.2.0                       # stamp every package.json
git commit -am "chore(release): v1.2.0"
git tag v1.2.0 && git push origin master v1.2.0
```

That push triggers `.gitea/workflows/build.yml`, which builds **both** images in
one matrix run and publishes:

| tag pushed | image tags produced |
| --- | --- |
| `v1.2.0` | `1.2.0`, `1.2`, `sha-<short>` |
| push to `master` | `master`, `sha-<short>`, `latest` |

Then dispatch a deploy from the Actions tab:

- **galactus** (office server, standalone Docker) — *Deploy to galactus*
- **cubex** (3-node Swarm) — *Deploy to Portainer*

> **The `v` is not part of the image tag.** `docker/metadata-action`'s
> `{{version}}` strips it. Git tag `v1.2.0`, dispatch `1.2.0`. Dispatching
> `v1.2.0` deploys nothing that exists.

Because api and web are built from one matrix run, they cannot drift at build
time. They *can* drift at deploy time if a stack is applied with only one image
moved — the web footer shows both versions and flags a mismatch, and the deploy
workflow's last step fails if the API does not report the tag you dispatched.

## What a deploy actually does

1. **db + minio** — `scope: full` only. Idempotent; data lives on named volumes.
2. **Pre-migrate backup** — `deploy/scripts/pre-migrate-backup.mjs` runs
   `mysqldump` *inside the still-running old API container*, via Portainer's
   Docker API. The file lands in that container's `BACKUP_DIR` volume as
   `pre-migrate-<tag>-<timestamp>.sql.gz`, which is exactly what the
   **Operaciones** admin screen lists and can restore. A dump taken on the CI
   runner would be unreachable by the only restore path the platform has.
3. **`prisma migrate deploy`** — as a workflow *step*, never the container
   `CMD`. If it were the CMD, N replicas would race each other applying the
   same migration.
4. **app** — the new api + web images.
5. **Verify** — `GET /version` on the running API must report the dispatched
   tag.

Rollback is `tag: 1.1.9` re-dispatched. **That rolls back code only.** The
schema stays where it is. Which brings us to the rule.

## The rule: expand / contract

Prisma has no down-migrations. There is no `prisma migrate down`, and there
never will be. So a schema change that the *previous* release cannot tolerate
turns a 30-second rollback into a restore-from-backup outage.

**Every schema change must leave the previous release working.** Split anything
destructive across two releases:

| | Release N (expand) | Release N+1 (contract) |
| --- | --- | --- |
| Rename a column | add the new column, write to both, read the old | drop the old column |
| Drop a column | stop reading and writing it in code | drop it |
| Add a required column | add it nullable (or with a default), backfill | make it `NOT NULL` |
| Split a table | create the new table, dual-write | stop writing the old, drop it |
| Add an enum value | add the value; old code must not choke on unknowns | start emitting it |

Ship N, let it soak, *then* ship N+1. If N has to be rolled back you just
re-dispatch the old tag — the expanded schema still satisfies it.

Restoring from the pre-migrate dump is the **emergency lever, not the routine
path**, and on galactus it is worse than it sounds: galactus is the replication
master, DDL replicates through the binlog, and restoring the master from a dump
diverges every replica. GTIDs will not line up and each replica needs a full
re-seed. Assume a restore is a multi-hour, whole-topology event.

## Migration history

`packages/database/prisma/migrations/0000_init/` is a **baseline**. It is the
full schema as it stood on 2026-07-30, generated with:

```bash
prisma migrate diff --from-empty \
  --to-schema-datamodel packages/database/prisma/schema.prisma --script
```

Until then the schema had only ever been applied with `prisma db push`, so no
history existed and the schema state was disconnected from the app version.

### One-time, on every database that already exists

`0000_init` describes tables those databases already have, so `migrate deploy`
would fail with **P3005 "the database schema is not empty"**. Mark it applied
instead of applying it — this writes a `_prisma_migrations` row and changes no
data:

```bash
DATABASE_URL=<the database> npx prisma@5 migrate resolve \
  --applied 0000_init --schema packages/database/prisma/schema.prisma
```

Do this once per database (prod, dev, any local copy). Verify first that the
live schema really does match the baseline — this should print an empty
migration:

```bash
prisma migrate diff --from-url "$DATABASE_URL" \
  --to-schema-datamodel packages/database/prisma/schema.prisma --script
```

If it prints actual statements, the live database has drifted from
`schema.prisma`. Reconcile *before* baselining, or the first real migration
will fail against a schema Prisma believes it already knows.

### From here on

```bash
# edit schema.prisma, then:
pnpm --filter @jorgecuadros/database exec prisma migrate dev --name add_foo
```

Commit the generated `migrations/<timestamp>_add_foo/` directory. `db push` is
now a local-scratch tool only — using it against a database with history
desynchronises it from `_prisma_migrations`.

## galactus vs cubex

`galactus` is standalone Docker (Portainer endpoint **3**), `cubex` is a 3-node
Swarm (endpoint **2**). They need different compose files because **plain
compose silently ignores Swarm's `deploy:` keys** rather than erroring:

| | Swarm (`deploy/*.stack.yml`) | standalone (`deploy/galactus/*.compose.yml`) |
| --- | --- | --- |
| restart | `deploy.restart_policy` | `restart: unless-stopped` — **without this nothing comes back after a host reboot** |
| placement | `node.labels.jorgecuadros_db == true` | dropped, one host |
| ports | `{mode: ingress}` long syntax | `"3306:3306"` |
| `depends_on` | ignored by Swarm | honoured, with `condition: service_healthy` |
| volumes | named | named (unchanged — the pinning hazard was a Swarm problem) |

Keep the two sets in sync when either changes.

On both hosts, cross-stack traffic goes over the **host address**, not compose
service DNS: db, minio and app are three separate stacks, so three separate
networks. `DATABASE_URL` and `S3_ENDPOINT` name the host and its published
port. Do not "simplify" them to `mysql:3306`.

### galactus is addressed by MagicDNS, and containers need help resolving it

galactus is Tailscale-only once it is installed in the office, so every URL
names `galactus.tail01aa2.ts.net`. Its LAN IP is a DHCP lease and has already
drifted once — never put a `192.168.4.x` address in a secret.

Containers on galactus cannot resolve that name by default. The host runs
systemd-resolved, whose `127.0.0.53` stub is unreachable from inside a
container, so Docker falls back to the upstream resolver listed in
`/run/systemd/resolve/resolv.conf` — the LAN router, which knows nothing about
the tailnet. Routing to `100.x` works fine; only the *lookup* fails, and the
symptom is Prisma **P1001 "can't reach database server"** on a container that
otherwise started cleanly.

`deploy/galactus/jorgecuadros-app.compose.yml` therefore pins the resolver:

```yaml
dns:        [100.100.100.100]        # Tailscale's fixed anycast MagicDNS address
dns_search: [tail01aa2.ts.net]       # this tailnet's suffix
```

Both are overridable (`TAILSCALE_DNS`, `TAILNET_SUFFIX`) if the tailnet changes.
Browser-facing origins need none of this — those names resolve on the client.

## Replication

galactus's MySQL is the **master**; every other MySQL in the estate is a
replica. Consequences that bite:

- `server-id` must be unique across the whole topology (prod `1`, cubex dev
  `11`). A duplicate breaks replication silently.
- GTID is on from first boot, so replicas attach with `SOURCE_AUTO_POSITION=1`.
- `binlog_expire_logs_seconds` is raised to 60 days in the galactus compose file
  (`MYSQL_BINLOG_EXPIRE_SECONDS`). MySQL 8.4 defaults to 30 days; a replica
  offline longer than the retention needs a full re-seed.

Still open, and **not** handled by anything in this repo:

- No replication user with `REPLICATION SLAVE` granted exists yet.
- Nothing sets `read_only` / `super_read_only` on the replicas, so a stray write
  to a replica will diverge it.
- The channel to the VPS crosses the public internet. It needs a tunnel or TLS —
  do not publish raw 3306.

## Known caveats in the deploy path

- The pre-migrate backup step sets `NODE_TLS_REJECT_UNAUTHORIZED=0` because
  Portainer serves a self-signed certificate. It is scoped to that one step,
  which talks to nothing but Portainer. Replacing the certificate and dropping
  the flag is the real fix.
- The runner lives on cubex and must reach the target host's Portainer (9443)
  **and** MySQL (3306). If it cannot reach 3306, run the migration by hand from
  a host that can and dispatch with `skip_migrate: true`.
- `bootstrap: true` lets the pre-migrate backup be skipped when no API container
  exists yet. Use it for a first-ever deploy only — it is the one switch that
  lets a migration run with no restore point.