feat(ops): show read-replica health on the Operaciones screen

my.jorgecuadros.com serves customer balances from the Oracle VPS replica.
A replica whose SQL thread has stopped does not error — it keeps answering,
with data frozen at the moment it stopped — so nothing on the customer site
looks wrong and the only signal is a customer complaining about a stale
balance. This puts the failure somewhere a human sees it.

Deliberately does not trust the two fields an operator reaches for first.
Replica_IO_Running reports Yes while the SQL thread is stopped, because the
network thread keeps downloading binlog it will never apply; verified by
stopping SQL_THREAD and watching IO stay Yes. Seconds_Behind_Source reads
NULL whenever EITHER thread is down, so the card renders "sin dato" rather
than "0 s" — showing zero there would report an outage as perfect health.
The problem string is resolved most-specific-first for the same reason.

Shells out to the mysql client because the API has no MySQL driver and the
image already ships one. --ssl is required (the replica sets
require_secure_transport); --ssl-verify-server-cert=0 is deliberate and is
NOT the trade-off the website makes: this hop never leaves Tailscale and the
replica's firewall admits only this host, so WireGuard authenticates the
peer, whereas the DreamHost leg crosses the public internet and pins the CA.

The account behind it holds REPLICATION CLIENT and nothing else — it cannot
read a single row. REPLICA_DB_* unset is a supported state and renders "no
configurada", which is correct in dev and before cutover.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-03 17:22:15 -07:00
co-authored by Claude Opus 5
parent 1d689d8f46
commit dac1f1982f
8 changed files with 309 additions and 1 deletions
+103
View File
@@ -14,6 +14,7 @@ import {
deleteBackup,
deleteIngest,
getOpsJob,
getReplicationStatus,
listBackups,
listIngest,
listOpsJobs,
@@ -26,6 +27,7 @@ import type {
IngestFile,
OpsJob,
OpsJobKind,
ReplicationStatus,
} from "@/lib/types";
const INGEST_MAX_BYTES = 2 * 1024 * 1024 * 1024;
@@ -227,6 +229,8 @@ function Operaciones() {
</div>
)}
<ReplicationCard />
{/* Ingest folder */}
<div className="card" style={{ padding: 20, marginBottom: 20 }}>
<h2 className="section-title">Carpeta de ingesta</h2>
@@ -596,3 +600,102 @@ function OpTile({
</div>
);
}
/**
* Health of the read replica behind my.jorgecuadros.com.
*
* Worth a panel because the failure mode is silent: a replica whose SQL thread
* has stopped keeps answering queries, just with data frozen at the moment it
* stopped. Nothing on the customer site looks wrong — the balances are simply
* out of date — so without this the only signal is a customer complaining.
*/
function ReplicationCard() {
const [status, setStatus] = useState<ReplicationStatus | null>(null);
const [failed, setFailed] = useState(false);
const load = useCallback(() => {
getReplicationStatus()
.then((s) => {
setStatus(s);
setFailed(false);
})
.catch(() => setFailed(true));
}, []);
useEffect(() => {
load();
const t = setInterval(load, 30_000);
return () => clearInterval(t);
}, [load]);
// Not configured is the normal state in dev and before cutover, so it is a
// quiet note rather than an alarm — showing red here would train people to
// ignore the card.
if (failed || (status && !status.configured)) {
return (
<div className="card" style={{ padding: 20, marginBottom: 20 }}>
<h2 className="section-title">Réplica del sitio de clientes</h2>
<p className="inline-form-note">
{failed
? "No se pudo consultar el estado de la réplica."
: "No configurada en este entorno."}
</p>
</div>
);
}
if (!status) {
return (
<div className="card" style={{ padding: 20, marginBottom: 20 }}>
<h2 className="section-title">Réplica del sitio de clientes</h2>
<p className="inline-form-note">Consultando</p>
</div>
);
}
return (
<div className="card" style={{ padding: 20, marginBottom: 20 }}>
<div className="row-actions" style={{ justifyContent: "space-between" }}>
<h2 className="section-title" style={{ margin: 0 }}>
Réplica del sitio de clientes{" "}
<span className={`badge ${status.healthy ? "badge-positive" : "badge-negative"}`}>
{status.healthy ? "Replicando" : "Detenida"}
</span>
</h2>
<button className="btn btn-ghost" type="button" onClick={load}>
Actualizar
</button>
</div>
{status.problem && (
<div className="state-box state-error" style={{ marginTop: 12 }}>
{status.problem}
</div>
)}
<div className="kv-grid" style={{ paddingLeft: 0, paddingRight: 0 }}>
<KV label="Servidor" value={status.host} />
<KV label="Origen" value={status.sourceHost} />
<KV label="Hilo de E/S" value={status.ioRunning} />
<KV label="Hilo SQL" value={status.sqlRunning} />
{/* Never render a null lag as "0 s": MySQL reports NULL whenever a
thread is down, so the honest word is "unknown", not "up to date". */}
<KV
label="Retraso"
value={status.secondsBehind === null ? "sin dato" : `${status.secondsBehind} s`}
/>
<KV label="Consultado" value={formatDateTime(status.checkedAt)} />
</div>
</div>
);
}
/** Matches the KV in the clientes/polizas/servicios detail pages. */
function KV({ label, value }: { label: string; value: string | null | undefined }) {
return (
<div>
<div className="kv-label">{label}</div>
<div className="kv-value">{value || "—"}</div>
</div>
);
}
+11
View File
@@ -59,6 +59,7 @@ import type {
LookupsResponse,
OpsJob,
OpsJobKind,
ReplicationStatus,
IngestFile,
BackupFile,
PropertyDetail,
@@ -939,6 +940,16 @@ export function deleteBackup(name: string): Promise<unknown> {
return apiFetch(`/ops/backups/${encodeURIComponent(name)}`, { method: "DELETE" });
}
/**
* Health of the read replica my.jorgecuadros.com serves customers from.
*
* A stopped replica does not error — it answers with stale balances — so this
* is the only place the failure is visible.
*/
export function getReplicationStatus(): Promise<ReplicationStatus> {
return apiFetch<ReplicationStatus>("/ops/replication");
}
export function listOpsJobs(): Promise<OpsJob[]> {
return apiFetch<OpsJob[]>("/ops/jobs");
}
+21
View File
@@ -72,6 +72,27 @@ export interface OpsJob {
finishedAt: string | null;
}
/**
* Health of the MySQL read replica that my.jorgecuadros.com queries.
*
* `secondsBehind` is null whenever MySQL reports NULL, which it does when
* EITHER thread is down — so null means "unknown", never "up to date". Read
* `healthy`/`problem` rather than inferring health from the lag.
*/
export interface ReplicationStatus {
configured: boolean;
healthy: boolean;
host: string | null;
ioRunning: string | null;
sqlRunning: string | null;
secondsBehind: number | null;
lastIoError: string | null;
lastSqlError: string | null;
sourceHost: string | null;
problem: string | null;
checkedAt: string;
}
/** One of the four legacy Access files expected in the ingest folder. */
export interface IngestFile {
name: string;