Progress / Phase 12

Phase 12: Operations

Lorehallow runs like a real live service, with metrics, logs, alerts, tested backups and deploys that don’t throw everyone out.

Phase 120 / 21 · 0%

This phase is still ahead.

12.1 Metrics and dashboards

0 of 3 done

Metrics completed for every service (the Phase 3 set plus database latency and errors, persistence queue depth and handoffs), each documented in docs/ops/metrics.md.
Decision Monitoring stack on the host (Prometheus and Grafana, or something lighter), reachable only through an SSH tunnel or an authenticated proxy.
Dashboards for server health, player activity and performance.

0 of 1 done

Structured logs carrying session and request ids, a retention policy, and documented ways to search them.

0 of 1 done

Alerts to the owner for: a service down, tick overruns, disk over 80%, a failed backup and certificates close to expiry.

0 of 4 done

Decision Zero-downtime strategy: drain and hand connections off through the gateway, or scheduled restarts with in-game countdowns and automatic reconnects.
Implement it: a server deploy loses no progress and causes at most a brief, automatic reconnect.
Database migrations follow expand-and-contract so the previous server version still works, documented in docs/ops/migrations.md.
Minimum client version announced by the server; outdated clients get an “Update required” screen linking to /downloads.

0 of 2 done

Runbooks in docs/ops/ for: server down, database full, bad deploy, compromised account, restoring from backup and the server at capacity.
Game-day exercise: simulate two incidents and follow the runbooks, fixing whatever was wrong in them.

0 of 1 done

0 of 3 done

24-hour soak (scaling.md day soak) on prod.
Case 1 Tier 3 (5,000 concurrent connections) measured on dev-m5 and prod, including Caddy’s memory per connection.
A nightly scheduled job runs the fuzz targets for an hour and a short spread load test, and alerts on failure.

0 of 2 done

docs/walkthroughs/phase-12-operations.md: metrics, alerts, deploy strategy and incident response.
Devlog draft, release notes, tag v0.12.0 and publish builds.

0 of 4 done