Scenario · Multi-Database
App database down, default database healthy
A sandboxed PostgreSQL incident — investigate with your own tools, submit a fix, and get deterministic Detect / Fix / Trap scoring.
L2 · 10–15 min · runs locally in Docker
Launch
Start this scenario
Boot it in a real PostgreSQL sandbox and investigate with psql, EXPLAIN and pg_stat_statements.
ride postgres start stage-06/02-app-database-down-default-healthyPart of these paths
Show the postmortem & investigation hints spoilers
App database down, default database healthy
Type: incident simulation · Topic: Multi-Database · Level: L2 · Duration: 10–15 min
Launch: ride postgres start stage-06/02-app-database-down-default-healthy
POSTMORTEM (root cause · how it was found · the fix · lesson)
Root cause: app_db had `datallowconn = false` (ALLOW_CONNECTIONS turned off), so it
refused every new connection. The default database stayed perfectly healthy, so a
generic health check that only probes the default database showed all-green while
the application was completely down.
How it was found: pg_database showed app_db with datallowconn = false while the
default database was fine; a direct connection to app_db failed.
The mitigation: ALTER DATABASE app_db WITH ALLOW_CONNECTIONS true.
Lesson: health-check every database your app actually uses, not just the default
one. A database-specific outage is invisible to a default-database probe. The
cluster wasn't out of connections, so raising max_connections is wrong, and an
index is irrelevant.
INVESTIGATION HINTS (the staged path to diagnose and fix)
1. Your generic health check hits the default database and it's green — but the app is down. Check connectivity per database: SELECT datname, datallowconn, datconnlimit FROM pg_database WHERE datname IN ('incident','app_db'); app_db has datallowconn = false.
2. app_db is refusing all new connections (ALLOW_CONNECTIONS was turned off). The default database being healthy tells you nothing about app_db.
3. Re-enable it: ALTER DATABASE app_db WITH ALLOW_CONNECTIONS true; Don't raise max_connections (the cluster isn't out of connections) and don't add indexes.