Outages are rarely sudden
Most database incidents announce themselves hours ahead: connections creep towards the pool limit, replication lag drifts upwards, a query plan flips and p95 latency doubles. The failure is only sudden if nobody was watching the slope.
The five to alert on
Connection saturation as a percentage of max_connections. Replication lag in seconds behind primary. Slow query count compared to the same hour last week. Disk headroom projected forward at the current growth rate. Lock wait time on your busiest tables.
Alert on trends rather than instantaneous thresholds — a single spike is noise, a sustained slope is an incident forming.
Pair monitoring with backups
When monitoring and backups live in the same place, a degradation alert also tells you exactly which restore point is closest to the moment things went wrong. That is the difference between a five-minute recovery and an afternoon of forensics.
