A misconfigured escalation policy has two failure modes, and both are bad. Either every engineer on the team gets paged for a minor hiccup, or nobody gets paged at all because the alert fell into a routing gap. The first creates fatigue. The second creates outages that last until a customer complains.
The difference between those outcomes is rarely the monitoring tool. It is the routing logic between "check failed" and "the right human is looking at it." That routing logic is the escalation policy.
This guide covers how to design escalation policies that match alert urgency to responder availability — without turning on-call into a sleep-deprivation experiment.
What an escalation policy actually does
An escalation policy is a set of rules that determines who gets notified about an incident, through which channel, and what happens if they do not respond within a defined window.
It sits between the alert rule and the human. The alert rule says "something is wrong." The escalation policy says "tell this person, then that person, then that person's manager, in this order, through these channels, at these intervals."
Without one, alerts go to a shared Slack channel where everyone assumes someone else is handling it. With a poorly designed one, alerts go to everyone simultaneously and nobody knows who owns the response.
A good escalation policy answers three questions:
Who should respond first? Usually the on-call engineer closest to the affected service.
What happens if they don't respond? The alert moves to the next tier after a defined timeout.
When should you skip the chain entirely? Some incidents are severe enough to page multiple people from the start.
Tiered escalation design
Most teams benefit from a three- or four-tier model. Each tier represents a wider circle of responders with increasing authority and decreasing system-specific context.
Tier 1 — Primary on-call. The engineer currently carrying the pager for the affected service. They have the most context and are expected to acknowledge within 5 minutes. This is the person who should handle 80% of alerts without escalation.
Tier 2 — Backup on-call. A second engineer on the rotation, often the previous week's primary. If Tier 1 does not acknowledge within the timeout window, the alert moves here. The backup may have slightly less context on recent changes but knows the system well enough to investigate.
Tier 3 — Engineering lead or team lead. If both on-call engineers are unreachable — phone off, traveling, handling a separate incident — the alert reaches someone with authority to pull in additional help, initiate a broader response, or contact people outside the rotation.
Tier 4 — Management or incident commander. Reserved for extended outages where Tiers 1–3 are either overwhelmed or the incident has business-level impact requiring communication with stakeholders, customers, or leadership.
The key insight: each tier adds responders without removing the previous ones. When Tier 2 is paged, Tier 1 is still expected to respond if they become available. The chain widens the net; it does not shift ownership.
Time-based auto-escalation
The most common escalation trigger is an unacknowledged alert after a fixed window. A reasonable starting point:
0 minutes: Alert fires. Primary on-call is paged via push notification and phone call.
5 minutes unacknowledged: Backup on-call is paged.
15 minutes unacknowledged: Engineering lead is paged. The alert is now treated as potentially unattended.
30 minutes unacknowledged: VP or incident commander is notified. At this point the question is not "who can fix this" but "why has nobody responded."
These windows are not universal. A team with excellent acknowledgment rates might extend them to 10/20/40. A team covering a payment-processing system where every minute matters might compress them to 3/10/20. The right intervals come from your actual response data — track acknowledgment times for a month and set the first escalation threshold just above your p90.
One nuance that is easy to miss: acknowledgment is not resolution. "I see this and I am looking at it" is enough to pause the escalation clock. The goal of escalation is to confirm someone is aware, not to confirm the fix is deployed.
When to skip levels
Not every incident should walk the chain. A complete site outage affecting all customers should not wait 15 minutes for the backup to get a turn. Severity-based skip logic lets critical alerts bypass the queue.
A common pattern ties configuration so the routing is automatic.
Override mechanisms
Schedules are plans. Reality is messier. Escalation policies need escape hatches:
Temporary overrides let an on-call engineer hand off the pager for a few hours — a doctor's appointment, a flight, a family event. The override routes alerts to the backup without changing the rotation schedule. Most on-call tools support this natively; if yours does not, a shared calendar with manual swaps works but introduces coordination risk.
Direct page allows anyone to invoke the escalation chain for an incident they observe but cannot fix themselves. A frontend engineer noticing database connection errors should be able to page the database on-call without navigating the org chart.
Escalation suppression prevents known maintenance from triggering the chain. If the team is performing a scheduled database migration that will cause transient errors for 20 minutes, suppress escalation for the affected monitors during the window. The alerts should still fire (so you have the audit trail), but they should not page anyone.
Manual escalation lets the responding engineer bump an incident to the next tier without waiting for the timeout. If the primary sees something they cannot diagnose — an unfamiliar service, a permissions issue, an infrastructure problem outside their domain — they should escalate immediately rather than burning the clock.
Channel routing by severity
Escalation is not just about who gets paged. It is about how. A 3 AM phone call carries a very different weight than a Slack message, and using the wrong channel for the wrong severity erodes trust in the system.
A practical channel matrix:
| Severity | Channels |
|---|---|
| SEV4 | Slack or email only |
| SEV3 | Push notification + Slack |
| SEV2 | Push notification + phone call + Slack |
| SEV1 | Phone call + SMS + push + Slack + incident channel |
The principle: as severity increases, add channels rather than switching them. A SEV2 alert still goes to Slack so the rest of the team has visibility, but the primary responder gets a phone call they cannot miss.
DevHelm's .
Document who is on call right now. It sounds obvious, but the most common escalation failure is not a misconfigured policy — it is an out-of-date schedule. Automate schedule publishing so the current on-call is always visible in your incident tooling.
Measuring whether it works
An escalation policy is not a set-and-forget artifact. Track these metrics monthly:
Acknowledgment time (p50 and p90). If p90 exceeds your first escalation window, either the window is too short or the primary on-call is routinely unreachable.
Escalation rate. The percentage of alerts that reach Tier 2 or beyond. A healthy rate is under 15%. Above 30% means Tier 1 is consistently failing to respond — investigate whether it is a coverage gap, alert fatigue, or tooling friction.
False escalations. Alerts that escalated because they were not acknowledged but turned out to be non-issues. These train backup and lead responders to deprioritize escalated alerts, which defeats the purpose.
and incident postmortems. The escalation policy should evolve as the team, the services, and the alert volume change. Google's with severity-based routing and multi-channel escalation in a few minutes.
Originally published on DevHelm.
↗ Original-Artikel auf dev.to lesenVollständiger Original-BerichtAusführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.
SOCIAL SHARE CARD GENERATOR