Who Should Wake Up at 3 a.m.? Three Layers Against On-call Fatigue

Most teams don't have an alerting problem of too few alerts — they have too many. A new on-call engineer jumps at every ping in week one, and by week three has trained themselves to glance at a notification and roll over. That isn't a discipline failure; it's human nature. The fix is to make the mechanism absorb it. Here are three layers worth building.

Layer one: separate "notify" from "page"

Alert fatigue thrives on conflating notification with page. Notifications can queue, wait until morning, and be digested in bulk. A page must cost something — it drags a human out of sleep. Every alert rule should declare an explicit escalation path: notify → page → escalate. A useful test: "if nobody touches this for two hours, do users feel it?" If not, it only earns a notification. For many teams the single biggest win is demoting roughly 60% of pages to notifications. The change is trivial and the relief is immediate.

Layer two: give pages a "was it worth it" score

The second source of fatigue is waking up for alerts nobody owns: "it paged me, it's not my fault, but I still have to look." Attach context and ownership to every page — recent deploys, affected endpoints, a link to the runbook. Go further and compute a cost-benefit ratio per service: over the past 30 days, how many pages did it fire, and how many actually needed human intervention? Services below the threshold get demoted by default, and restoring reliability becomes a to-do for the service owner rather than a nightly startle for whoever holds the pager. Publish the metric, and teams will clean up their own noisy rules — nobody wants their service wearing the "boy who cried wolf" badge.

Layer three: make rest part of the system

Rotation design is routinely ignored. More than a week of continuous on-call, less than 24 hours between rotations, or a junior engineer solo-paging without a backup all measurably raise misjudgment rates. What works: pair a primary with a secondary who absorbs gray-zone night alerts; force a no-page buffer after each rotation to pay down alert debt instead of snapping back into feature work; hold a weekly retro for the "worst wake-up award" and fix the root cause of the least-justified page of the week. Killing one bad alert beats buying a shiny new alerting platform.

Closing thought

The goal of an on-call system isn't "someone is always awake" — it's "someone is awake only when they should be." All three layers share one premise: treat human attention as a scarce resource to be scheduled, not an infinite consumable. Who deserves to be woken at 3 a.m. is ultimately an engineering parameter that needs constant tuning. Tune it well, and your on-call engineer stays sharp on the mornings that actually matter.