An incident has three states:
| State | Meaning | Escalation |
|---|---|---|
| Triggered | The monitor is down and nobody has taken it yet | Running |
| Acknowledged | Someone has taken it | Stopped |
| Resolved | The problem is over | Stopped |
The alert call#
The call starts with the alert, for example:
"This is an alert from Acme Labs. Checkout API is down since 2:14 AM. Reason: status 503."
Then the menu: "Press 1 to acknowledge this alert. Press 2 to escalate it to the next person." In Hindi, the same call is spoken in Hindi.
| Key | Result |
|---|---|
| 1 | Acknowledged. Escalation stops for everyone. You hear "Acknowledged. Thank you." |
| 2 | The next step of the policy is paged now, without waiting |
| No key | The alert and menu are repeated once. If still nothing, the call ends with "We will keep trying your team" and escalation continues |
Between 21:00 and 09:00 IST the call may be a plain spoken alert without the menu. See Night Mode.
Other ways to acknowledge#
- Email. Alert emails have an Acknowledge button. Opening the link shows a page with one button; pressing it acknowledges. (Opening the link alone never acknowledges, so email scanners can't do it by accident.)
- Dashboard. Open the incident on the Incidents page and press Acknowledge.
Acknowledging an incident that is already acknowledged or resolved does nothing.
Resolving#
- Automatically. When the check passes again, the heartbeat pings again, or your monitoring tool sends a resolved alert, the incident resolves itself.
- Manually. Press Resolve on the incident.
When an incident resolves, any calls still waiting are cancelled, and everyone who was paged gets one email: "Monitor is back up".
The incident timeline#
Each incident keeps a timeline of what happened, for example:
| Time | Event |
|---|---|
| 02:14 | Incident opened: status 503 |
| 02:14 | Step 1: paging Asha |
| 02:14 | Calling Asha (+919876••3210) |
| 02:19 | Step 2: paging Ravi |
| 02:20 | Acknowledged by Ravi via phone (pressed 1) |
| 02:41 | Check passed again (resolved by auto) |
Use it for post-incident reviews: how long until someone acknowledged, and how long the outage lasted.
FAQ#
If I acknowledge, will I get called again for the same outage? No. Escalation stops until the incident resolves. A new outage opens a new incident.
Does acknowledging fix the monitor? No. It only stops the calls. The incident resolves when the monitor recovers or you press Resolve.
Can someone acknowledge from a phone that isn't on the call? Yes, from the email link or the Incidents page in the portal.