Skip to main content

Incidents

Everything that fires lands here as an incident with a number (numbered per workspace, starting at 1), a severity, the host and the entity it is about, the measured value against its threshold, and a timeline. Incidents born from events rather than thresholds carry the reason the source itself gave (a failed backup task quotes the task's own error text), and that same sentence travels to Telegram, Slack, Teams and Jira.

The lifecycle​

An incident opens when a rule's condition holds for its configured time. From there:

  • Acknowledge says a human took it. The incident moves to the active tab wearing the acknowledger's avatar, and it stops counting as "needs attention" everywhere.
  • Resolve closes it. A resolve by a person (or by an AI agent through the API) holds: while the condition is still true, nothing reopens that incident and no notification is sent for it, and the hold lifts the moment the condition clears. The next time the condition fires after that, a new incident opens with a new number. So a hand resolve is "I know, it will clear" rather than a mute that silently dies: you hear about the same condition again only once it has gone away and come back.
  • Auto-resolve happens when the condition clears (for most rules within a minute of the metric recovering; a source that stops reporting entirely closes after a longer stale window). An auto-resolved incident that fires again within the hour reopens the same incident rather than minting a twin, so flapping does not spam numbers; a condition that returns after more than an hour is a new incident, because an outage that ended at breakfast and one that starts at dinner are two stories, not one twelve-hour incident. Event-born incidents close on quiet: a file that stops changing, a forecast that stops being made, a task type that succeeds again. Switching a service off closes its open incidents immediately. An incident only ever opens from a rule that is actually firing, with the value measured at that moment, never from a rule whose hold time is still counting.

Acknowledge and resolve work from several places, all equivalent: the console buttons, the inline buttons under the Telegram message and on the browser push notification, the take on and resolve buttons in the alert e-mail (signed links bound to the recipient, valid 24 hours, acting only after a confirmation click), the buttons on the Slack card, the /wakora ack N slash command, the phone call's keypad (1 to acknowledge, 2 to resolve) when voice alerting is on, and the API. Whoever acted is on the timeline together with the channel it came from; an action performed by an AI agent through MCP wears an "AI" badge instead of a person's avatar. With Jira connected, a create issue button files the incident as a ticket, see Integrations.

Reading a row​

A row expands into the full story: the timeline (fired, notified, escalation steps and phone calls with who was dialled and how the call ended, acknowledged, reopened, resolved, and who did what), the runbook attached to the rule, the host's log lines from fifteen minutes before the incident started to fifteen minutes after (an error-storm incident shows the errors only; a side with nothing to show says so), and jumps: the host page pinned to that time, the trace when the incident is about a slow site, the Logs module with the filter prefilled.

Suppressed incidents (born inside a maintenance window) arrive pre-acknowledged with a note, so the history is complete but nobody was paged.

Noise discipline​

  • One digest per storm, not a message per incident: notifications queue briefly and go out once the burst has settled (the first message can therefore lag the incident by a minute or so; phone escalation does not wait on this).
  • A package upgrade is reported as a single deploy audit, however long the upgrade ran, not as a trickle of per-package incidents.
  • A reopen renotifies at most once per half hour, and digests label reopens honestly instead of calling them new.
  • Related signals supersede each other (an expired certificate closes its own "expiring" warning; a dead domain mutes that site's certificate incidents), and a down host inhibits its other incidents: you get the cause, not the echo.

Publishing to a status page​

An expanded incident offers post to status page when you run one: it prefills a public update (state and title) which you edit and publish. See Status pages.