A factory runbook repaired the failed host, yet the CloudWatch alarm is still ALARM. The metric has not come back inside its threshold. Why has the alarm not cleared?
Select an answer to reveal the explanation.
Short Explanation
The alarm is a gauge, not a receipt for the repair you just finished. You fixed one host, and the alarm stays ALARM until the metric itself is back inside the threshold for its evaluations. Recreating the alarm does not substitute for that metric.
Full Explanation
CloudWatch sets an alarm to ALARM when the metric breaches, and it returns the alarm to OK only after the datapoints meet the OK condition. Finishing a runbook is not one of those datapoints. If the metric has not come back inside the threshold, the state remains ALARM even though the host looks repaired. A runbook that repairs one host can leave a fleet metric in breach when other hosts, or the latest periods, are still outside the threshold. Deleting and recreating the alarm starts a new evaluation and can drop the history, and it does not prove the metric is healthy. The alarm is not latched until a person clears it; the service recalculates state from the metric on each evaluation. An unconfirmed email subscription can block a notification, and it does not freeze the state shown on the alarm. Caveat: missing-data treatment can move the alarm to insufficient data or keep the prior state, so confirm the metric is still publishing. Operational check: compare the latest datapoints with the threshold, and treat the incident as cleared only after those datapoints are inside it.