An e-commerce platform uses Lambda to restart degraded ECS tasks when alarms fire. During a partial outage the remediator invoked hundreds of restarts per minute, throttled the ECS API, and worsened the incident. Which controlled remediation design adds safety constraints to automated restarts while preserving recovery?
Select an answer to reveal the explanation.
Short Explanation
You want a calm dispatcher, not a stampede. Queuing with limits and a shutoff switch smooths retries and pauses when trouble spikes, and bulkhead thinking keeps one failure from flooding everything.
Full Explanation
The correct design buffers alarm bursts in SQS to convert hundreds-per-minute spikes into controlled throughput, enforces capped concurrency to bound ECS API call rate and prevent throttling, uses backoff retries to handle throttling without amplifying load, and uses a DynamoDB circuit breaker to count failures and pause restarts during partial outages then resume to preserve recovery. The SNS with reserved concurrency approach limits parallelism and isolates failures in a dead-letter queue but fails burst control because without buffering and backoff delivery still drops events or retries aggressively during throttling. The scheduled polling approach adds batching and backoff to protect the API but fails timely event-driven recovery because minute-interval scans ignore alarm state, add detection delay, and can still launch bulk restarts without concurrency bounds or breaker logic. The Step Functions with approval flag approach adds orchestrated retries and governance but fails autonomous recovery because manual approval blocks remediation at scale during an outage and lacks queued concurrency limits. Exam caveat: separate prevention via concurrency caps, resilience via backoff retries, and harm reduction via circuit breakers since exam stems often omit exactly one mechanism. Practical check: alarm storm test while watching SQS age, Lambda concurrency, ECS throttle errors, and breaker state to confirm throughput stays within quota and pauses then resumes.