A platform engineering team is building a resilient Claude wrapper that must handle overloaded (529) responses gracefully without abandoning in-flight user requests. The system serves 10,000 concurrent users. During a 529 wave, the naive implementation attempts to queue all retries in memory, but the queue grows unbounded during sustained overload events and causes OOM crashes. What is the correct resilience architecture?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — resilient queue design for overload scenarios requires: (1) bounded queue capacity to prevent OOM, (2) graceful degradation (503 + Retry-After) when the queue is full rather than accepting requests that can't be served, (3) durable persistence so in-flight requests survive server restarts during extended incidents, and (4) back-pressure propagation to upstream systems so load balancers stop sending traffic. This is a complete resilience pattern.
Full explanation below image
Full Explanation
Resilient queue design for overload scenarios requires: (1) bounded queue capacity to prevent OOM, (2) graceful degradation (503 + Retry-After) when the queue is full rather than accepting requests that can't be served, (3) durable persistence so in-flight requests survive server restarts during extended incidents, and (4) back-pressure propagation to upstream systems so load balancers stop sending traffic. This is a complete resilience pattern. Option A (more memory) is operational whack-a-mole — it increases headroom but doesn't bound the problem, and extended 529 waves can still exhaust any finite memory. Option C (drop requests) is user-hostile and defeats the purpose of building resilience. Option D (synchronous blocking retries) holds threads during backoff delays, limiting throughput to the number of available threads — this fails at 10,000 concurrent users where thread exhaustion would occur before memory exhaustion.