A platform engineering team is building a resilient Claude wrapper that must handle overloaded (529) responses gracefully without abandoning in-flight user requests. The system serves 10,000 concurrent users. During a 529 wave, the naive implementation attempts to queue all retries in memory, but the queue grows unbounded during sustained overload events and causes OOM crashes. What is the correct resilience architecture?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you need a bounded retry queue that rejects overflow with 503 + Retry-After, persists queue state, and signals back-pressure upstream. That keeps overload contained instead of letting retries eat memory or threads.
Full Explanation
Resilient queue design for overload scenarios requires: (1) bounded queue capacity to prevent OOM, (2) graceful degradation (503 + Retry-After) when the queue is full rather than accepting requests that can't be served, (3) durable persistence so in-flight requests survive server restarts during extended incidents, and (4) back-pressure propagation to upstream systems so load balancers stop sending traffic. This is a complete resilience pattern. The drop-all-requests approach is user-hostile and defeats the purpose of building resilience. The increase-server-memory-to-64GB approach is operational whack-a-mole — it increases headroom but doesn't bound the problem, and extended 529 waves can still exhaust any finite memory. The synchronous-blocking-retries approach holds threads during backoff delays, limiting throughput to the number of available threads — this fails at 10,000 concurrent users where thread exhaustion would occur before memory exhaustion.