A SaaS platform allows users to submit analysis requests to Claude. During peak hours, the platform receives 1,000 requests/minute but the API rate limit is 500 requests/minute. An architect must design a queue that prevents rate limit errors while maintaining fairness across users. Which approach is most appropriate?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because a token bucket queue handles the mechanics of smoothing request bursts to stay within rate limits, per-user limiting prevents one user from monopolizing capacity, exponential backoff handles transient 429s from the API gracefully, and priority lanes allow the platform to monetize higher service tiers. This is a production-grade queue architecture.
Full explanation below image
Full Explanation
B is correct because a token bucket queue handles the mechanics of smoothing request bursts to stay within rate limits, per-user limiting prevents one user from monopolizing capacity, exponential backoff handles transient 429s from the API gracefully, and priority lanes allow the platform to monetize higher service tiers. This is a production-grade queue architecture. A is wrong because FIFO without per-user limits allows one user to submit 999 requests and starve all other users — it is unfair and creates a denial-of-service vector. C is wrong because immediately dropping excess requests fails the user experience — users lose work during peak periods, creating churn; queuing is preferable to rejection. D is wrong because semantic caching can supplement but not replace a queue — many analysis requests are unique and cannot be cache-hit; and the cache hit rate on user-specific analyses is typically very low.