A platform must implement rate limit handling for a high-volume Claude integration processing medical coding (ICD-10) tasks. The system submits 1,200 requests/minute but the API tier allows 1,000 requests/minute and 100K tokens/minute. The engineering team implements token bucket rate limiting on their side with a 60-second refill window. During testing, they observe burst periods where 400 requests arrive within 10 seconds, exhausting the local token bucket and causing 429 errors that trigger unnecessary retries. Which rate limiting architecture prevents this pattern?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — a leaky bucket algorithm is architecturally correct for this problem. The core issue is that the team's token bucket allows bursts — a full bucket at 60-second intervals means 1,000 tokens are available simultaneously, enabling 400 requests to fire in 10 seconds.
Full explanation below image
Full Explanation
A leaky bucket algorithm is architecturally correct for this problem. The core issue is that the team's token bucket allows bursts — a full bucket at 60-second intervals means 1,000 tokens are available simultaneously, enabling 400 requests to fire in 10 seconds. A leaky bucket enforces a constant output rate (1,000/60 ≈ 16.7 requests/second), preventing any burst from exhausting the budget. Requests are queued and drained at a fixed rate — exactly what a smoothing mechanism requires. Option B (sliding window with 10-second granularity) is closer but more complex to implement correctly; it also allows some burst behavior within the 10-second window and requires accurate tracking of all requests in the sliding window across distributed instances. Option C's more frequent refill partially addresses the problem but still allows micro-bursts within each 10-second window. Option D (priority queue) addresses request ordering but not rate limiting — it doesn't prevent the burst from exhausting the rate limit; it just determines which requests get submitted first when capacity is available.