A developer platform is building a multi-tenant Claude integration where each tenant has different rate limits negotiated with Anthropic. Tenant A has a 100,000 TPM (tokens per minute) limit, Tenant B has 50,000 TPM, and Tenant C has 200,000 TPM. All tenants share a single Anthropic API key. The platform must enforce per-tenant rate limits before requests reach the Anthropic API to prevent one tenant's traffic from exhausting another's allocation. What is the correct architectural design?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — multi-tenant token rate limiting requires per-tenant token bucket counters backed by a shared cache (Redis is standard for distributed rate limiting). Critically, the Anthropic API provides a POST /v1/messages/count_tokens endpoint that returns the exact token count for a request before it is sent — this enables accurate pre-request budget checks.
Full explanation below image
Full Explanation
Multi-tenant token rate limiting requires per-tenant token bucket counters backed by a shared cache (Redis is standard for distributed rate limiting). Critically, the Anthropic API provides a POST /v1/messages/count_tokens endpoint that returns the exact token count for a request before it is sent — this enables accurate pre-request budget checks. The token bucket refills at the per-tenant TPM rate. Option A (per-tenant API keys) would work but requires negotiating separate keys and limits with Anthropic for each tenant, is operationally complex, and doesn't allow the platform to add its own policy controls. Option C (request-count proxy) is inaccurate — a single request can consume anywhere from 10 to 100,000+ tokens, making request counts a poor proxy for token budgets. Option D (letting Anthropic 429s route back) creates resource waste, poor tenant UX, and doesn't enforce per-tenant limits — all tenants share one key and one global limit.