An organization deploys Azure API Management (APIM) as an AI gateway in front of Azure OpenAI Service. The security team wants to enforce that no single application can consume more than 60,000 tokens per minute (TPM) across all Azure OpenAI calls routed through APIM. Which APIM policy implements this token-based rate limiting?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you need the 'azure-openai-token-limit' policy with a tokens-per-minute cap and a counter key tied to each app's subscription key. It counts prompt and completion tokens in Azure OpenAI responses, enforces the 60,000 TPM limit per application, and returns HTTP 429 when exceeded.
Full Explanation
The correct policy is the 'azure-openai-token-limit' policy configured with a token-per-minute limit and a counter key based on the calling application's subscription key. Azure API Management includes this purpose-built policy to count prompt and completion tokens returned in Azure OpenAI responses and enforce a tokens-per-minute (TPM) limit. By setting the counter-key to the calling subscription key, which identifies each application, the policy enforces per-application TPM limits and returns an HTTP 429 response when the limit is exceeded. This prevents any single application from monopolizing Azure OpenAI capacity. The 'rate-limit-by-key' policy counting requests per minute is wrong because it counts API requests (calls per minute), not tokens—an application could stay under the request rate limit while consuming far more tokens per call. The 'quota-by-key' policy with a weekly quota on API calls is wrong because it sets a total number of calls or bandwidth over a longer time window (weeks/months), not a real-time per-minute token rate limit. The 'ip-filter' policy combined with a 'rate-limit' policy is wrong because 'ip-filter' controls access by source IP, and combining it with a request-based 'rate-limit' still doesn't account for token consumption.