A content moderation pipeline processes 2 million short social media posts per day using Claude. The posts average 50 tokens each. The system prompt is 800 tokens. An architect wants to reduce inference cost. Which optimization has the greatest leverage?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — a is correct because prompt caching has the highest leverage: the 800-token system prompt represents 94% of the total input tokens for each 50-token post (800/(800+50)). With 2M requests/day, caching eliminates ~1.6B redundant system prompt tokens daily.
Full explanation below image
Full Explanation
A is correct because prompt caching has the highest leverage: the 800-token system prompt represents 94% of the total input tokens for each 50-token post (800/(800+50)). With 2M requests/day, caching eliminates ~1.6B redundant system prompt tokens daily. Cache hit pricing is typically 10x cheaper than standard input pricing, making this a massive cost reduction with no quality impact. B is wrong because model choice affects per-token pricing, but Haiku's lower quality on nuanced moderation decisions may require additional review passes that offset savings; the system prompt caching optimization is independent and larger in magnitude. C is wrong because summarizing 50-token posts (already very short) risks losing the exact content needed for accurate moderation; the savings (20 tokens × 2M = 40M tokens) are far smaller than the system prompt caching savings (800 tokens × 2M = 1.6B tokens). D is wrong because the Anthropic API does not support multi-document batching within a single messages API call in the way described; and system prompt caching already addresses the dominant cost driver.