A production system makes 1,000 API calls per day where each call uses an identical 2,000-token system prompt followed by unique user messages averaging 500 tokens. The architect wants to reduce input token costs. Which feature should be applied?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — prompt caching is the correct solution. By adding cache_control: {type: 'ephemeral'} to the system prompt content block, Anthropic caches the computed KV state of the prompt.
Full explanation below image
Full Explanation
Prompt caching is the correct solution. By adding cache_control: {type: 'ephemeral'} to the system prompt content block, Anthropic caches the computed KV state of the prompt. Subsequent calls within the cache TTL (approximately 5 minutes, reset on each cache hit) are charged cache read prices — significantly lower than full input token pricing. With 1,000 calls/day sharing a 2,000-token system prompt, the savings are substantial. Option B (compression) reduces token count slightly but at the cost of instruction quality; savings are modest and the tradeoff is poor. Option C (moving to user message) doesn't reduce costs — the tokens are still processed as input regardless of their message role. Option D (splitting across calls) is architecturally incorrect — each call needs all instructions to operate correctly, and splitting breaks the instruction context.