A codebase assistant uses prompt caching to reduce costs. The system prompt is 4,000 tokens and is shared across all users. User-specific conversation history follows the system prompt. After deploying cache_control breakpoints, the team discovers cache hit rates are below 10%. What is the most likely root cause?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — a is correct because prompt caching works by hashing the token sequence up to and including the cache_control breakpoint. If the breakpoint is placed after user-specific content, each user's request produces a different hash because their conversation history differs.
Full explanation below image
Full Explanation
A is correct because prompt caching works by hashing the token sequence up to and including the cache_control breakpoint. If the breakpoint is placed after user-specific content, each user's request produces a different hash because their conversation history differs. The cache never hits. The breakpoint must be placed at the end of the longest stable prefix — in this case, immediately after the shared system prompt — so every user's request shares the same cached prefix. B is wrong because the minimum token threshold for prompt caching is 1,024 tokens for claude-3-5 models; a 4,000-token system prompt meets this threshold. C is wrong because prompt caching invalidation is based on token content changes, not a time-based frequency limit; there is no documented per-minute invalidation limit. D is wrong because claude-3-5-sonnet fully supports prompt caching for system prompts; this is one of the primary documented use cases.