An architect is designing a prompt caching strategy for a system with a 3,000-token system prompt. The system processes 5,000 requests per day but has a quiet overnight period of 6 hours with no traffic. What is the implication for cache hit rates?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — Anthropic's prompt cache expires after about 5 minutes of inactivity, but each hit resets that timer. With 5,000 requests/day, your cache stays warm during busy periods, yet any gap longer than 5 minutes — including the 6-hour overnight lull — causes a miss and full-price input tokens.
Full Explanation
Anthropic's prompt caching has a default cache lifetime of approximately 5 minutes of inactivity, and each cache hit resets that timer. For a system processing 5,000 requests/day, peak traffic keeps the cache warm because requests arrive faster than the timer expires. During the 6-hour overnight gap, the cache expires, so the first request after the gap is a cache miss at full input token prices and rebuilds the cache. Architects must factor in cache miss rates during traffic lulls when calculating expected cost savings. The 24-hour expiry claim is wrong because the default lifetime is about 5 minutes of inactivity, so overnight gaps do affect hit rates. The permanent-cache claim is wrong because cached prefixes expire automatically and must be kept warm by ongoing traffic. The 1-hour expiry claim is wrong because the documented default lifetime is about 5 minutes, so gaps longer than 5 minutes can cause misses.