A translation platform processes documents in 40 languages and uses few-shot examples in the system prompt to demonstrate translation style and tone. They currently include 5 examples per language (200 examples × average 150 tokens = 30,000 tokens) in the system prompt for every request regardless of target language. With prompt caching, this entire prompt is cached. However, the cache is invalidated whenever any example changes (e.g., updating French examples invalidates the cache for Spanish requests too). What caching architecture minimizes cache invalidation frequency?

The problem is that a monolithic prompt creates a single cache dependency — any change invalidates the entire cache for all languages. The optimal architecture separates stable content (universal instructions) from variable content (language-specific examples) and caches only the stable prefix. The universal 2,000-token prefix is cached and almost never changes. The 750-token language-specific block varies per request but is small, so its cost on cache misses is low. When French examples are updated, only French-language requests pay cache_write cost for the language block — the universal prefix remains cached for all requests. The single large prompt with all 200 examples is the current antipattern, because it accepts full cache invalidation whenever any example changes. The 40 separate cached prompts approach is technically valid but operationally complex, and cache storage is maintained per-organization so 40 concurrent cache entries compete for cache capacity. The zero-shot approach eliminates the cache issue but at the cost of translation quality — few-shot examples materially improve style consistency.