A complex research assistant uses a 6,000-token system prompt that includes detailed domain expertise, output formatting rules, and 12 few-shot examples. The team wants to add more few-shot examples but is approaching the practical limit before context compression hurts performance. Where should cache_control breakpoints be placed to maximize savings on repeated calls?
Select an answer to reveal the explanation.
Short Explanation
You want a two-tier cache hierarchy: put one breakpoint after the static domain expertise and formatting rules, and another after the few-shot examples. That way the stable prefix stays cached even when examples change, and you still get full savings when nothing changes.
Full Explanation
A two-tier cache hierarchy is correct because different parts of the system prompt have different update frequencies. Domain expertise and formatting rules rarely change; few-shot examples may be updated periodically. A breakpoint after the static domain expertise and formatting rules caches the deepest stable prefix. A breakpoint after the few-shot examples caches the less-stable but still-repeated examples. When examples are updated, the stable-prefix cache still hits, saving the cost of the stable prefix. When nothing changes, both cache tiers hit. Placing a single breakpoint at the end of the entire system prompt is wrong because it works only when all content is equally stable; it misses the opportunity to recover partial cache savings when the few-shot examples are updated. Placing a breakpoint before each individual few-shot example is wrong because it's impractical—12 breakpoints approaches the API limit on simultaneous cache breakpoints—and the granularity provides little additional savings over the two-tier hierarchy. Placing breakpoints only around the few-shot examples, not the domain expertise, is wrong because domain expertise and formatting rules are also stable and potentially long; excluding them from caching leaves savings on the table.