A developer wants to add prompt caching to a RAG pipeline where the system prompt contains static instructions (2,000 tokens), followed by dynamically retrieved documents that change per query (average 8,000 tokens), followed by the user query (average 200 tokens). Where should cache_control breakpoints be placed to maximize cache hit rate?

The correct approach is to place a single cache_control breakpoint after the static system prompt, at the 2,000-token boundary, and not cache the dynamic documents. Cache_control breakpoints should be placed at the end of stable, shared prefixes. The 2,000-token static system prompt is identical across all queries, so caching it saves 2,000 tokens of processing on every call. The dynamically retrieved documents change per query, so caching them produces near-zero hit rates and wastes cache write costs. The optimal single breakpoint is therefore at the system-prompt boundary.

The option that places breakpoints after the static system prompt and after the dynamic documents is wrong because caching the dynamic documents adds cache write overhead, billed at input token cost, on every miss with negligible savings unless the same exact document set is retrieved for many identical queries, which is rare in a RAG system.

The option that places breakpoints at the beginning of the system prompt, after static instructions, and after each retrieved document chunk is wrong because the API supports a limited number of cache breakpoints; placing them at every chunk boundary is not supported and adds complexity without benefit for content that changes per query.

The option that says not to use cache_control because prompt caching is only beneficial when the entire input is static is wrong because partial caching of stable prefixes, even a 2,000-token static system prompt, produces real savings at scale; the static/dynamic distinction is what determines where to break, not an all-or-nothing rule.