A system prompt is 4,000 tokens and contains instructions, persona definition, output format rules, and 10 few-shot examples. An architect wants to optimize for both cost and prompt effectiveness. What restructuring approach is recommended?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you want prompt caching on the static system-prompt parts, then you profile which few-shot examples actually drive output quality before trimming anything. Blindly cutting examples or compressing instructions can break behavior you haven't measured yet.
Full Explanation
The architect-level approach is to apply prompt caching to static portions—instructions, persona, examples—making the 4,000-token prompt nearly free on cache hits, then profile which few-shot examples contribute most to output quality via ablation testing. Once low-impact examples are identified, remove them with confidence. Reducing examples to 3 and compressing instructions into bullets is wrong because it can degrade taught behaviors and obscure meaning without a quality baseline. Moving examples to conversation history as simulated prior turns is wrong because system prompts need not contain only behavioral instructions; examples can be valid stable system content, and moving them can lose caching if reconstructed each session, change semantic framing, and make them look like prior dialogue rather than durable guidance. Splitting the system prompt across multiple API calls is wrong because instructions must be present in each call to be effective; splitting breaks coherent instruction context.