A system prompt is 4,000 tokens and contains instructions, persona definition, output format rules, and 10 few-shot examples. An architect wants to optimize for both cost and prompt effectiveness. What restructuring approach is recommended?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — the architect-level approach is to first apply prompt caching (making the 4,000-token system prompt nearly free on cache hits), then systematically profile which few-shot examples are actually contributing to output quality through ablation testing. Blindly reducing examples (Option A) risks degrading the behaviors they teach without a quality baseline.
Full explanation below image
Full Explanation
The architect-level approach is to first apply prompt caching (making the 4,000-token system prompt nearly free on cache hits), then systematically profile which few-shot examples are actually contributing to output quality through ablation testing. Blindly reducing examples (Option A) risks degrading the behaviors they teach without a quality baseline. Once profiling identifies low-impact examples, they can be removed with confidence. Option B (moving examples to conversation history) can work but loses the caching benefit on those examples if they must be reconstructed each session, and changes their semantic framing. Option D (splitting across calls) is architecturally incorrect — instructions must be present in each call to be effective; splitting them across calls breaks coherent instruction context.