An enterprise FAQ multi-agent solution on Azure repeatedly pays for full model completions on near-identical user questions such as “How do I reset my VPN password?” and “What’s the process to reset VPN credentials?” Specialists and the orchestrator share the same grounding corpus. The team wants to cut token cost and latency for semantically equivalent questions while still allowing true novel questions to hit the models. Which caching strategy best fits?
Select an answer to reveal the explanation.
Short Explanation
Near-duplicate questions should not keep burning full multi-agent runs. Semantic caching embeds the new question, finds a sufficiently similar prior question, and serves the stored answer—saving tokens and latency—while still allowing truly new questions through. Turning caching off is the opposite of the cost goal. Caching only tool HTTP codes does not help FAQ answers. Prompt caching of system prefixes is useful but does not catch paraphrase-level user question duplicates. Answer D is the one that targets semantic equivalence of user asks.
Full Explanation
Correct answer: D. Caching strategies for multi-agent solutions include prompt caching, semantic caching, and response caching. Semantic caching is specifically designed to detect near-duplicate natural-language queries via embeddings and reuse prior answers when similarity is high enough, which directly reduces token spend and latency for FAQ-style traffic.
A is incorrect because disabling caching maximizes cost and latency with no quality benefit for repeated semantic intents.
B is incorrect because tool status-code caching does not address repeated generative answers for paraphrased questions.
C is incorrect because prompt caching of system or static prefixes helps reuse shared prompt tokens but does not match paraphrased user questions to prior completions; it solves a different reuse problem.
Exam focus: distinguish semantic caching (query similarity) from prompt-prefix caching and raw response keys.