An enterprise FAQ multi-agent solution on Azure repeatedly pays for full model completions on near-identical user questions such as “How do I reset my VPN password?” and “What’s the process to reset VPN credentials?” Specialists and the orchestrator share the same grounding corpus. The team wants to cut token cost and latency for semantically equivalent questions while still allowing true novel questions to hit the models. Which caching strategy best fits?

The correct strategy is semantic response caching that embeds incoming questions, matches near-duplicates against prior question-answer pairs, and returns cached answers when similarity exceeds a threshold. Caching strategies for multi-agent solutions include prompt caching, semantic caching, and response caching. Semantic caching is specifically designed to detect near-duplicate natural-language queries via embeddings and reuse prior answers when similarity is high enough, which directly reduces token spend and latency for FAQ-style traffic.

Disabling all caching and always forcing fresh completions is incorrect because it maximizes cost and latency with no quality benefit for repeated semantic intents.

Caching only HTTP status codes from tool calls and never caching language model outputs is incorrect because tool status-code caching does not address repeated generative answers for paraphrased questions.

Using prompt caching exclusively for system prompts and never for user-question similarity is incorrect because prompt caching of system or static prefixes helps reuse shared prompt tokens but does not match paraphrased user questions to prior completions; it solves a different reuse problem.

Exam focus: distinguish semantic caching (query similarity) from prompt-prefix caching and raw response keys.