Your multi-agent FAQ solution repeatedly pays full inference cost for near-identical policy questions. You need a caching strategy that preserves correctness when policies change. Which design is best?
Select an answer to reveal the explanation.
Short Explanation
You need to layer the caches: use prompt caching where supported, semantic caching for near-duplicate questions, and response caching with time-to-live rules tied to policy versions. That keeps FAQ answers cheap while making sure policy changes bust the cache. Forever caching without invalidation just ships stale policy.
Full Explanation
The best design is layered caching: prompt caching where supported, semantic caching for near-duplicate queries with similarity thresholds, and response caching with time-to-live rules tied to policy document versions for invalidation. Caching strategies for multi-agent solutions include prompt caching, semantic caching, and response caching. Production designs combine them with time-to-live rules and source-version invalidation so near-duplicate FAQ traffic is cheap without serving stale policy.
The forever cache with no invalidation when source policies update is wrong because infinite retention without policy-version invalidation serves outdated answers. The raw GPU kernel cache is wrong because low-level kernel caching does not address LLM prompt/response reuse. Disabling caching entirely because multi-agent systems cannot use caches is wrong because caching is a first-class multi-agent optimization technique.
Exam tip: Always pair semantic cache hits with freshness/invalidation rules.