A public-health chatbot sees many near-identical outbreak FAQ questions that still trigger fresh foundation-model calls. Which optimization best reduces unnecessary invocations?
Select an answer to reveal the explanation.
Short Explanation
If ten people ask nearly the same outbreak FAQ, you should not buy ten full model rides. Semantic caching recognizes “close enough” questions and reuses a good prior answer.
Full Explanation
Public-health outbreak FAQs are highly repetitive: many residents ask near-identical questions about symptoms, isolation, testing sites, or boil notices. Semantic caching (or prompt caching with similarity matching) reuses prior completions when a new question is close enough to a stored one, cutting unnecessary foundation-model invocations while keeping answers consistent. For municipal chatbots under load, that reuse is a primary Domain 4 efficiency control and also reduces variance that can confuse residents hearing slightly different wordings of the same guidance.
Increasing temperature so answers vary even when questions match fights consistency and defeats cacheability—exactly wrong for standardized outbreak guidance. Deleting prior answers so every resident forces a brand-new completion maximizes spend. Translating every FAQ into five languages before deciding whether to call the model adds preprocessing cost unrelated to detecting duplicates and does not itself reduce FM invocations for matching English (or primary-language) repeats.
Exam caveat: Distinguish semantic/similarity cache reuse from temperature tricks and cache deletion; efficiency answers emphasize fewer redundant completions. Operational check: set a similarity threshold with clinical review for stale or updated guidance invalidation, bypass cache when official guidance version changes, log hit rate and cost saved, and spot-check cached answers against the current outbreak bulletin.