Evening peaks spike inference cost for repeated trash-schedule questions. What cost-aware tactic still preserves availability?
Select an answer to reveal the explanation.
Short Explanation
If everyone asks the same trash day, don’t burn a fresh inference each time—cache the frequent answers. Shutting down at peak or recomputing everything from scratch wastes money or strands residents.
Full Explanation
Evening peaks that spike inference cost for repeated trash-schedule questions are best handled by caching frequent answers while preserving availability. Trash schedules are relatively stable civic facts; serving cached responses (with appropriate TTLs and invalidation on schedule changes) cuts repeated model invocations without taking the agent offline. Cost-aware architecture for municipal FAQ-like intents should prefer deterministic or cached replies for high-duplication queries rather than burning GPU tokens on identical content.
Shutting down the agent every evening so nobody can ask fails conceptually because evening is often when residents need the service; availability must be preserved. Recomputing every identical schedule answer from scratch with no cache fails economically and operationally under peak duplication. Answering only during weekday mornings when queues are empty fails because it denies service at peak demand rather than engineering for it.
Exam caveat: caches must invalidate when public works updates schedules or holidays; stale trash guidance creates resident distrust. Also distinguish cacheable FAQs from personalized case actions that must not be cached across residents.
Operational check: identify top repeated intents, implement response or retrieval caching with TTL/event invalidation, load-test evening peaks, and verify GPU spend drops while p95 availability remains within SLO.