p95 vector retrieval latency for the 311 agent breaches the published SLO. What is an appropriate vector-database response?
Select an answer to reveal the explanation.
Short Explanation
When the 311 queue clocks out waiting on search, it’s time to wrench on the index—not shrug. Adjusting HNSW (or similar ANN) knobs is how you pull p95 latency back under the city’s SLO.
Full Explanation
When p95 vector retrieval latency for the 311 agent breaches the published SLO, an appropriate vector-database response is to tune HNSW (or equivalent ANN) parameters—efSearch, construction parameters, sharding, replicas—so retrieval returns within the city’s latency budget while monitoring recall. Municipal agents under interactive SLOs need index performance work, not indifference to resident wait times.
Ignoring latency and tracking only embedding model version numbers fails the service objective. Replacing the vector index with an unindexed full-table scan of every PDF typically worsens latency at scale. Adding random sleep delays to make latency “predictable” deliberately degrades resident experience.
Exam caveat: aggressive ANN tuning can trade recall for speed—measure both against the SLO and quality gates. Operational check: load-test p95 retrieval after parameter changes, compare recall@k on a fixed civic query set, and promote only configurations that meet latency without unacceptable quality loss. Pair latency dashboards with recall@k alerts so SLO recovery does not silently erode answer quality. Document the chosen HNSW settings alongside the 311 SLO so future capacity changes do not silently undo latency wins.