Parks compares vector-only and hybrid search on recreation-policy documents before choosing a production pattern. What should they evaluate?
Select an answer to reveal the explanation.
Short Explanation
Before the parks desk picks a search pattern, they time the lap and check how many right policies came back—latency and recall, not hallway vibes.
Full Explanation
Parks should evaluate latency and recall of vector-only versus hybrid search on recreation-policy documents before choosing a production pattern. Those metrics answer whether adding keyword fusion improves relevant policy retrieval enough to justify any extra cost or complexity for municipal assistants. Shipping whichever pattern was mentioned first in a hallway conversation fails by concept because anecdote is not evidence of search quality or responsiveness on the actual policy corpus. Evaluating only the color of the parks logo and ignoring search metrics fails because branding attributes have no bearing on neighbor quality or query performance. Disabling metrics collection so no latency or recall data exists fails by removing the feedback loop required to make an informed pattern choice. Exam caveat: optimize for the query mix parks actually receives—title-like keyword questions and paraphrase questions may prefer different winners; a single aggregate score can hide that split. Operationally, build a labeled recreation-policy query set, run both patterns under realistic concurrency, record p95 latency and recall@k (and optionally nDCG), pick the pattern that meets civic SLOs, and re-evaluate after major policy corpus updates.