IT swaps the embedding model for ordinance RAG and nearest neighbors for key sections change, but no retrieval evaluation is rerun. What change-control gap exists?
Select an answer to reveal the explanation.
Short Explanation
New embedding glasses change which ordinance paragraphs look 'nearest.' If you swap the model and skip re-eval, you're flying blind—treat embedding changes like any other production change.
Full Explanation
Change control and re-evaluation are required when replacing embedding models in municipal RAG. Swapping the embedding model changes the geometry of the vector space, so nearest neighbors for key ordinance sections can shift even when PDFs are unchanged. Skipping retrieval evaluation after the swap means IT cannot know whether critical sections still retrieve correctly for clerk and counsel workflows. Treat embedding-model changes like production model changes: ticket, evaluate, compare golden queries, then promote.
Claiming embedding model swaps never affect nearest-neighbor results fails by concept: different models embed differently; neighbor sets routinely change. Requiring ordinance text to be re-fine-tuned into the LLM whenever embeddings change fails because RAG’s point is to ground on documents; the missing control is retrieval re-eval, not mandatory fine-tunes. Requiring public notice before any municipal embedding update fails as a universal rule and does not substitute for technical re-evaluation of retrieval quality.
Exam caveat: Embedding upgrades are not invisible infrastructure tweaks. Golden-set retrieval metrics must pass before civic cutover.
Operational check: Maintain a golden query set for key ordinance sections, run it on old and new embedding indexes, gate promotion on recall/precision thresholds, document the embedding model version in the index manifest, and roll back if critical neighbors regress.