A transit authority updates bus-route PDFs, then riders report wrong answers from the Foundry RAG assistant. What should operations monitor?

When transit-route PDFs change and riders suddenly get wrong answers, operations must monitor grounding quality metrics that show whether the Foundry RAG assistant still cites the updated route knowledge. The practice works because RAG correctness is not proved by a successful upload alone—retrieval must return the new chunks, citations must point at current documents, and answer faithfulness checks must detect residual references to obsolete detours or schedules. Grounding metrics close the loop between content publishing and rider-facing truth for the transit authority.

Watching indexer job throughput metrics that track daily re-embedding of route PDFs fails because throughput only measures how many PDFs were re-embedded, not whether the new chunks are actually retrieved and cited; a high daily count can still skip or misroute specific route PDFs, so it hides the per-document grounding regression riders are experiencing. Watching chatbot endpoint latency percentiles that track response times for rider queries fails because latency percentiles measure how quickly answers arrive, not whether those answers cite the updated route PDFs; a fast response can still be grounded in pre-update route PDFs, so response time says nothing about whether retrieved passages reflect the newly published schedule. Tracking daily query volume trends that track RAG assistant usage across bus routes fails because usage trends show how many riders ask questions, not whether those answers cite the current route knowledge; volume can stay healthy or even grow while grounding quietly regresses.

Exam caveat: aggregate “answers generated” counters can look healthy while grounding has silently regressed; prefer citation hit rates, retrieval recall on known route IDs, and graded faithfulness samples. Operational check: after each route-book publish, run a canary question set for changed routes, compare cited document versions/timestamps, alert when grounding scores drop, and block full traffic cutover until the updated corpus is confirmed in the index.