A county metrics team notices RAG answers degrading and suspects the ingestion path. Which observability focus best isolates whether the data pipeline is harming GenAI quality?
Select an answer to reveal the explanation.
Short Explanation
If the pantry shelves aren’t getting restocked, the kitchen can’t cook. Watch ingestion failures and reject rates in CloudWatch — those pipeline health signals explain why RAG quality slides even when the FM itself is fine.
Full Explanation
When a county metrics team sees RAG quality degrade and suspects ingestion, observability must focus on the data pipeline that feeds the corpus—not only on model spend. CloudWatch (or equivalent) metrics for ingestion failure rates, validation rejects, and incomplete corpus refreshes isolate whether GenAI answers are suffering because knowledge never arrived, arrived corrupted, or refreshed only partially. Those signals connect civic RAG quality to pipeline health the team can remediate.
Tracking only Bedrock token spend while ignoring ingestion outcomes fails conceptually: cost metrics do not explain missing or stale chunks. Disabling all logging to reduce latency fails because it removes the evidence needed to diagnose resident-facing quality drops. Measuring GPU utilization on unrelated batch training clusters fails because that signal is outside the RAG ingestion and retrieval path that serves county answers.
Exam caveat: GenAI quality problems are frequently data-path problems. Professional designs instrument ingestion and refresh completeness alongside inference metrics.
Operational check: dashboard ingestion failures, validation reject reasons, and refresh completeness SLOs; correlate spikes with RAG evaluation score dips; page owners when incomplete refreshes leave domains unindexed; and confirm token dashboards are secondary, not primary, for this failure mode.