An open-data portal’s RAG pipeline begins producing empty embeddings and failed skillset runs. What should operators alert on?

Operators should alert on data ingestion quality for the open-data portal’s civic search and RAG pipeline when empty embeddings and failed skillset runs appear. The practice works because RAG and search depend on successful document cracking, enrichment, embedding, and indexing; empty vectors and skillset failures mean content never becomes retrievable, so the generative layer will miss or hallucinate open-data answers. Ingestion quality alerts catch pipeline breaks at the source before residents notice blank or invented results. Tracking vector index storage size for the civic search and RAG pipeline fails because storage volume can look normal even when embeddings are empty or skillsets fail—capacity metrics do not confirm that records produced usable vectors or completed enrichment. Tracking ingestion job success rate for the civic search and RAG pipeline fails because the rate can stay high while successful jobs still contain documents with empty embeddings or failed enrichment steps—job-level success counts do not confirm that records produced usable vectors or completed enrichment. Monitoring search query latency for the civic search and RAG pipeline fails because query speed tells operators nothing about whether embeddings are empty or skillsets are erroring on specific documents—latency reflects user-facing search performance, not whether content cracked, enriched, and indexed successfully. Exam caveat: partial success (some documents indexed, others empty) can hide behind a high ingestion job success rate for the civic search and RAG pipeline—alert on rates of empty embeddings and per-skill failures, not only job-level success counts. Operational check: monitor indexer/skillset error logs, track embedding emptiness ratio, quarantine failing document types, reprocess after fixes, and validate a sample of newly published open-data records are searchable before marking the pipeline healthy.