A city open-data pipeline continuously embeds new documents into a RAG index with no malware or injection scanning on ingest. What job control should be added?
Select an answer to reveal the explanation.
Short Explanation
You need a door check before new documents enter the index: scan for malware and hostile content before embedding. That gate helps keep poisoned content out; disabling hashes, allowing public direct submissions, or rebuilding only when storage is full does not.
Full Explanation
A city open-data pipeline that continuously embeds new documents into a RAG index without malware or injection scanning needs malware and content-injection scanning before continuous embed jobs. Continuous embedding is a high-frequency trust boundary: hostile documents or prompt-injection-laden text can compromise workers or poison retrieval that later answers residents’ questions. Scanning at ingest, before chunking and embedding, stops unsafe content before it is indexed citywide. Disabling ingest hashes to make jobs faster fails because integrity checks help detect unexpected mutation and support forensics. Allowing public contributors to submit documents directly to the embedding queue fails because it removes attribution and access control from the poisoning path. Rebuilding the index only when storage is full fails because capacity triggers are unrelated to content safety. Exam caveat: continuous RAG ingest without a door check is a primary poisoning and worker-compromise scenario; speed and anonymous submission are distractors. Practical check: make malware and hostile-content scanning a hard gate before embedding, quarantine failures, retain content hashes, prevent public direct queue submissions, and sample the live index for unscanned documents so they can be reprocessed or purged.