County records plans to embed open-records PDFs that sometimes contain driver’s license numbers. Which control belongs in the RAG ingestion pipeline?
Select an answer to reveal the explanation.
Short Explanation
You can't un-ring a bell once it is in the index. Redact license numbers before those PDFs become embeddings. Encrypting, tokenizing, or pseudonymizing them before you embed still leaves recoverable values you'll have to protect, so they don't wipe the numbers out like redaction does.
Full Explanation
In a RAG ingestion pipeline for county open-records PDFs, redacting driver's license numbers before creating embeddings is the control that stops sensitive identifiers from entering the vector store. Once DLNs are embedded and indexed, similarity search and answer synthesis can resurface them even when the original PDF is later restricted, because the corpus already contains the sensitive tokens or close reconstructions. Pre-embedding redaction aligns RAG build with municipal records and privacy practice: public or semi-public retrieval corpora should not become a backdoor identity warehouse. The pipeline should scan and strip restricted identifiers at ingest, then embed only the sanitized text so retrieval cannot answer identity fishing questions from the index itself. Encrypting driver's license numbers before creating embeddings from the PDFs fails because encryption is reversible and preserves the identifiers in decryptable form; the ciphertext can be decrypted back to the original DLNs, so the retrieval corpus still holds recoverable identity data instead of removing it. Tokenizing driver's license numbers before creating embeddings from the PDFs fails because tokenization substitutes vaulted tokens that map back to the originals; the tokens remain linkable and detokenizable, so the sensitive numbers stay recoverable in the pipeline rather than being irreversibly removed. Pseudonymizing driver's license numbers before creating embeddings from the PDFs fails because pseudonymization replaces them with consistent surrogates that remain re-identifiable with additional information; the surrogates preserve linkability and privacy risk for an open-records corpus instead of eliminating it like redaction does. Exam caveat: Redaction must happen before the text is embedded, not after it is already in the index. Encrypting before embedding still leaves decryptable values that can be recovered, so it does not remove secrets the way redaction does. Operational check: Add DLN (and similar ID) detection to the county RAG ingest job so it scrubs the text that is actually embedded, not after embeddings are already created; refuse or quarantine documents that fail sanitization; rebuild any previously indexed collections that predate redaction; spot-check retrieval with identity-seeking probes to confirm numbers do not return; and document the redaction step in the pipeline runbook for auditors reviewing the open-records assistant.
Prefer to study offline? Printable PDF study book + test engine ($7.50, unofficial). These questions stay free here.