Feedback for a benefits-eligibility agent sometimes includes case numbers. What must happen before that feedback enters the training or eval store?
Select an answer to reveal the explanation.
Short Explanation
You can keep feedback useful while protecting residents by redacting case numbers and other identifiers before it enters the training or eval store. Leaving them unmasked, posting unredacted feedback publicly, or skipping capture altogether either risks privacy or loses valuable improvement data.
Full Explanation
A privacy-preserving feedback pipeline redacts municipal identifiers such as case numbers before writing feedback to training or evaluation stores. Resident feedback may also contain names or other linkable fields; storing them in cleartext turns an improvement dataset into a confidential-data store with the wrong controls. Redaction keeps iterative tuning viable while protecting resident confidentiality and meeting civic privacy expectations. Keeping case numbers unmasked so reviewers can retrieve full resident files conflates operational casework access with feedback retention and expands the breach blast radius. Posting unredacted feedback on a public dashboard exposes protected information and undermines trust. Skipping feedback capture whenever a case number appears discards legitimate improvement signals instead of sanitizing them. Exam caveat: useful feedback must still comply with municipal privacy and retention rules; the need to improve the agent does not justify unrestricted retention of personal information. Practical check: submit synthetic feedback containing known case-number patterns, confirm the ingest pipeline redacts or hashes them before writing, verify reviewers cannot recover cleartext from the eval store alone, and document the redaction rules beside the store schema.