A housing authority wants to use real case notes to evaluate prompt changes without exposing tenant identities in the evaluation dataset. Which privacy approach is most appropriate?
Select an answer to reveal the explanation.
Short Explanation
Evaluation needs the shape of real casework, not the names on the file folders. Anonymize the notes first—like redacting a case packet before a training exercise—so prompt tests stay useful without outing tenants.
Full Explanation
Anonymizing or de-identifying case notes before they enter GenAI prompt-evaluation datasets is the privacy approach that lets a housing authority use realistic casework to evaluate prompt changes without exposing tenant identities. Evaluation needs linguistic and situational structure—lease disputes, inspection findings, hardship narratives—not names, addresses, case IDs, or other direct identifiers. De-identification (and stronger anonymization where required) produces datasets suitable for offline prompt A/B tests, rubric scoring, and regression suites while reducing re-identification risk if evaluation artifacts leak to contractors or shared drives.
Pasting full tenant dossiers into a shared Slack channel for faster reviewer feedback spreads identifiable case data across chat retention, mobile devices, and third-party SaaS—maximizing exposure for marginal speed. Fine-tuning publicly on named eviction cases maximizes realism at the cost of publishing personal housing matters, which is ethically and legally inappropriate. Disabling evaluation entirely so privacy risk “never has to be managed” abandons quality assurance; privacy and evaluation are both required, managed through minimized datasets—not by skipping measurement.
Exam caveat: privacy-preserving ML/GenAI development commonly pairs real-domain evaluation with de-identified corpora. Do not equate “use production data raw” with good evaluation practice, and do not treat “skip eval” as a valid privacy control.
Operational check: define an identifier inventory for housing case notes, run de-identification before any eval export, spot-check for residual direct identifiers, restrict eval-bucket access to the prompt-engineering role, and re-run the pipeline whenever new note fields are added to production systems.