Automated model tests fail when a new fine-tune of the housing-benefits assistant starts leaking sample SSNs in outputs. What pipeline control is working?

Automated model testing that blocks releases leaking sensitive sample data is the pipeline control working when a housing-benefits assistant fine-tune starts emitting sample SSNs. Pre-promotion evaluation should probe for memorization, over-sharing, and policy violations using held-out tests and safety suites. Failing the release when outputs include sensitive sample identifiers prevents a harmful fine-tune from becoming the production civic assistant. That works because fine-tuning can increase memorization and change refusal behavior even when the base model was acceptable—municipal benefits workflows are high-stakes for identity data.

Automated model testing that tracks fine-tune lineage for release rollback fails as the working control here because tracking lineage for rollback is registry versioning that records and reverts after the fact rather than testing outputs to block a release leaking sensitive sample data during pre-promotion testing. Automated model testing that scrubs sample SSNs from training datasets is a data-prep description, not the pipeline control that detects and blocks the leak; it does not substitute for automated tests that fail a release when sample SSNs appear in outputs. Automated model testing that redacts sensitive patterns in live user outputs fails as the working pipeline control because redacting live outputs is production filtering that mitigates after deployment instead of stopping the release during automated testing before residents ever see the output.

Exam caveat: “sample” SSNs in eval sets must be synthetic or carefully vaulted; otherwise the test harness itself becomes a sensitive data store. Operational check: gate housing-benefits promotions on automated leakage and privacy evals, quarantine failed fine-tunes, review training-data provenance for identity fields, and re-test after each remediation before any production traffic sees the new weights.