Regression tests fail after a public-health chatbot prompt-template change breaks PII redaction. What CI/CD control caught the issue?

Unit and regression testing that protects AI-related security properties is the CI/CD control that catches a public-health chatbot prompt-template change breaking PII redaction. Prompt and template edits are code-like changes: they can silently disable safety filters, alter system instructions, or weaken redaction patterns that previously stripped identifiers from model outputs. Regression tests encoding those security properties fail when redaction no longer holds, blocking promotion. That works because AI behavior drifts with content changes, and municipal public-health contexts often process sensitive personal information that must not leak through conversational interfaces.

Unit and regression testing that validates prompt firewall bypass detection fails because it checks whether adversarial prompts can evade input guardrails, not for PII redaction failures caused by a prompt-template change. Unit and regression testing that verifies output watermarking for provenance fails because it checks for embedded signals of AI origin, not whether a prompt edit has weakened redaction of sensitive personal information. Unit and regression testing that audits training data licensing compliance fails because it reviews legal permissions for training datasets, not whether a new prompt template breaks redaction and leaks PII.

Exam caveat: tests must cover realistic prompt and tool-output paths; a unit test on a toy string may miss redaction failures that appear only with multi-turn chat or retrieval-augmented context. Operational check: maintain regression fixtures with synthetic PII/PHI-like samples for the public-health bot, require those suites on every prompt-template change, and alert when redaction assertions flake or are skipped so the control cannot quietly atrophy.