A 311 product team wants safety assurance before each resident-assistant release. Which practice should be part of the release gate?
Select an answer to reveal the explanation.
Short Explanation
Before the new 311 bot goes live, red-team it on purpose—like a fire drill for bad prompts. Automated adversarial suites catch safety holes before residents do.
Full Explanation
A 311 product team needs safety assurance before each resident-assistant release. Running automated adversarial test suites against the assistant before release should be part of the gate: misuse, injection-style inputs, and policy-bypass attempts are exercised deliberately so failures are found in CI rather than by residents. Adversarial evaluation is a core GenAI safety practice for civic deployments.
Shipping every build to all residents with no safety tests because schedules are tight fails by concept: schedule pressure does not remove public harm risk. Testing only happy-path FAQ questions and never attempting misuse or injection cases fails coverage—safety bugs hide in adversarial inputs. Postponing all safety evaluation until after a major public incident occurs fails preventative governance and treats residents as unpaid red-teamers.
Exam caveat: adversarial suites must be maintained as prompts and policies evolve; a one-time test at launch is not a release gate. Suites should include fair-housing, toxicity, self-harm, and injection cases appropriate to 311—not only generic insults. Fail-the-build thresholds need owners, and false positives should be tuned without quietly disabling the gate.
Operational check: integrate an automated adversarial suite into the 311 assistant pipeline, introduce a deliberate safety regression in a branch, confirm the release gate fails, fix and re-run to green, and retain artifacts showing which cases were executed for audit.