After enabling a new constituent-email agent, the mayor's office needs to watch for rising unsafe outputs. Which practice should they follow?
Select an answer to reveal the explanation.
Short Explanation
You can't rely on safety benchmark scores alone; you'll miss rising unsafe outputs in real constituent traffic. Keep an eye on safety signal rates after the constituent-email agent goes live so spikes get caught early. Safety red-team pass rates won't tell you the replies went off the rails, and neither will safety disclaimer rates.
Full Explanation
After enabling a new constituent-email agent, the mayor’s office should monitor safety signal rates. Launch is not the end of risk. New agents can drift, hit novel prompts, or degrade as content mixes change. Continuous safety-signal monitoring after enablement catches rising unsafe outputs early—before volume and political exposure grow. That post-launch practice is what the stem asks for, and it belongs in the same operational cadence as latency and delivery health for civic email assistants. Monitoring safety benchmark scores after enabling the new constituent-email agent fails because scores on a fixed benchmark set do not reflect live safety signal rates from real constituent traffic and cannot catch rising unsafe outputs as prompts drift. Monitoring safety red-team pass rates after enabling the new constituent-email agent fails because pass rates on a fixed adversarial set do not reflect live safety signal rates from real constituent traffic and cannot detect rising harmful replies. Monitoring safety disclaimer rates after enabling the new constituent-email agent fails because disclaimer presence measures formatting compliance, not safety, and an email can include a disclaimer yet still be unsafe. Exam caveat: post-enable monitoring for civic email agents must include safety signal rates—not safety benchmark scores alone, safety red-team pass rates alone, or safety disclaimer rates. Operational check: baseline safety signals at enablement, dashboard rates daily for the constituent-email agent, alert on spikes, keep filters on, and review flagged samples with the mayor’s office communications lead.
Prefer to study offline? Printable PDF study book + test engine ($7.50, unofficial). These questions stay free here.