A county must release a benefits-eligibility agent but leadership requires evidence it behaves safely. What should the team enable before release?

Before releasing a benefits-eligibility agent, the county should enable evaluators and safety evaluations so leadership has evidence the civic agent behaves safely under realistic prompts. The practice works because offline/online evals measure correctness, harmful-output rates, refusal quality, and policy adherence on curated municipal scenarios before production traffic begins. Safety evaluations convert “the demo looked fine” into repeatable evidence suitable for risk acceptance on benefits decisions that affect residents. Running harm severity and jailbreak evaluators for the civic agent after production release fails because waiting until after release provides no pre-release evidence for leadership sign-off, leaving residents exposed to unsafe eligibility guidance during initial live traffic. Adding safety filters and blocklist controls for the civic agent prior to production release fails because filters are preventive controls, not evaluative evidence, and they cannot substitute for structured evaluators and safety evaluations that quantify agent behavior for leadership sign-off. Using violence and hate evaluators for the base model prior to production release fails because base-model scores do not test the deployed civic agent with its prompts, tools, and workflows, and generic violence and hate coverage misses civic-specific safety risks such as inaccurate eligibility guidance and mishandling of resident data. Exam caveat: eval suites must include adversarial and edge eligibility questions, not only brochure-style FAQs, or scores will overstate readiness. Operational check: define release gates (safety score thresholds, regression vs last build), store eval reports with the change record, require sign-off from program and security owners, and re-run evals after material prompt, tool, or model changes.