A parks vision model A/B test runs in production without documented experimental change controls. What should the auditor recommend?

A parks vision model A/B test running on live cameras changes what the municipality sees and how staff may respond to park conditions. The auditor should recommend experimental and canary change governance so production A/B tests of civic AI are controlled, time-bounded, and documented—including which variant served which cameras, success metrics, stop criteria, and rollback. That preserves accountability to residents, enables incident analysis, and prevents silent, indefinite experimentation on public infrastructure.

Tracking A/B test traffic splits in production monitoring dashboards and retaining model performance metrics for routine operational review fails because dashboards and metrics are not an auditable change record; they do not establish a documented variant assignment, duration limit, or rollback path for a live civic camera system. Running the test for a fixed one-month window and documenting only the selected version afterward fails because it still omits the real-time record of which variant served which cameras, the success metrics, and the stop criteria—exactly the traceability a controlled experimental change needs while it is running, not just at its conclusion. Managing A/B model variants through standard source code version control and documenting the promoted model in release documentation fails because version control and release documentation are not experimental change governance for live cameras; they do not establish which variant served which cameras, success metrics, stop criteria, or an operational rollback path while the test is running.

Exam caveat: experimental or canary language does not remove change obligations—AAIA expects controlled experimentation still under change governance. Operational check: confirm a documented experimental change for the parks A/B test with variant assignment, duration limits, monitoring ownership, and a clear exit or promote path—not an open-ended, unlogged split on live cameras.