A city audit plan treats a new permitting GenAI as a standard web app and skips AI-specific risk assessment in scoping. What should the auditor change?

Performing AI-aware scoping and risk assessment when planning municipal GenAI audits works because permitting chatbots introduce model, retrieval, hallucination, bias, data residency, and decision-influence risks that cannot be resolved by deferring AI risk assessment to fieldwork testing of permit approval workflows. Relying on the vendor assurance report to set scope for the permitting model review under-scopes fieldwork: assurance evidence may not address prompt injection, unsafe tool use, or incorrect permit guidance specific to the city’s deployment. AI-aware planning expands the risk universe and maps procedures to those risks before sampling begins.

Deferring AI risk assessment to fieldwork testing of permit approval workflows fails because generative decision and content risks must be identified during scoping, not discovered during testing. Relying on the vendor assurance report to set scope for the permitting model review fails conceptually by outsourcing risk identification to the vendor and ignoring model and data controls specific to the municipality. Evaluating access controls for the permit portal during compliance testing activities fails as a planning substitute: access control testing occurs during fieldwork and does not replace AI-aware scoping and risk assessment before sampling begins.

Exam caveat: AI-aware scoping should be proportionate—lightweight GenAI with no decision influence still needs some AI risk review, but not the same depth as automated permit determinations.

Operational check: update the city audit planning checklist with GenAI risk categories (model/data/bias/decision/vendor), require a documented AI risk assessment before finalizing scope, and retain the scoping memo showing why permitting GenAI was not scoped by deferring AI risk assessment to fieldwork testing of permit approval workflows.