Before releasing an internal LLM API to city departments, AI V&V procedures are updated. What extraction-related check should be included?

Before releasing an internal LLM API to city departments, updated AI verification and validation procedures should include extraction-resistance and related abuse tests as part of civic AI V&V. Opening departmental access expands the attack surface for query-based extraction, automated probing, and misuse; V&V must demonstrate that quotas, output minimization, monitoring, and other controls meaningfully raise the cost of approximation and abuse under realistic civic client patterns.

Verifying transport encryption on department API traffic protects data in transit, but it does not test whether an attacker can systematically query the model to build a functional copy, so it is not an extraction-resistance check. Reviewing prompt filter blocklists for known jailbreak strings addresses prompt-injection safety, not extraction resistance; a deployment can block jailbreaks and still leak a usable model through ordinary queries. Inspecting training data invoices and licenses traces data provenance and licensing compliance, but it does not verify that the deployed API resists query-based model extraction. V&V before an internal API goes to departments needs to specifically probe extraction-resistance, which none of the other checks cover.

Exam caveat: “extraction-resistant” is evidence-based and contextual, not absolute. V&V should state test methods, budgets, access assumptions, and residual risk rather than unsupported claims in model cards or release notes.

Operational check: add extraction and abuse suites to the pre-release checklist for the internal LLM API, record results against go/no-go criteria, and require named remediation owners before enabling department-wide credentials in production.