Procurement accepts a vendor 'aligned municipal model' that provides no documentation of safety evaluation harnesses. What should the city demand?
Select an answer to reveal the explanation.
Short Explanation
If a vendor says 'trust us, it's aligned for cities' but won't show the test harness, that's a handshake without a seatbelt check. Ask for the safety eval evidence before the PO clears.
Full Explanation
Procurement accepting a vendor "aligned municipal model" with no documentation of safety evaluation harnesses leaves the city without evidence that alignment claims were tested. The city should demand evidence of the alignment and safety evaluation harnesses used to test the customized or moderated model—coverage, methods, civic-relevant scenarios, and results—before treating the product as assured for resident-facing use.
A marketing one-pager asserting the model is "fully aligned" without test detail is assertion, not assurance. Immediate production cutover so residents can supply free adversarial examples externalizes testing onto the public and is reckless for municipal services. Removing all contractual audit rights to accelerate purchasing weakens ongoing verification and is the opposite of responsible GenAI procurement.
Exam caveat: Vendor alignment marketing is not a substitute for evaluable safety evidence. Distinguish sales collateral from harness documentation and results. GAIPS procurement items reward demanding test evidence over brand trust or speed-to-production shortcuts.
Operational check: Require evaluation reports and harness descriptions in the RFP or security questionnaire, map vendor tests to city use cases (311, benefits language, youth services, multilingual prompts), reserve contractual audit or attestation rights, and withhold production cutover until residual gaps are covered by municipal test suites or accepted risk with compensating gateway controls.