A city permitting chatbot can be tricked by crafted prompts into revealing internal staff notes about applications. What AI-specific threat should the auditor emphasize?
Select an answer to reveal the explanation.
Short Explanation
If a clever prompt can make the bot spill staff notes, that's prompt injection — an AI-era leak path. Hallucination, training-data poisoning, and model bias are different issues; they don't explain crafted prompts pulling internal notes. Lock down retrieval and output controls around permitting data.
Full Explanation
Emphasizing prompt-injection threats that exfiltrate nonpublic staff notes through the civic GenAI interface identifies an AI-specific attack path against permitting chatbots. The correct emphasis works because adversarial instructions in user input can override intended boundaries, cause the model to ignore system policies, and disclose retrieved internal application notes that should never be resident-facing. Weak input handling, overly broad retrieval, and insufficient output filtering make municipal permitting GenAI a confidentiality risk distinct from classic web flaws.
Hallucination errors that invent incorrect zoning requirements in chatbot answers fail as the highlighted threat here because the scenario describes crafted prompts overriding boundaries to disclose internal notes, not the model generating inaccurate zoning content.
Training-data poisoning that corrupts permit outcomes during scheduled model retraining fails conceptually for this question: it is a model-integrity issue during retraining, not the AI-specific disclosure path described where user prompts extract resident-invisible staff notes.
Model-bias risks that skew chatbot guidance across different applicant neighborhoods fail because they do not explain how crafted prompts cause the chatbot to reveal internal staff notes; bias concerns fairness in outputs, not prompt-driven exfiltration of nonpublic application notes.
Exam caveat: successful prompt-injection defenses are layered; blocking a few known strings without retrieval scoping and output redaction still leaves residual leakage risk. Operational check: test adversarial prompts against the permitting bot, verify staff notes are excluded from resident-visible context or strongly redacted, log suspicious extraction attempts, and require security sign-off before enabling retrieval of nonpublic permit annotations.