School-district security finds an assistant inventing disciplinary policy that staff might follow. How should this be treated in AI vulnerability management?
Select an answer to reveal the explanation.
Short Explanation
When the bot invents discipline rules, that’s an integrity break—not a prompt injection, bias violation, or data poisoning concern. Input sanitization, safety filters, and training data curation won’t stop made-up policy. You ground it, force human checks, and stop staff from treating fiction like policy, because made-up rules still get kids in real trouble.
Full Explanation
When a school-district assistant invents disciplinary policy that staff might follow, vulnerability management should treat hallucinated authoritative guidance as an integrity vulnerability. False rules can drive unfair discipline, due-process failures, and compliance breaches even when delivered confidently. Remediation with grounding (retrieval from official policy), citation requirements, human review for policy answers, and monitoring for unsupported claims works because integrity controls bind outputs to approved sources rather than generative improvisation.
Treating hallucinated policy guidance as a prompt injection attack fails because fabricating disciplinary rules is an integrity failure, not an input-based hijack; input sanitization and jailbreak filters will not stop the model from inventing false policy. Flagging hallucinated disciplinary guidance as a bias violation fails because invented policy is an integrity problem, not a fairness skew; safety filters and RLHF alignment tuning do not prevent the model from issuing false rules. Handling hallucinated policy output as a data poisoning concern fails because invented disciplinary rules are a generation-time integrity failure, not corrupted training data; training data curation and model retraining will not stop the model from fabricating policy.
Exam caveat: grounded systems can still mis-retrieve or mis-summarize; human review remains necessary for high-impact student decisions. Operational check: force policy answers through approved corpus retrieval with citations, block or flag answers lacking sources, sample assistant outputs weekly for unsupported disciplinary claims, and train staff that only published board policy—not chat text—authorizes action.
Prefer to study offline? Printable PDF study book + test engine ($7.50, unofficial). These questions stay free here.