An emergency management agency is assessing the risk that a GenAI assistant could produce hallucinated shelter locations during an active disaster response. Which characterization best captures this risk from a business-risk-management standpoint?
Select an answer to reveal the explanation.
Short Explanation
Think of hallucination like a confident stranger giving you directions without actually knowing the area: the answer sounds sure of itself, but it can be completely wrong. During a disaster, that's the last thing you want feeding shelter locations to the public, so the fix is a human in the loop, not just a newer model.
Full Explanation
Hallucination describes a GenAI model producing fluent, confident output that is not grounded in fact, and it is a known characteristic of how these models generate text rather than a rare malfunction. In a life-safety context like active shelter guidance, the correct mitigation is procedural: verify high-stakes output against authoritative sources and route it through human review before publication, since the risk cannot be engineered away entirely. Framing hallucination as a connectivity or storage failure misdiagnoses the mechanism — it happens even when systems are fully online and functioning normally, so a disaster-recovery infrastructure plan does nothing to address it. Framing it as something a newer model version simply removes is a common but risky misconception; newer models can reduce hallucination frequency, but they do not eliminate the underlying tendency, so treating an upgrade as a substitute for review leaves the agency exposed. Framing hallucination as a tone issue misunderstands what is actually unreliable — the concern is factual accuracy, not how something is phrased, so style guidelines address a different problem entirely. A scope caveat: hallucination risk rises with how open-ended or unfamiliar the question is, so shelter and evacuation queries deserve tighter controls than routine ones. As an operational check, the agency should confirm a documented verification step exists specifically for any GenAI output naming a location or action a resident might follow.