A housing chatbot grounded on lease FAQs still invents a fee; staff learn GenAI can produce plausible but false text. What risk is illustrated?

Hallucination risk is the tendency of generative models to produce fluent, plausible text that is factually wrong. A housing chatbot grounded on lease FAQs that nevertheless invents a nonexistent fee illustrates that retrieval or FAQ grounding reduces—but does not eliminate—hallucinations. Municipal staff must treat GenAI answers as provisional, especially for fees, deadlines, eligibility, and legal interpretations that affect residents’ housing outcomes.

Data currency risk where the chatbot retains outdated FAQ content after housing policy updates revise fee schedules is not what the scenario shows; the chatbot invents a nonexistent fee rather than relying on stale content, and FAQ grounding can still fail to prevent fabrication. Privacy disclosure risk where the chatbot reveals sensitive tenant details drawn from lease records in answers is not illustrated here; the problem is a fabricated fee, not the exposure of private tenant data. Automation bias risk where housing staff over-rely on chatbot drafts when reviewing tenant fee disputes is a separate issue; the scenario centers on the model generating a false fee, which is why verification remains necessary regardless of staff reliance.

Exam caveat: “grounded” or “RAG” systems can still hallucinate when retrieval misses, chunks are partial, or the model answers beyond retrieved text. Operational check: constrain answers to cited FAQ passages, show source snippets to users and staff, block fee and penalty statements unless an exact approved source is retrieved, and offer a clear path to a housing specialist.