A municipal clerk bot invents a case citation that never existed, creating legal exposure for the city attorney. How should security treat this failure?
Select an answer to reveal the explanation.
Short Explanation
A made-up court cite is not a cute typo—it is the bot hallucinating. Treat it like an integrity failure: ground answers, check cites, and keep a human in the loop before anyone relies on it. Fine-tuning on verified codes and benchmarks, tweaking chunk overlap and top-k, or lowering temperature for consistent answers won't give you a real case.
Full Explanation
Treating a municipal clerk bot that invents a nonexistent case citation as an AI integrity and hallucination failure is correct because fabricated legal authorities create direct liability for counsel who rely on them. Harmful hallucinations are not benign phrasing errors; they are integrity failures when invented facts, cites, or holdings enter operational workflows. The control response is grounding against authoritative sources, automated or human citation verification, and mandatory human review before any AI output is used as legal advice or court-facing language.
Framing the event as an AI integrity and knowledge gap that needs domain fine-tuning on verified codes, curated case corpora, and periodic accuracy benchmarking fails because fine-tuning and benchmarks alone do not close the gap for future queries into unseen case law, and they skip the compensating controls—grounding, citation verification, human review—that must gate legal output regardless of how well the model was tuned. Framing the event as an AI integrity and grounding gap that needs larger chunk overlap, higher top-k retrieval, and vector index optimization for code search fails because tuning retrieval recall does not verify that an emitted citation exists in a primary source, and it skips the compensating controls—citation verification and attorney review—that must gate legal output regardless of search performance. Framing the event as an AI integrity and generation fault that needs lower sampling temperature, constrained decoding limits, and repeat-query consistency verification fails because a cooler, constrained, consistently repeated hallucination is still a fabrication; those controls do not ground the citation to an authoritative source or replace mandatory attorney review.
Exam caveat: grounding and citation checks reduce but do not eliminate hallucination risk, especially when source corpora are incomplete or jurisdictionally mismatched; human legal review remains non-delegable for binding advice, and is not satisfied by domain fine-tuning on verified codes, curated case corpora, and periodic accuracy benchmarking alone, by larger chunk overlap, higher top-k retrieval, and vector index optimization, or by lower sampling temperature, constrained decoding limits, and repeat-query consistency verification. Operational check: require the clerk assistant to emit citation strings only when a retrieval step returns a matching primary source identifier, log retrieval misses, and block release of legal drafts until an attorney attests that each cited authority was opened and confirmed.
Prefer to study offline? Printable PDF study book + test engine ($7.50, unofficial). These questions stay free here.