Northshore Medical Group has deployed a Copilot Studio agent that helps clinicians look up treatment protocols from a curated SharePoint knowledge base. The compliance team is concerned that the agent might be making up information not present in the approved documents — a risk they call 'hallucination.' The AI team lead, Marcus, wants to choose the most appropriate evaluation method to surface this specific risk. Which evaluation method should Marcus prioritize to detect whether the agent's responses are fabricating information not found in the source documents?
Select an answer to reveal the explanation.
Short Explanation
Groundedness is like fact-checking an article against original source documents — it doesn't care if the writing's pretty or on-topic, it only cares if every factual claim traces back to the source. When hallucination's the risk, groundedness is your tool.
Full Explanation
In artificial-intelligence evaluation frameworks, groundedness specifically measures whether the agent's response is anchored in and supported by provided source documents or knowledge base. It is designed precisely to detect hallucination — the scenario where artificial intelligence generates plausible-sounding but fabricated information not present in source material. In the medical compliance context like Northshore Medical Group's, this is the highest-priority metric because incorrect clinical information could cause patient harm.
The relevance evaluation method is incorrect because relevance measures something different: how well the response addresses the user's question. The agent could give highly relevant answers, meaning they correctly address what the user asked, while still hallucinating details not in the source. Relevance and groundedness are orthogonal dimensions — you need both, but relevance alone does not catch fabrication.
The coherence evaluation method is incorrect because coherence evaluates logical flow and internal consistency of the response. The response can be perfectly coherent, meaning well-structured, grammatically logical, and internally non-contradictory, while still containing fabricated information. Coherence is about how information hangs together, not whether it comes from authorized sources.
The fluency evaluation method is incorrect because fluency assesses quality of language — grammar, sentence structure, and readability. High fluency is unrelated to accuracy or source fidelity. In fact, fluency can make hallucinations harder to spot, because smooth prose sounds authoritative even when fabricated.
The groundedness evaluation method is correct because groundedness is specifically designed to answer the question: 'Is every claim in this response traceable to the knowledge source?' The evaluation engine compares the agent's output against retrieved context and scores how well supported each assertion is. Low groundedness scores are direct signals of hallucination risk.
Exam tip: On the exam, associate 'hallucination / fabrication / source fidelity' with groundedness, and 'how well it answers the question' with relevance. These two are commonly confused. If the scenario mentions curated knowledge base and compliance concerns about invented content, groundedness is always the answer.