Northshore Medical Group has deployed a Copilot Studio agent that helps clinicians look up treatment protocols from a curated SharePoint knowledge base. The compliance team is concerned that the agent might be making up information not present in the approved documents — a risk they call 'hallucination.' The AI team lead, Marcus, wants to choose the most appropriate evaluation method to surface this specific risk. Which evaluation method should Marcus prioritize to detect whether the agent's responses are fabricating information not found in the source documents?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Groundedness is like a fact-checker comparing a journalist's article against original source documents — it doesn't care if the writing is pretty or on-topic, it only cares if every factual claim traces back to the source. When hallucination is the risk, groundedness is your tool. Correct answer: C.
Full explanation below image
Full Explanation
In AI evaluation frameworks, groundedness specifically measures whether the agent's response is anchored in and supported by the provided source documents or knowledge base. It is designed precisely to detect hallucination — the scenario where an AI generates plausible-sounding but fabricated information not present in the source material. In a medical compliance context like Northshore Medical Group's, this is the highest-priority metric because incorrect clinical information could cause patient harm.
Option A is incorrect because relevance measures something different: how well the response addresses the user's question. An agent could give a highly relevant answer (it correctly addresses what the user asked) while still hallucinating details not in the source. Relevance and groundedness are orthogonal dimensions — you need both, but relevance alone does not catch fabrication.
Option B is incorrect because coherence evaluates the logical flow and internal consistency of the response. A response can be perfectly coherent (well-structured, grammatically logical, internally non-contradictory) while still containing fabricated information. Coherence is about how information hangs together, not whether it comes from authorized sources.
Option D is incorrect because fluency assesses the quality of language — grammar, sentence structure, and readability. High fluency is unrelated to accuracy or source fidelity. In fact, fluency can make hallucinations harder to spot, because smooth prose sounds authoritative even when fabricated.
Option C is correct because groundedness is specifically designed to answer the question: 'Is every claim in this response traceable to the knowledge source?' The AI evaluation engine compares the agent's output against the retrieved context and scores how well supported each assertion is. A low groundedness score is a direct signal of hallucination risk.
Exam tip: On the AB-620 exam, associate 'hallucination / fabrication / source fidelity' with groundedness, and 'how well it answers the question' with relevance. These two are commonly confused. If the scenario mentions a curated knowledge base and compliance concerns about invented content, groundedness is always the answer.