The City of Maplewood's digital services team has deployed a Copilot Studio agent to answer citizen questions about permit applications, zoning regulations, and municipal services. The agent uses a curated knowledge base of official city documents. Because incorrect information could mislead residents and create legal liability, the agency director requires a formal approval process before the agent goes live. The evaluation lead, Simone, presents evaluation results showing the agent's average groundedness score. The director asks: 'What groundedness score indicates that the agent's responses are adequately supported by the official documents?' What is the generally accepted minimum groundedness score threshold that indicates acceptable agent quality in Copilot Studio AI evaluation?
Select an answer to reveal the explanation.
Short Explanation
On Copilot Studio's 1-5 groundedness scale, you'll need 3 or higher to call responses adequately grounded. Below 3, you're seeing too many answers drift from the official documents. Aim for 4 or 5 when you can, but 3 is the practical passing floor.
Full Explanation
Copilot Studio's AI evaluation framework scores groundedness, along with metrics such as relevance and coherence, on a scale of 1 to 5, where 1 represents the worst quality and 5 represents the best. Microsoft's evaluation framework guidance establishes that a score of 3 or higher generally indicates acceptable agent quality, meaning responses are mostly grounded in the knowledge source with acceptable levels of unsupported content. The correct answer is the groundedness score of 3 out of 5 or higher, indicating that most responses are adequately grounded in the knowledge source. For high-stakes deployments like government services, teams often target 4 or higher as a goal, but 3 is the minimum acceptable baseline before production deployment. Microsoft also advises calibrating thresholds to the risk profile of the use case, but 3 remains the general baseline. The alternative claiming a groundedness score of 1 out of 5 is incorrect because 1 represents the lowest possible quality, with nearly all responses unsupported by source content and severe hallucination risk; that would immediately disqualify an agent from production deployment, especially where legal liability is cited. The alternative claiming a groundedness score of 2 out of 5 is incorrect because 2 indicates below-average source support, where the majority of responses have inadequate grounding. That is below the acceptable threshold and should trigger immediate remediation of the knowledge base or agent configuration. The alternative claiming a groundedness score of 5 out of 5 is incorrect because requiring perfect source alignment for any production deployment is an unrealistically high bar. Perfect groundedness is difficult to achieve across all query types, especially with complex or ambiguous questions, and setting 5 as a mandatory minimum would prevent many capable and safe agents from deploying. The realistic target for high-quality agents is 4 or above, with 3 as the acceptable floor. Exam tip: memorize the scale: 1-2 is unacceptable and needs remediation, 3 is the minimum acceptable threshold, and 4-5 is good to excellent quality. On this exam, threshold questions often ask what score indicates acceptable quality, and the answer is almost always 3 or higher on the 1-5 scale. See Microsoft Learn guidance on agent evaluation and readiness: https://learn.microsoft.com/en-us/microsoft-copilot-studio/analytics-agent-evaluation-overview and https://learn.microsoft.com/en-us/microsoft-copilot-studio/guidance/evaluation-triage-readiness, plus Microsoft Copilot blog guidance on custom graders: https://www.microsoft.com/en-us/microsoft-copilot/blog/copilot-studio/custom-graders-in-copilot-studio-setting-high-standards-for-agent-evals/.