LexAI Solutions is building a Copilot Studio agent for a law firm to help paralegals look up case law and statutory references. The agent quality lead, Tomas, is designing the test case library. He has written 40 test cases that all follow the same pattern: a well-formed legal query in formal language, matched against a clear, single-source answer. His colleague points out that real paralegals will also ask vague questions, use informal phrasing, mix multiple legal topics in one question, and occasionally try to get the agent to give legal advice it is not authorized to provide. What type of test cases is Tomas's library currently missing?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Tomas's test set is like a driving exam that only tests highway driving in perfect weather — it never checks how the driver handles a roundabout in a snowstorm or a pedestrian who jumps out. Real-world agent failures happen on edge cases and adversarial inputs, not just clean happy-path queries. Correct answer: D.
Full explanation below image
Full Explanation
A comprehensive test case library for an AI agent evaluation should cover at least three categories of test scenarios: happy path cases (standard, well-formed queries with clear answers), edge cases (ambiguous phrasing, multi-topic questions, unusual input formats), and adversarial cases (inputs specifically designed to elicit policy violations, unauthorized behavior, or model confusion).
Tomas's library of 40 well-formed formal legal queries covers only the happy path. In real deployment, paralegals will use a wide range of input styles, some of which will expose agent weaknesses that never appear in happy path testing.
Option D is correct because edge cases and adversarial test cases are the missing categories. Edge cases test the boundaries of agent capability — what happens when a query spans multiple topics, uses informal language, or contains ambiguity that requires clarification? Adversarial cases test guardrails — what happens when someone asks the agent to provide unauthorized legal advice or attempts to manipulate the agent's behavior? Both are essential for enterprise agents deployed in sensitive professional contexts like law firms.
Option A is incorrect because happy path test cases are exactly what Tomas already has. His 40 well-formed, single-source, formally phrased questions represent the happy path — the scenario where everything is clean and expected. This is the category he has covered thoroughly, not the one he is missing.
Option B is incorrect because regression test cases are a specific type used after bug fixes or model updates to verify that previously passing scenarios have not broken. While regression testing is important in mature evaluation pipelines, it is not the category of test case that addresses real-world user variation and adversarial behavior — the problem identified by Tomas's colleague.
Option C is incorrect because load testing measures system performance under concurrent usage, which is an infrastructure and scalability concern rather than an AI quality evaluation concern. Test sets in Copilot Studio's evaluation framework are for quality assessment, not load testing.
Exam tip: The AB-620 exam expects candidates to know the three major test case type categories: happy path, edge cases, and adversarial inputs. If a scenario shows a test set that only contains clean, well-formed, single-answer questions, the answer will involve adding edge or adversarial cases.