You are designing an agent that automatically triages GitHub Issues, assigns labels, and routes them to the correct project board column. Before implementing the agent, you need to define its success criteria. Which definition most completely captures both the output contract and measurable success?
Select an answer to reveal the explanation.
Short Explanation and Infographic
A sports team doesn't define winning as 'everyone got on the field and didn't foul out' — they define it as a score. Success criteria for agents must be equally specific: what output is produced, what shape it takes, and how correctness is measured. Option B does all three — it names the artifact, its schema, and a quantified accuracy benchmark against a validation set.
Full explanation below image
Full Explanation
Defining success criteria and output contracts is a foundational step in agent architecture. Without them, you cannot validate behavior, compare agent versions, or detect regressions. A good definition combines three elements: (1) a structured output artifact with a defined schema, (2) measurable quality criteria, and (3) a reference benchmark or validation method.
Option B is correct. It specifies the output artifact (a triage record per issue), its schema (labels, destination column, confidence score, reasoning trace), and a measurable quality bar (95% label accuracy vs. a human-reviewed validation set). This is a complete output contract — a downstream system or human reviewer knows exactly what to expect and how to evaluate it.
Option A defines success in terms of throughput (closing issues) rather than quality. Closing all issues within 24 hours could mean auto-closing them without review — a destructive outcome that satisfies the metric but violates intent. Activity is not the same as correctness.
Option C defines success as 'no exceptions thrown.' This is an operational health check, not a success criterion. An agent can complete every API call successfully while producing incorrect labels on every issue. Execution health and output quality are separate concerns.
Option D specifies latency (5-minute SLA) and a notification behavior (comment). These are valid SLA requirements, but they say nothing about the accuracy or structure of the triage decision itself. Speed with wrong labels is worse than slow with right ones.
The exam objective is: 'Define inputs, outputs, and success criteria for agents.' The output contract must be schema-driven and quality must be measurable against a defined benchmark — not inferred from execution health or throughput alone.