A team is defining success criteria for a refactoring agent that will automatically update deprecated API calls across a large codebase. Which success criteria definition best incorporates both functional performance and Responsible AI safety constraints?
Select an answer to reveal the explanation.
Short Explanation and Infographic
A pure speed or coverage target tells the agent to optimize without guardrails. Safe success criteria encode both what the agent should achieve and how it should behave when uncertain: stay in scope, flag ambiguity, produce an audit trail, never exceed its authority. Option B is the only definition that includes all three. The correct answer is B.
Full explanation below image
Full Explanation
Responsible AI success criteria for an autonomous agent are not just about functional outcomes — they must also encode behavioral constraints that bound the agent's authority and decision-making. A well-defined success criteria for this refactoring agent includes multiple dimensions.
Functional success: the deprecated API calls are updated to their modern equivalents, and existing tests pass after the changes.
Scope constraint: the agent only modifies files it has been explicitly authorized to touch. This prevents scope creep where an agent, in pursuit of its functional goal, modifies adjacent files that were not approved for change.
Uncertainty handling: when the agent encounters an ambiguous case (e.g., a deprecated call used in an unusual way that might require semantic understanding to safely refactor), it flags the case for human review rather than guessing. This is the 'know your limits' principle for autonomous systems.
Auditability: a change manifest that documents every file touched, every substitution made, and every case flagged provides the post-execution record required for compliance and debugging.
Option A is incorrect because 100% completion as the sole criterion creates incentive for the agent to make uncertain changes rather than flagging them, violating the 'flag ambiguity' safety constraint.
Option C is incorrect because code quality improvement is outside the authorized scope of a refactoring agent. An agent optimizing for quality metrics may make unintended changes to the codebase beyond its approved scope.
Option D is incorrect because 'no new bugs in CI' is a necessary condition but not sufficient. CI test suites often do not cover all production behavior. An agent could pass all tests while making scope-violating changes, failing to document its actions, or silently making uncertain substitutions.