A platform team is building an agent to automate the migration of 500 microservices from Java 8 to Java 17. Before deploying the agent, the team needs to define what a successful outcome looks like. The project manager suggests: 'The outcome is successful when all 500 services have been migrated.' An architect argues this definition is incomplete. Which outcome specification would be MOST appropriate for evaluating the agent's performance?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Saying 'migration complete when all services are migrated' is a circular definition — it tells you nothing about quality. Updating a pom.xml without verifying the service actually builds and passes tests is like declaring a house renovation complete because you bought the materials. Real outcome specification has objective, externally verifiable success signals, not agent self-attestation.
Full explanation below image
Full Explanation
Outcome specification for agentic tasks must be externally verifiable, multi-dimensional, and failure-distinguishing. The PM's definition ('all 500 services migrated') is circular: it makes 'migrated' the success criterion without defining what migrated means. This allows an agent to game the metric by making superficial changes.
The correct outcome specification uses four dimensions:
1. Functional correctness: Services compile without errors under the new runtime. Compilation failure is an objective, binary test — no ambiguity. 2. Behavioral preservation: Existing test suites must pass at a high rate (≥98%). This ensures the migration did not break existing functionality. The 2% tolerance accounts for flaky tests pre-existing before the migration. 3. Security integrity: Dependency upgrades that accompany Java 17 migrations can introduce or resolve CVEs. Checking for new critical CVEs ensures the migration does not inadvertently worsen the security posture. 4. Deployment validation: CI pipeline success to staging confirms the artifact not only compiles but also packages, deploys, and passes integration-level checks in a representative environment.
Option A (pom.xml commits) measures an agent action (it wrote a file) not an outcome (the service works). A service with a corrupted pom.xml might still commit successfully. Option C (agent self-attestation) is fundamentally flawed: allowing the agent to declare its own success creates a metric that the agent can trivially satisfy without performing real work, and removes the external verifiability requirement. Option D (migration report) measures documentation output, not migration quality. A detailed report of incorrect changes is worthless. The exam principle is that outcome specifications must be based on independently verifiable evidence, not agent-reported actions or artifacts.