An engineering team is evaluating where in their SDLC to introduce agentic automation. They track four metrics: story-point velocity, defect escape rate, mean time to merge (MTTM) for pull requests, and on-call incident frequency. After a six-week pilot, agent-assisted code review reduced MTTM by 40% but defect escape rate increased by 12%. Agent-assisted test generation had no measurable velocity impact but reduced defect escape rate by 18%. Which conclusion BEST describes where agentic automation is currently adding genuine value?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of MTTM like checkout speed at a grocery store — faster lines feel great until the cashier starts letting shoplifters through. A 40% faster merge time paired with a 12% higher defect escape rate means the review agent is creating downstream rework. Test generation, by contrast, directly cuts defects reaching customers — the metric that actually matters most. Faster merges with more bugs is a bad trade.
Full explanation below image
Full Explanation
This question tests whether candidates understand the difference between proxy metrics (velocity, merge speed) and outcome metrics (defect escape rate, incident frequency). Agent ROI must be measured against outcomes that matter to the business, not just throughput.
Option A is wrong because throughput improvements do not 'always' outweigh quality trade-offs. A 40% reduction in MTTM sounds impressive, but if the review agent is rubber-stamping PRs and allowing 12% more defects to escape into production, those defects will create rework, incidents, and customer impact that cost far more than the time saved. Speed without quality is anti-value.
Option B is correct. Test generation did not improve velocity, but a reduction in defect escape rate by 18% is directly tied to customer experience and production stability. Defects that don't escape don't become incidents, don't trigger on-call responses, and don't require emergency patches. The ROI is real even if it's not visible in story points. The absence of velocity improvement is simply neutral, not negative.
Option C applies a too-strict standard. Requiring all four metrics to improve simultaneously before declaring value ignores that individual SDLC interventions target specific stages. A test agent is not designed to move story-point velocity; it's designed to catch bugs earlier.
Option D is wrong because deploying an agent that demonstrably increases defect escape rate is harmful. The review agent needs root-cause analysis before any production deployment — it may need calibration, a stricter escalation policy, or human checkpoints before final merge approval.