A senior engineer is reviewing the autonomy configuration for a code-review agent that has been in production for six months. The agent started at a conservative autonomy level but the team wants to expand its scope. Which set of signals MOST appropriately indicates that the agent is ready for expanded autonomy?
Select an answer to reveal the explanation.
Short Explanation
Volume and vibes aren't readiness signals — they're popularity signals. You don't expand autonomy based on low overrides alone, because engineers may have just gotten used to clicking through. You need volume, measured quality, and clean safety record over defined window.
Full Explanation
Expanding agent autonomy requires multi-dimensional evidence, not one metric. Each signal type captures different aspect of readiness.
The signal based on more than 1,000 pull requests and no manual overrides is weak because it only measures volume and one behavioral signal. Zero overrides sounds positive, but it could mean engineers find agent comments high-quality, engineers are ignoring agent comments entirely, or agent is producing comments so cautious they never require correction. Without quality measurement, you cannot distinguish these interpretations. Volume alone tells you agent is used, not that it is accurate.
The composite signal combining processed volume, low override rate, engineer-reported quality surveys above threshold, and zero security/compliance false-negative incidents attributable to agent missed findings over defined observation window is correct because it spans four dimensions. Volume provides sufficient sample size to observe edge cases and variance. Low override rate shows engineers are accepting agent conclusions, not routinely correcting them, which is behavioral signal. Quality surveys provide structured measurement of perceived correctness, completeness, and actionability, distinguishing 'not overridden' from 'actually helpful.' Zero security/compliance false-negative incidents is most critical gate. If agent missed security vulnerability or compliance violation that caused production incident, volume and survey scores are irrelevant. This safety signal must be clean over meaningful observation window, ideally 90 days minimum. Only when all four dimensions are positive does composite indicate readiness.
The signal based on positive retrospective sentiment is anecdotal and subject to recency bias, social desirability bias, and confirmation bias. Engineers who like agent's general personality may not accurately report whether it is catching important issues.
The signal based on upgrading underlying model to newer, more capable version that scores higher on coding benchmarks is laboratory performance, not field performance. Model that scores higher on coding benchmarks may still produce worse results in your specific context, including codebase conventions, security policies, and domain-specific requirements. Benchmark improvement is reason to re-evaluate, not readiness signal itself.