A code-generation agent is evaluated solely on whether its output compiles without errors. Business stakeholders report that the agent frequently generates code that compiles correctly but is unmaintainable and violates team coding standards. What change best aligns the evaluation signal with business intent?
Select an answer to reveal the explanation.
Short Explanation
You’re measuring whether the engine starts, not whether the car is roadworthy. Add automated static analysis for maintainability and standards compliance so success means what the business actually cares about.
Full Explanation
The existing evaluation signal, compilation success, is necessary but insufficient. It captures technical correctness at the syntactic level but ignores the business-defined quality requirements: maintainability and standards compliance. The correct change is to require the agent to pass both compilation checks and an automated static analysis scan covering maintainability and standards compliance before a run is considered successful. That extends the success criteria to include checks such as linting rules, cyclomatic complexity thresholds, and naming convention enforcement, making the evaluation signal align with what the business actually considers success while remaining fully automated and scalable.
The lines-of-code metric is wrong because lines of code is an anti-metric; more code is not inherently better, and in many cases verbose code is less maintainable. Tracking generated lines of code actively misaligns the agent with business intent by incentivizing verbosity over quality.
The manual human review approach is wrong because it is not scalable and defeats the purpose of automated evaluation. While human review has a role in periodic audits, requiring it for every run creates a bottleneck and increases latency. Automated static analysis can cover most standards compliance checks without human involvement.
The lower-temperature tuning approach is wrong because model temperature affects output diversity and randomness, not adherence to coding standards. Lowering temperature makes the agent more deterministic, but it does not teach the agent what the coding standards are. The agent will produce more consistent code that still violates standards if those standards are not represented in its evaluation criteria or instructions.