A code-generation agent is evaluated solely on whether its output compiles without errors. Business stakeholders report that the agent frequently generates code that compiles correctly but is unmaintainable and violates team coding standards. What change best aligns the evaluation signal with business intent?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Measuring only whether a car's engine starts ignores whether it's roadworthy — and roadworthy is what business cares about. On the exam: evaluation signals must cover the full definition of success, not just the easiest-to-measure proxy; add automated static analysis to close the gap between compilation success and code quality.
Full explanation below image
Full Explanation
The existing evaluation signal — compilation success — is a necessary but insufficient condition. It captures technical correctness at the syntactic level but ignores the business-defined quality requirements: maintainability and standards compliance. The correct fix (option B) is to extend the success criteria to include automated static analysis checks (e.g., linting rules, cyclomatic complexity thresholds, naming convention enforcement). This makes the evaluation signal align with what the business actually considers success, and it remains fully automated — keeping the pipeline scalable.
Option A is incorrect because lines of code is an anti-metric — more code is not inherently better, and in many cases verbose code is less maintainable. This metric actively misaligns the agent with business intent by incentivizing verbosity over quality.
Option C (manual human review) is not scalable and defeats the purpose of automated evaluation. While human review has a role in periodic audits, requiring it for every run creates a bottleneck and increases latency. Automated static analysis can cover most standards compliance checks without human involvement.
Option D is incorrect because model temperature affects output diversity and randomness, not adherence to coding standards. Lowering temperature makes the agent more deterministic, but it does not teach the agent what the coding standards are. The agent will produce more consistent code that still violates standards if those standards are not represented in its evaluation criteria or instructions.