A developer asks an agent: 'Clean up the dead code in our payment module.' The agent interprets this as permission to remove any code that is not referenced by the main execution path, including feature flags that are currently disabled but are scheduled for activation next quarter. The agent deletes 3 feature flag implementations, which pass all automated tests (since they are not active), but cause a major incident when the quarterly feature release begins. Which evaluation approach would BEST have detected this intent misalignment before the deletion occurred?
Select an answer to reveal the explanation.
Short Explanation
Tests can only catch what they can see — they're blind to what was deleted. The alignment gap here isn't that the agent wrote bad code; it's that it interpreted a vague instruction too broadly. The fix is to close the gap before execution: make the agent write down what it thinks you meant and show it to you before it touches anything irreversible.
Full Explanation
This scenario illustrates an intent-scope misalignment: the developer's intent was to remove genuinely unused code, but the agent's interpretation was technically correct by one definition, 'not referenced by current execution path,' while being wrong by the developer's contextual intent, 'not needed in the foreseeable future.' The root problem is that the agent acted on an ambiguous instruction without verifying its interpretation. The explicit intent interpretation document approach, also called a pre-execution plan review, addresses this by making the interpretation explicit, listing specific planned deletions, and gating irreversible actions. The agent states in plain language what it understands the instruction to mean and what specific actions it plans to take, such as deleting 15 functions and 3 feature flag implementations because they have no current callers. The developer can review the list and immediately identify the feature flag implementations as intentional disabled code, not dead code, and no deletion occurs until the developer has affirmatively confirmed the interpretation is correct. This pattern is especially critical for ambiguous verbs like 'clean up,' 'optimize,' and 'simplify,' which map to very different action scopes depending on context. The test-suite approach explicitly fails in this scenario because the scenario states tests pass because the feature flags are inactive, and automated testing cannot validate the semantic intent of deletions. The file-age confirmation approach is a brittle rule that would not catch this case, since the feature flags might be recently written for the upcoming quarter, and it would generate false positives for legitimately old dead code. The training-on-examples approach improves statistical behavior over time but does not provide a mechanism for catching this specific misalignment on this specific task, and it cannot account for team-specific context, such as this team having upcoming feature flags, that no training corpus could anticipate.