A developer asks an agent: 'Clean up the dead code in our payment module.' The agent interprets this as permission to remove any code that is not referenced by the main execution path, including feature flags that are currently disabled but are scheduled for activation next quarter. The agent deletes 3 feature flag implementations, which pass all automated tests (since they are not active), but cause a major incident when the quarterly feature release begins. Which evaluation approach would BEST have detected this intent misalignment before the deletion occurred?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Tests can only catch what they can see — they're blind to what was deleted. The alignment gap here isn't that the agent wrote bad code; it's that it interpreted a vague instruction too broadly. The fix is to close the gap before execution: make the agent write down what it thinks you meant and show it to you before it touches anything irreversible.
Full explanation below image
Full Explanation
This scenario illustrates an intent-scope misalignment: the developer's intent was to remove genuinely unused code, but the agent's interpretation was technically correct by one definition ('not referenced by current execution path') while being wrong by the developer's contextual intent ('not needed in the foreseeable future').
The root problem is that the agent acted on an ambiguous instruction without verifying its interpretation. An intent interpretation document (also called a 'pre-execution plan review') addresses this by:
1. Making the interpretation explicit: The agent states in plain language what it understands the instruction to mean and what specific actions it plans to take (e.g., 'I will delete the following 15 functions and 3 feature flag implementations because they have no current callers') 2. Listing specific planned deletions: The developer can review the list and immediately identify the feature flag implementations as intentional disabled code, not dead code 3. Gating irreversible actions: No deletion occurs until the developer has affirmatively confirmed the interpretation is correct
This pattern is especially critical for ambiguous verbs ('clean up,' 'optimize,' 'simplify') that map to very different action scopes depending on context.
Option A (test suites) explicitly fails in this scenario — the scenario states tests pass because the feature flags are inactive. Automated testing cannot validate the semantic intent of deletions. Option C (file age heuristic) is a brittle rule that would not catch this case (the feature flags might be recently written for the upcoming quarter) and would generate false positives for legitimately old dead code. Option D (training) improves statistical behavior over time but does not provide a mechanism for catching this specific misalignment on this specific task. It also cannot account for team-specific context (this team has upcoming feature flags) that no training corpus could anticipate.