A team revises their agent's constraint instructions to fix a reported failure where the agent was modifying files outside the target directory. After the revision, the original failure is resolved. However, the evaluation suite now shows three new failures: the agent refuses to follow symlinks that legitimately point into the target directory, fails to process files in nested subdirectories, and incorrectly rejects valid relative paths. What does this outcome indicate?

This question tests your understanding of constraint revision side effects — a common failure mode in agent tuning where a fix for one problem is scoped too broadly and inadvertently breaks adjacent, legitimate behaviors.

The correct answer is the option stating that the constraint revision was too broad, and the team should narrow the instruction change and run the full evaluation suite before deploying future revisions. The original instruction change to prevent the agent from accessing files outside the target directory was correct in intent but too broad in implementation. A constraint that correctly blocks '/outside/dir/file.txt' may also block '/target/dir/../../other' (relative path traversal), symlinks that resolve inside the target, or deeply nested subdirectory paths if the pattern matching is too aggressive. The lesson is twofold: (1) constraint revisions should be as narrow as possible, targeting the specific prohibited behavior without blocking similar-but-legitimate behaviors; and (2) the full evaluation suite must be run after every revision to catch exactly these lateral side effects before deployment.

The claim that the agent model is incompatible with the revised constraint instructions and needs to be replaced is incorrect. The issue is instruction specificity, not model compatibility. Changing the model would not fix an overly broad constraint instruction — it would reproduce the same problem on a different model.

The claim that the evaluation suite is flawed and should be updated to reflect the new behavior rather than treating it as a failure is incorrect and dangerous. The evaluation suite is correctly identifying real regressions — behaviors that worked before and are now broken. Updating the suite to accept the new broken behaviors would permanently encode the regression as expected, eliminating the safety net for future changes.

The claim that symlinks, nested subdirectories, and relative paths should be removed from the agent's scope to prevent future constraint conflicts is incorrect. Removing symlinks, nested subdirectories, and relative paths from scope is an extreme scope reduction that would make the agent significantly less useful. These are standard file system patterns that any robust file-processing agent should support. The correct fix is to narrow the constraint instruction, not eliminate the features it unintentionally breaks.

The key principle: every constraint revision has lateral effects. Narrow scope changes plus post-revision full-suite evaluation is the required workflow.