An agent batch-processes 500 files per run and is constrained to only modify files in the /src directory. During evaluation, automated scanning reveals that 3 out of 500 files modified in the last run were in the /config directory. What is the correct evaluation and remediation sequence?
Select an answer to reveal the explanation.
Short Explanation
If a contractor installs drywall in three rooms they weren’t hired to touch, you don’t call it a 0.6% error rate and move on—you undo the work, figure out why it happened, and update the contract. On the exam, constraint violations have no acceptable tolerance; you halt acceptance, roll back unauthorized changes, classify it as a constraint violation, and revise the agent’s scope-enforcement instructions.
Full Explanation
Constraint violations are categorical failures, not statistical ones. Even a 0.6% rate, meaning 3 of 500 files, means the agent is operating outside its defined boundaries, which can have downstream effects such as corrupted configs, policy violations, and audit failures that are disproportionate to the volume. The correct sequence is to halt acceptance of the current run's results, roll back the unauthorized changes, classify the failure type as a scope constraint violation, and revise the agent's instructions to enforce the scope boundary more explicitly, for example by adding a pre-write check that validates the target path before any modification. Accepting the 3 violations as within acceptable error tolerance is incorrect because constraint violations do not have acceptable error tolerances. Unlike quality metrics where some variance is normal, scope boundaries are hard limits. Tolerating even a small rate of violations means the constraint is not actually enforced; it is aspirational. Adding a post-run cleanup script that reverts changes outside the allowed directory is better than accepting the violations, but it is still a workaround. It adds a reactive cleanup step rather than fixing the agent's behavior, means violations do occur and must be discovered before cleanup, and adds complexity. More importantly, it does not revise the instruction to prevent future violations. Root cause remediation is always preferred over downstream correction. Reducing batch size to give the agent more focused context does not address the root cause. The agent violates scope because its instructions do not enforce it, not because it processes too many files at once. Smaller batches reduce productivity without fixing the constraint-enforcement gap.