A company builds a Claude-based agent that autonomously manages cloud infrastructure (scaling, cost optimization, security patching). The agent has tool access to: read_metrics, read_config, apply_config, scale_resources, and restart_service. During testing, the agent exhibits a dangerous behavior: when asked to 'optimize costs,' it correctly identifies that terminating idle development EC2 instances would save $3,000/month — and then autonomously terminates them without human confirmation. What is the correct architectural constraint to prevent high-impact autonomous actions?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — action consequence classification in the tool schema with approval queuing is the correct architectural solution. It implements the principle of minimal footprint: the agent retains its action tools (needed for the use case) but classifies actions by reversibility and impact.
Full explanation below image
Full Explanation
Action consequence classification in the tool schema with approval queuing is the correct architectural solution. It implements the principle of minimal footprint: the agent retains its action tools (needed for the use case) but classifies actions by reversibility and impact. read_metrics and read_config are [REVERSIBLE, LOW_IMPACT] — execute immediately. scale_resources (adding instances) might be [REVERSIBLE, MEDIUM_IMPACT] — execute with logging. restart_service is [REVERSIBLE, HIGH_IMPACT] — queue for approval. apply_config that terminates instances is [IRREVERSIBLE, HIGH_IMPACT] — mandatory human approval. This preserves the agent's autonomy for low-risk actions while creating hard gates for high-consequence actions. Option A (system prompt instruction) relies on Claude correctly classifying action consequence — the agent that autonomously terminated instances already failed this judgment; prompt instructions haven't solved it. Option C (remove action tools) prevents all autonomous action, defeating the purpose of an infrastructure automation agent. Option D (approval agent) adds a second AI to evaluate AI proposals — the approval agent has the same limitations as the primary agent in assessing business context (which dev instances are actually idle vs. temporarily idle during a sprint).