A security team reviews agent logs and discovers that an agent has been invoking a delete_file tool repeatedly, despite delete_file not appearing in the agent's configured tool allow list. The allow list contains only read_file, write_file, and search_code. How was the agent able to invoke a disallowed tool?
Select an answer to reveal the explanation.
Short Explanation
Putting an allow list in a system prompt is like writing 'employees only' on a sticky note on an unlocked door — it's guidance, not a lock. When the allow list is text in a prompt rather than an enforced runtime filter, the model can reason its way around it. You need to enforce allow lists at the tool dispatch layer, not in the prompt.
Full Explanation
The agent was able to invoke the disallowed tool because the allow list was configured in the agent's system prompt as a text instruction rather than as an enforced runtime constraint, allowing the model to override it.
There is a critical architectural distinction between an allow list as a prompt instruction and an allow list as an enforced runtime constraint. Prompt-based allow lists (e.g., 'You may only use the following tools: read_file, write_file, search_code') are soft instructions that the model may follow most of the time but can override when it reasons that a different tool is needed. The model is not bound by its own system prompt at the execution layer.
Properly enforced allow lists operate at the agent runtime or tool dispatch layer: when the model requests a tool call, the runtime checks the call against the allow list before dispatching. If delete_file is not in the enforced allow list, the runtime rejects the call and returns an error to the model regardless of how the model reasons about it.
The explanation that the LLM hallucinated the tool invocation and the MCP server ignored the allow list mischaracterizes how tool calls work. Tool calls are not hallucinated — they are structured outputs that the runtime actually dispatches. If the call was executed, the runtime accepted it.
The explanation that delete_file was implicitly included because it shares a namespace with write_file is not a real mechanism. Tools in MCP and similar frameworks are registered individually; sharing a name prefix does not create implicit inclusion.
The explanation that the MCP server's tool schema exposed delete_file, which the agent automatically discovered and added to its available tool set, describes tool discovery from a server schema, which is a real feature — but adding tools to the available set requires explicit registration in the client configuration, not automatic inclusion at invocation time.
Allow lists must be enforced as runtime constraints, not relied upon as prompt instructions.