An agent for generating database migrations is frequently producing incorrect output. A developer reviews the logs and concludes the model is reasoning incorrectly about the schema, so they rewrite the reasoning instructions in the system prompt. After the change, the failures persist at the same rate. A deeper log review later reveals the agent was calling a get_schema tool with the wrong parameter format, receiving empty results, and then reasoning about an empty schema. What was the misdiagnosis?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Blaming the navigator when the GPS is sending wrong coordinates — that's what happens when a tool-use error gets diagnosed as a reasoning error. The model was reasoning fine; it just wasn't getting any real data to reason about because its tool call was malformed. The fix has to target the root cause: the tool invocation, not the thought process.
Full explanation below image
Full Explanation
Failure classification in agentic systems requires distinguishing between two categories of errors:
1. Reasoning errors: the model receives correct inputs (context, tool results) but draws incorrect conclusions, applies wrong logic, or makes faulty decisions based on available information.
2. Tool-use errors: the model constructs a tool call incorrectly (wrong parameter format, missing required fields, wrong tool selected), resulting in errors or empty/incorrect tool results that then feed into subsequent reasoning.
In this scenario, the agent received empty schema results because its tool call parameter format was wrong. The subsequent reasoning — about an empty schema — may have been internally consistent but was reasoning about nothing. Rewriting the reasoning instructions cannot fix a problem that occurs before the model even has data to reason about.
The diagnostic approach for distinguishing these categories: examine the full trace including tool call inputs and outputs before the model's reasoning step. If tool results are empty or erroneous, the root cause is in the tool invocation layer. If tool results are correct but the reasoning about them is wrong, the root cause is in the reasoning layer.
Option A (instructions too long to be followed) is a distractor about prompt design, not about the actual failure mechanism.
Option C (tool bug returning empty results regardless) would have been caught in tool unit tests and would manifest as consistently empty results for all inputs, not specifically for the agent's malformed parameter format.
Option D (insufficient training data) is a model capability claim that would require broad evidence across many scenarios, not a diagnosis for a specific recurring failure with a specific tool call pattern.