After a multi-agent workflow completes, a developer wants to understand why Agent B produced unexpected output. The workflow system logs all agent inputs, outputs, tool calls, and timestamps. Which analysis approach is most effective for diagnosing the root cause?
Select an answer to reveal the explanation.
Short Explanation
Diagnosing an agent is like reading a black-box flight recorder. Start with what the agent received, then follow each decision through its tool calls and intermediate results until you find where the flight path went off course. Tracing inputs to tool calls to outputs is the most direct path to root cause.
Full Explanation
Post-hoc analysis of multi-agent behavior is essentially log-driven debugging. When one agent produces unexpected output, the diagnostic process follows a specific sequence through the available logs.
First, verify inputs: check what context the agent under review actually received. Was it given the correct outputs from the preceding agent? Were any required fields missing or malformed? Incorrect inputs from an upstream agent are the most common cause of unexpected downstream behavior.
Second, trace tool calls: review the sequence of tools the agent invoked, the arguments it passed, and the responses it received. One tool returning unexpected data, such as a search tool returning stale results, may have caused the agent to take a wrong turn.
Third, examine intermediate outputs: if the agent produced intermediate artifacts before its final output, compare them against expected behavior at each step to pinpoint exactly where divergence occurred.
The correct approach is the one that reviews the agent's input log to verify what context it received, then traces its tool calls and intermediate outputs to identify where behavior diverged from expectations. This is the canonical post-hoc analysis approach: start at inputs, trace the execution path, and identify the divergence point. It uses the data the logging system was designed to capture.
The re-run approach is wrong because re-running the entire workflow may not reproduce the original behavior if any inputs, tool responses, or model outputs are non-deterministic. Even if it does reproduce, it only confirms the problem exists; it doesn't explain why it happened.
The downstream-output approach is wrong because starting with the downstream agent's output is starting at the wrong end of the causal chain. The downstream agent may have masked, corrected, or compounded the original agent's error, making it harder to isolate the original cause.
The Git-diff approach is wrong because reviewing the Git diff shows what the agent changed but not why. The diff is the output artifact, not the execution trace. Without understanding the inputs and tool calls that produced the diff, you cannot determine root cause from the diff alone.