A multi-agent pipeline that generates API documentation produced an incorrect output three days ago. The pipeline included a research agent, a drafting agent, and a formatting agent. Logs exist but are unstructured. Audit artifacts were not created during the run. What limitation does this create for post-hoc analysis?
Select an answer to reveal the explanation.
Short Explanation
Trying to reconstruct a car accident without dash cam footage, witness statements, or a police report means you're guessing based on damage, not evidence. Without structured audit artifacts, you can't reconstruct which agent produced which portion or what intermediate decisions were made, so you need a full pipeline re-run with debugging enabled to approximate the original execution. On the exam, remember that missing audit artifacts make post-hoc analysis guesswork, which is why audit artifact creation is required in multi-agent pipeline design.
Full Explanation
The correct answer is the statement that without structured audit artifacts, it is impossible to reconstruct which agent produced which portion of the output or what intermediate decisions were made, requiring a full pipeline re-run with debugging enabled to approximate the original execution. Structured audit artifacts serve as the authoritative record of a pipeline run: they capture each agent's input, intermediate outputs, decisions made, and the final result in a structured, queryable format. Without them, post-hoc analysis is severely limited: the team cannot definitively determine which agent produced which content, cannot trace a specific incorrect claim back to its originating agent, and cannot distinguish whether the error occurred in the research phase, the drafting phase, or the formatting phase. Unstructured logs may capture some events but not the structured agent-attribution data needed for precise fault localization. The claim that the pipeline cannot be re-run until the logging infrastructure is fixed is incorrect because the pipeline's ability to re-run is unrelated to the logging infrastructure. The pipeline ran three days ago, so its infrastructure is presumably intact; the post-hoc analysis limitation is about missing historical evidence, not current operational capability. The claim that the formatting agent's output cannot be validated because the final artifact was already published is incorrect because validation does not depend on whether the artifact was published. The issue is the lack of intermediate audit data, not the published state of the final artifact; the final artifact exists and can be inspected. The claim that the root cause cannot be classified because no errors were logged, so the failure must be considered a model-level issue, is incorrect because the absence of error logs does not indicate a model-level issue, and it certainly does not make the failure unclassifiable. Silent correctness failures, meaning wrong output without errors, are common in agentic systems and can be attributed to instruction, context, or tool failures, but without audit artifacts you cannot determine which. Treating the failure as necessarily a model issue is a premature and incorrect conclusion.