A product manager pilots a roadmap-summarization custom agent used by five teams and needs to verify what sources it accessed, which actions it took, and whether outputs met quality bars before wider rollout. The agent runs inside GitHub with run histories and logs. Which practice should the manager use to review custom agent activities effectively?
Select an answer to reveal the explanation.
Short Explanation
Don’t judge the show by applause alone. Auditing run logs reveals what your agent touched and how well it performed.
Full Explanation
The correct practice is right because GitHub run histories and activity logs together answer all three pilot questions: accessed files and data sources, sequence of tool calls and actions, plus outputs with citations and errors to judge quality trends before scale. Analyzing dashboard metrics and team satisfaction ratings is useful for perceived quality but fails on provenance because ratings provide no verifiable record of which sources were accessed or which actions were taken. Exporting run histories to confirm accessed repositories and tools is strong for scope governance but fails on outcome validation because successful execution does not prove summaries were accurate, complete, or well cited. Comparing sampled outputs to source roadmaps to validate citation accuracy is strong for quality assurance but fails on behavioral audit because matching text to sources does not reveal the actual tool path, retries, or hidden errors. Exam caveat: when the stem lists sources, actions, and quality plus GitHub logs, eliminate choices covering only two pillars even if they sound mature and data-driven. Practical check: open a recent run, list each tool call with inputs and cited source, score the summary against a rubric, and track error patterns to guide rollout fixes.