A program manager oversees three custom support agents that summarize threads, update fields, and comment in GitHub. Leadership wants proof of correct behavior, error trends, and prompt improvements. Which review practice best evaluates custom agent activities?
Select an answer to reveal the explanation.
Short Explanation
Great coaches watch the game tape. You'll improve fast by auditing run logs weekly to see what your agents actually did and where prompts need tweaks.
Full Explanation
The correct practice fits because three agents doing summarization, field updates, and GitHub commenting require proof tied to actions, and only weekly review of run logs plus outputs provides auditability of what ran with which inputs, a consistent sample for error trends, and concrete failures to refine instructions. Monitoring weekly aggregate success rates and dashboards to track trends and report overall performance remains useful management reporting but fails by traceability, since rates show movement without linking a miscount to the thread, field, prompt step, or tool call that caused it, so proof and prompt fixes are impossible. Reviewing configuration versions and prompt drafts weekly to standardize instructions across teams supports governance but fails by intent versus execution, checking design rather than production behavior and missing actual errors. Sampling completed GitHub comments and field updates weekly to verify output quality and coach tone improves quality but fails by output-only review, lacking run context to separate prompt flaws from retrieval or tool errors. On exams, when the stem asks for proof, trends, and improvement together, favor log-plus-output review over metrics-only, design-only, or sample-only work even if each sounds professional. Practically, open last week history, verify two summaries, one update, and one comment against sources, label cause, and apply one fix to instructions.