A product manager supervises a deployed triage agent that labels incoming GitHub issues overnight and assigns owners. Stakeholders report missed SLAs, duplicate labels, and unexplained reassignments this week. Which supervision practice should the manager use to monitor agentic workflow runs and restore reliable triage?
Select an answer to reveal the explanation.
Short Explanation
You're reading the flight recorder, not just restarting the plane. Careful run observability reveals why labels slipped. It's like checking game film to fix missed plays.
Full Explanation
Reliable supervision of a deployed triage agent demands joint review of run history, logs, tool calls, and labeled outcomes because missed SLAs, duplicate labels, and unexplained reassignments point to different workflow layers that only trace-plus-outcome inspection can connect to support precise rule refinement. Auditing aggregate SLA compliance and staffing counts is sound for capacity planning but stays at metric level without exposing per-run tool behavior, so duplicate-label logic and faulty reassignments remain undiagnosed. Reviewing labeled samples with owner feedback improves taxonomy and guidelines yet evaluates outputs only, leaving scheduling, tool invocation, and error paths unmonitored. Enabling real-time alerts for failures and reassignments aids rapid human response but provides reactive detection rather than systematic diagnosis, so underlying patterns persist across overnight runs. For exams, distinguish monitoring that explains and fixes workflow causes from alerting that notifies, scaling that adds resources, and output review that refines labels, selecting trace-plus-outcome coverage when the stem lists diverse failures. Practically, filter overnight runs for SLA breaches, duplicates, and owner changes, then reconcile each log and tool-call sequence with expected triage policy before updating rules.