An airline's rebooking agent writes a trace whenever a tool call fails, and nobody reviews those traces. The operations board shows only how many runs started. What should operations monitor?
Select an answer to reveal the explanation.
Short Explanation
A stack of failed call slips does no good in a drawer nobody opens. You should read whether each tool call succeeded or failed, along with how many runs started. A chat score still measures a conversation this rebooking agent does not have.
Full Explanation
An agent that calls tools is judged by whether those calls worked, not only by how many times it woke up. The rebooking agent already writes a trace when a tool call fails, so the first monitoring gap is that nobody reads tool success and failure on each run. A board that shows only the count of runs started can look healthy while every rebooking call is failing underneath it. A thumbs-up rate needs a passenger chat transcript, and this agent does not produce one, so the score stays empty while failed tool calls pile up. The character count of the system prompt is an authoring detail, and a longer instruction does not tell you that yesterday's ticket tool returned an error. Run volume can stay on the board as context. It is not a substitute for the trace. Exam caveat: tool success is a runtime signal, separate from how satisfied a traveler felt on a different channel. Operational check: open yesterday's rebooking traces, mark each run as tool success or tool failure, and confirm the operations review includes that split rather than the run count alone.