A water utility work-order agent must prove it finishes tasks and uses tools correctly, not merely that it returns text. Which evaluation focus is most appropriate?

Measuring task-completion rate and tool-success rate for the work-order assistant is the most appropriate evaluation focus when a water utility agent must prove it finishes tasks and uses tools correctly, not merely that it returns text. Agent usefulness is defined by whether work orders progress and whether tool calls (status updates, asset lookups, scheduling) succeed; those rates expose brittle tool use that polite prose can hide. Evaluating agents as task systems aligns metrics with field operations outcomes. Tracking only the font size of generated status emails fails by concept: presentation styling is unrelated to task or tool success. Ignoring tool failures if the agent’s prose remains polite fails by concept—politeness does not close work orders or correct failed API calls. Evaluating only offline classical ARIMA forecasts of pipe diameter fails by concept: an unrelated forecasting metric does not assess the agent’s tool-using work-order behavior. Exam caveat: agent evaluation privileges task and tool success over text cosmetics or unrelated classical models. Operational check: define successful work-order completion criteria, log each tool call outcome, compute completion and tool-success rates by job type, investigate polite-but-failed traces, and include those rates in the water utility’s AI ops dashboard beside latency.