An emergency-management agent's evaluation suite checks whether it invokes the correct alerting API with correctly formatted parameters, as a step separate from scoring whether its natural-language summary reads well. Why does the team keep these two checks separate rather than folding tool-invocation correctness into the general quality score?
Select an answer to reveal the explanation.
Short Explanation
A press release can read beautifully and still go out with the wrong phone number attached. Checking the alerting-API call separately from the summary's prose is how the team catches that gap — a fluent summary tells you nothing about whether the underlying alert was actually placed correctly.
Full Explanation
Tool-invocation correctness and output-quality scoring measure genuinely independent properties of an agent's behavior: a language-quality score judges the readability and coherence of the generated summary, while a correct-invocation check verifies the underlying alerting API was called with the right name and correctly formatted parameters, and neither one implies the other — an agent can write a fluent summary while calling the wrong endpoint or malformed arguments, which is precisely the failure mode that matters most for an emergency-management system. Keeping the checks separate preserves the ability to catch that combination instead of letting a high-quality summary mask a broken alert call. There is no barrier to scoring natural-language quality for sensitive content; sensitivity affects what the agent should say, not whether its prose can be evaluated. Tool-invocation checks are not development-only scaffolding to be dropped at production — production is exactly where a bad alerting call has the highest consequence, making the check more important, not less. Separating the checks is a design choice within a single evaluation approach, not something that forces a different framework. Scope caveat: correct invocation alone doesn't guarantee the alert reached its recipients, since delivery failures can occur downstream of a well-formed API call. Operational check: for a flagged low-quality summary, confirm the tool-invocation log shows the alerting API call succeeded independently, before assuming the alert itself failed.