A historic millstone-shop agent is judged on whether it actually reserved the demo and sent the email, not on how pretty the confirmation paragraph is. Which measures fit?
Select an answer to reveal the explanation.
Short Explanation
Think of whether the millstone shop actually reserved the demo and sent the email. Task-success and tool-success measure those actions. A pretty confirmation paragraph does not book the slot.
Full Explanation
A millstone-shop agent judged on whether it reserved the demo and sent the email needs task-success and tool-success measures of completed actions. Those metrics ask whether the intended side effects finished through tools and workflows, not whether the confirmation paragraph reads elegantly. For agentic municipal or community systems, completed reservations and delivered messages are the product residents experience—not decorative confirmation copy.
Prose beauty alone cannot reserve a demo slot on the shop calendar. ROUGE overlap with a sample email does not book the appointment or prove delivery. A writing benchmark scores fluency; it does not confirm the confirmation was actually sent or that the reservation API succeeded for the visitor.
Exam caveat: agents can narrate success while tools fail. Always verify tool outcomes separately from generated text that claims the work is done, especially when the stem contrasts pretty paragraphs with real bookings.
Operational check: instrument reservation and email tools with success and failure logs, define task-success as “demo booked and message delivered,” and reject releases where narrated confirmations exceed verified tool completions for the millstone-shop agent.