A permitting bot returns fluent but wrong zone codes even though traces show correct tool data returned. What failure type fits?

A permitting bot that returns fluent but wrong zone codes while traces show correct municipal tool data returned is exhibiting a model-output failure: the tools succeeded, yet the model ignored or mishandled the grounded results when composing the resident-facing answer. Isolating that failure type prevents chasing the wrong system and focuses remediation on grounding instructions, answer templates, or post-checks against tool results.

Diagnosing a tool-schema mismatch blocking proper zone code extraction fails because the traces show the municipal tool returned correct data, so the schema allowed extraction; the problem is the model ignoring that grounded result when composing its answer. Checking whether the tool silently returned a cached value instead of the actual zone fails to explain this case because the traces already show the tool returning correct, current zone data—the discrepancy appears only once the model composes its answer, after a valid tool result. Attributing the error to retrieval recall loss across ordinance documents fails because recall loss would prevent the ordinance data from reaching the tool, yet the traces show correct tool data was returned and the model simply miswrote the final answer.

Exam caveat: fluent wrongness is especially dangerous in civic permitting—residents may act on a wrong zone code even when backend data was correct. Operational check: add an assertion that the final zone code in the assistant message must equal a value present in the latest tool result, and fail closed to escalation when the strings diverge.