The customs-compliance agent calls multiple tools in sequence, an HS-code lookup, a tariff-rate lookup, and a final summarization step, and the team wants an evaluation approach purpose-built for scoring this kind of multi-step, tool-calling agent rather than a single-turn question-and-answer model. Which Databricks capability is designed for this?
Select an answer to reveal the explanation.
Short Explanation
A multi-step agent deserves a multi-step grade, not just a thumbs-up or thumbs-down on the last sentence it wrote. Mosaic AI Agent Evaluation is built to look inside the sequence, did it call the right tool, in the right order, with the right arguments, before it ever gets to scoring the final summary.
Full Explanation
Mosaic AI Agent Evaluation is purpose-built for evaluating agentic applications that reason and act across multiple steps, assessing not only the final output but also whether the agent selected the correct tools, called them with correct arguments, and reasoned appropriately at each intermediate stage, which is exactly the HS-code lookup, tariff-rate lookup, and summarization sequence described. Unity Catalog Information Schema queries expose metadata about registered objects, such as a model's schema or permissions, which is useful for governance and discovery but has no concept of scoring a sequence of tool calls or reasoning quality. Applying a standard classification accuracy metric to only the final text output is the closest-looking trap: it does produce some quality signal, but it treats the whole multi-step process as a black box, so an agent that called the wrong tool but stumbled into a correct-sounding final answer would score well, and a genuinely correct process interrupted by a formatting slip in the last step would score poorly, neither of which is useful diagnostic signal. Databricks SQL query history records executed queries against warehouses, an operational and auditing log for the Data Analyst surface, unrelated to agent behavior evaluation. Agent Evaluation is the capability for scoring this class of tool-calling, multi-stage-reasoning application; on current Databricks it is exposed through MLflow 3 GenAI evaluation (mlflow.genai.evaluate with built-in and custom scorers and LLM judges over traces), so you may see either name.