Meridian Line's customs-compliance agent has a curated set of past queries with hand-verified correct HS-code answers. The team wants to compare two candidate prompt versions on this set and see, for each version, the fraction of answers matching the verified correct code exactly. Which evaluation approach fits this specific comparison?

When a curated set of queries already has hand-verified correct answers, the natural and most precise evaluation is a reference-based metric that directly compares each generated response's stated answer against the known-correct value, here whether the stated HS code matches the verified code exactly, run identically against both candidate prompt versions on the same set so the comparison is apples to apples. LLM-as-judge is valuable precisely when a reference answer is not available, having a hand-verified reference for every item in the set makes a judge model an unnecessary, less precise substitute for a direct comparison that can be computed exactly, so 'reference available' argues against, not for, defaulting to judge-based scoring here. A retrieval recall@k metric evaluates whether relevant passages were retrieved, not whether the final generated answer matches a verified correct code, and the task described is specifically about comparing the final stated answer, not retrieval quality on its own. Inference-table analysis is built around observing live production traffic after the fact, it is not the tool for a controlled offline comparison of two prompt versions against a fixed, curated evaluation set. Exam framing: choose a reference-based metric when verified answers exist for the evaluation set, and reserve LLM-as-judge for cases where they do not.