Meridian Line wants its customs-compliance assistant to hand off to a human broker whenever the retrieved filings do not actually support an answer. The application needs to detect that condition programmatically rather than by reading the prose. What is the cleanest prompt-level design?
Select an answer to reveal the explanation.
Short Explanation
When you need your application to make a decision, give it a field to read rather than prose to interpret. A required sentinel such as INSUFFICIENT_EVIDENCE turns uncertainty into a branch your code can test and use to route the case to a broker.
Full Explanation
Applications need machine-readable signals, so make the signal part of the required output contract: require a structured response with an explicit field the model sets when retrieved context does not support an answer, then branch on that field to route the case to a human broker. Because the sentinel is part of the schema, you can test it on an evaluation set and measure false-positive and false-negative rates. Scanning answers for hedging phrases targets a similar signal, but is brittle: hedging vocabulary is unbounded, confident hallucinations may contain no hedges, and the parser can miss the cases that matter most. Comparing generated answers with filings by text similarity measures lexical overlap, not whether the filings support the answer; a correct paraphrase may score low, while a fluent fabrication using retrieved vocabulary may score high. Having a broker review every response discards the automation the assistant is meant to provide and does not scale. The exam caveat is that refusal and escalation should be designed behaviours with an explicit representation; a retrieval-side similarity threshold can complement that signal, but should not replace it or be treated as proof of support. As a practical check, evaluate the sentinel against supported and unsupported examples, then measure its false-positive and false-negative rates.