A county permitting-office team is assembling a golden dataset for their permit-application chatbot. Most prompts are straightforward status-check questions, but the team adds a smaller set of deliberately ambiguous prompts asking about multiple permit types at once. What makes those ambiguous multi-permit prompts a valuable addition to the golden dataset rather than a redundant variation on the status-check prompts?

A golden dataset earns its value from the failure modes it can surface, not from raw prompt count. A straightforward status-check prompt tests retrieval and formatting against a single, unambiguous target, so a dozen variations of it mostly re-test the same capability. The ambiguous multi-permit prompt adds a genuinely different task: the agent must first recognize that the request spans more than one permit type, then either ask a clarifying question or handle both intents correctly, which exercises intent disambiguation logic the happy-path prompts never touch. Counting total prompts as a quality signal confuses volume with coverage — a large set of near-duplicate happy-path items can still miss entire classes of real-world confusion. Treating the ambiguous prompt as a substitute for testing each permit type individually conflates two different concerns: intent disambiguation and per-type retrieval correctness both need their own coverage. Harder automatic scoring is a side effect of ambiguity, not the reason to include it — the reason is the behavior it uniquely tests. Scope caveat: edge cases should be a deliberate minority of the set, since an evaluation suite dominated by ambiguous prompts loses its ability to catch basic regressions. Operational check: tag each golden prompt by the capability it exercises and confirm disambiguation has its own tag distinct from simple retrieval.