A team must expand evaluation coverage for a multi-agent benefits advisor before open enrollment. Real labeled production transcripts are scarce because of privacy constraints. They need diverse scenarios (edge cases, adversarial phrasings, multi-turn tool paths) to test prompts, tools, and memory. Which continuous-improvement technique best addresses the gap?
Select an answer to reveal the explanation.
Short Explanation
Privacy limits real transcripts, so you generate synthetic data—including edge and adversarial cases—under privacy-safe policies, then evaluate memory, tools, prompts, and knowledge systematically. Reusing three happy paths forever misses risk. Shipping blind is reckless, and one executive demo isn’t a test strategy.
Full Explanation
The correct technique is generating synthetic evaluation data (and adversarial variants) under privacy-safe policies, then running structured evaluations across memory, knowledge, tools, and prompts. Continuous improvement includes synthetic data generation to broaden evaluation coverage when real labeled data is limited. Evaluations should span memory, knowledge, tools, and prompts—the exam’s evaluation surface for multi-agent solutions—using privacy-safe synthetic and adversarial cases. Reusing the same three happy-path examples forever is incorrect because tiny static happy-path sets fail to detect regressions on edge and multi-turn tool paths. Shipping to production without evaluations until real incidents generate free training data is incorrect because waiting for production incidents is not an evaluation strategy. Evaluating only by asking the CEO to click through the demo once is incorrect because informal demo clicks lack coverage, repeatability, and scoring rigor. Exam focus: synthetic data enables scalable multi-agent eval under data constraints.