You need a continuous improvement loop for a multi-agent sales assistant. Human review capacity is limited, but you must detect quality regressions in grounded product claims weekly. Which approach aligns with AI-500 continuous improvement practices?
Select an answer to reveal the explanation.
Short Explanation
You need automated weekly checks that score grounded claims with LLM-as-a-judge on fixed and synthetic scenarios, then alert you when scores regress. Combine that with user feedback and semantic optimization so failures feed prompt and tool fixes. Annual-only embedding refreshes, no logging, and unmeasured full production prompt pushes leave you blind or risky.
Full Explanation
The correct approach is to optimize prompt and model performance through continuous improvement: LLM-as-a-judge, synthetic scenarios, semantic optimization, and user feedback. For sales grounding, define rubrics such as faithfulness to the catalog and no inventing SKUs, run judges on fixed/gold and synthetic cases, and page owners when scores drop. The annual-only embedding retrain approach is incomplete: embedding refresh matters for RAG but is not a substitute for ongoing multi-agent quality evaluation. The disable-all-logging approach is wrong because observability is required to detect and explain regressions. The ship-prompt-changes-directly-to-100%-of-production-traffic approach is wrong because unmeasured production experiments risk brand and compliance damage; use canaries and gated rollouts with evaluation. Keep human review for high-severity edge cases while judges cover volume.