After launch, you need continuous improvement for a multi-agent assistant: detect quality regressions, generate hard test cases, and fold user thumbs-down signals into optimization. Which strategy best fits AI-500 continuous improvement practices?
Select an answer to reveal the explanation.
Short Explanation
You need a continuous improvement loop: use LLM-as-a-judge scoring, synthetic hard cases, and user feedback to build datasets, then optimize prompts/tools with gated releases. Never re-evaluating, random prompt edits, or disabling telemetry breaks the loop and hides regressions.
Full Explanation
Continuous improvement skills include LLM-as-a-judge frameworks, synthetic data generation, semantic optimization loops, and user feedback loops. The correct strategy combines these into a controlled production learning system.
Never re-evaluating after launch ignores drift and regression. Randomly rewriting system prompts in production changes behavior without scientific comparison. Disabling tracing and feedback collection removes the data needed for optimization and incident response.
Pair automated judges with periodic human calibration; keep synthetic data labeled and diverse.
Exam tip: Measurement precedes optimization — judges and feedback before prompt rewrites.