After launch, you need continuous improvement for a multi-agent assistant: detect quality regressions, generate hard test cases, and fold user thumbs-down signals into optimization. Which strategy best fits AI-500 continuous improvement practices?
Select an answer to reveal the explanation.
Short Explanation
Continuous improvement is a loop: LLM-as-judge scores, synthetic tough cases, user feedback into datasets, then prompt/tool optimization with release gates. Option B. Never re-evaluating (A) lets silent regressions through. Random prod prompt edits (C) are chaos. Turning off telemetry (D) blinds the loop.
Full Explanation
Continuous improvement skills include LLM-as-a-judge frameworks, synthetic data generation, semantic optimization loops, and user feedback loops. Option B combines these into a controlled production learning system.
Option A ignores drift and regression.
Option C changes behavior without scientific comparison.
Option D removes the data needed for optimization and incident response.
Pair automated judges with periodic human calibration; keep synthetic data labeled and diverse.
Exam tip: Measurement precedes optimization — judges and feedback before prompt rewrites.