After three months in production, evaluation data shows Meridian's customer-service virtual assistant correctly resolves rebooking requests but frequently mishandles baggage-status questions, escalating them to human agents far more often than necessary. What is the appropriate CPMAI-aligned response?
Select an answer to reveal the explanation.
Short Explanation
Model evaluation isn't a one-and-done report card — it's a feedback loop. You find the weak spot, you retrain that specific piece, you check it worked, and you roll it forward. That's iteration, not demolition.
Full Explanation
CPMAI treats model evaluation and maintenance as an ongoing, iterative cycle: findings from production evaluation feed back into targeted improvement, not a binary pass/fail verdict. Here, the finding is specific — baggage-status handling is weak while rebooking is strong — so the correct response is to target that specific intent for retraining or fine-tuning, then re-validate the improvement before expanding the change to full production traffic. Scrapping the entire assistant discards a component that is working well (rebooking) and is a disproportionate response to a localized weakness; iteration is meant to avoid exactly this kind of costly overcorrection. Ignoring the finding because rebooking works elsewhere means one weak capability continues degrading customer experience and burdening human agents unnecessarily — evaluation findings exist to be acted on. Permanently removing baggage-status handling abandons the capability rather than improving it, which forgoes an achievable fix and is not what the evaluation data supports; the data points to a fixable gap, not an unsolvable one.