During Model Evaluation, Meridian's team discovers the generative-AI knowledge assistant for maintenance manuals gives confidently wrong answers because the underlying document set it was built from was missing entire chapters of the newest aircraft's maintenance manual. What is the correct CPMAI response?
Select an answer to reveal the explanation.
Short Explanation
This isn't a tweak-the-model problem, it's a missing-data problem — and CPMAI is built to send you back a phase when evaluation uncovers one. Go get the missing manual chapters, then come back and re-evaluate.
Full Explanation
CPMAI explicitly treats the six phases as iterative rather than strictly linear: the Model Evaluation and Maintenance task calls for determining when to iterate back to previous phases, and a root cause traced to incomplete source data — missing maintenance-manual chapters — is a Data Understanding/Data Preparation problem, not something a model-level fix can solve. The correct response is to loop back, source the missing chapters, incorporate them into the data used to build or ground the assistant, and then re-run evaluation. Retraining only output formatting does nothing about the actual knowledge gap causing confidently wrong answers — the model doesn't know what it was never given. Proceeding to Operationalization while relying on technicians to catch every wrong answer is dangerous for a maintenance-procedure assistant at an airline and defeats the purpose of the evaluation gate; it shifts safety risk onto already-burdened staff instead of fixing the root cause. Lowering the evaluation criteria to force a pass is exactly the kind of shortcut CPMAI's methodology exists to prevent — it optimizes for looking done rather than being ready, and on a safety-relevant tool the consequences of a bad answer are real. The exam point: iteration isn't limited to redoing Model Development — evaluation findings sometimes point all the way back to data-phase work.