Meridian's predictive-maintenance model scores 98% accuracy on the training data used to build it, but when engineers apply it to sensor readings from aircraft it has never analyzed, its false-alarm rate spikes and it misses real bearing failures. What is the most likely explanation, and what should the PM do?
Select an answer to reveal the explanation.
Short Explanation
Overfitting is a model that aced the practice test by memorizing the answer key instead of learning the material — great on the aircraft it trained on, lost on anything new. The fix isn't to shrug it off, it's to make the model learn general patterns, not fleet-specific quirks.
Full Explanation
High training accuracy paired with poor performance on unseen data is the textbook signature of overfitting: the model has captured noise and aircraft-specific idiosyncrasies from the training fleet rather than the general relationship between sensor readings and bearing wear. The correct PM response is to push the data science team toward regularization techniques, a more diverse and representative training set, and a fresh validation pass before any redeployment decision. Underfitting is the opposite failure — a model too simple to capture the pattern at all, which would show poor performance on the training data too, not 98% there. Simplifying further would make this worse. Blaming sensor miscalibration without evidence dodges the real diagnostic signal; the pattern described (great on training, bad on new data) is a model-generalization problem, not necessarily a hardware problem, and assuming it away skips the investigation a PM should require. Treating 98% training accuracy as proof of readiness ignores exactly what validation data is for — this is the central reason CPMAI stresses model validation as a gate before deployment, not a formality.