Meridian's cargo and passenger demand-forecasting model achieves strong statistical accuracy (low mean absolute error) in testing, but the finance team reports that pricing and capacity decisions based on its forecasts have not improved route profitability. What should the PM conclude?
Select an answer to reveal the explanation.
Short Explanation
A model can nail the statistics and still miss the point if the business never asked for low error, it asked for better pricing decisions. Evaluating a model only on its own technical scorecard is grading the wrong test.
Full Explanation
This scenario is the core distinction CPMAI draws between technical KPIs and business KPIs: a model can hit strong statistical marks (low error) while failing to move the actual business outcome it was commissioned to improve. The PM's job is to recognize that gap and revisit the evaluation criteria so they explicitly tie back to route profitability, not just forecast error, and investigate whether the forecasts are being translated into pricing/capacity decisions correctly, whether the error is concentrated on the routes that matter financially, or whether the KPI itself needs redefinition. Abandoning the model on the basis of low error being 'meaningless' overcorrects — the error metric still has diagnostic value, it is simply insufficient alone. Dropping statistical accuracy entirely from evaluation would remove a legitimate technical signal; the issue is that it must be paired with business measurement, not replaced. Dismissing finance's complaint because the model 'passed' its technical evaluation is exactly the failure mode CPMAI warns against — a model that never demonstrates business value is a project failure regardless of its accuracy score.