Meridian's computer-vision system for detecting foreign-object debris on the ramp reports 96% precision and 91% recall in lab testing. Operations leadership instead wants to know whether the system is reducing ramp safety incidents. What does this situation illustrate about model evaluation?
Select an answer to reveal the explanation.
Short Explanation
Precision and recall tell you how good the model is at spotting debris in a lab. Incident counts tell you whether that translated into a safer ramp. You need both gauges on the dashboard, not just the one that's easiest to compute.
Full Explanation
A comprehensive evaluation plan, as CPMAI frames it, deliberately separates technical KPIs — model-level measures like precision and recall that assess how well the algorithm performs its narrow task — from business KPIs, which measure whether that technical performance translates into the outcome the organization actually cares about, here a reduction in ramp safety incidents. Both belong in the evaluation plan because strong technical numbers do not guarantee the operational benefit materializes; incidents could still occur due to response-time gaps, alert fatigue, or coverage blind spots the lab metrics don't capture. Treating precision/recall as the only relevant metrics ignores that a model can be technically excellent yet operationally inert if its outputs aren't acted on effectively. Ignoring incident data until perfect precision is reached sets an unrealistic bar (100% precision is not a realistic target for any vision system) and needlessly delays learning from real-world deployment signal. Assuming technical and business metrics always move together is the exact assumption CPMAI's KPI-alignment guidance warns against — they must be tracked and reconciled deliberately, not presumed to correlate.