An AI model for supply chain disruption prediction is retrained quarterly using recent data. After the latest retraining, the model's predictions become significantly more conservative, flagging 40% fewer disruptions than the previous version. Before promoting the new version, what validation step is MOST important?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because a 40% reduction in disruption flags could indicate either a genuine improvement (fewer false positives) or a dangerous regression (missing real disruptions). Back-testing on historical periods with known disruptions is the definitive test: if the new model fails to flag events that clearly occurred, it has regressed and should not be promoted.
Full explanation below image
Full Explanation
B is correct because a 40% reduction in disruption flags could indicate either a genuine improvement (fewer false positives) or a dangerous regression (missing real disruptions). Back-testing on historical periods with known disruptions is the definitive test: if the new model fails to flag events that clearly occurred, it has regressed and should not be promoted. Infrastructure verification (A) is an operational compliance check, not a model quality check. Stakeholder surveys (C) provide qualitative confidence but cannot replace empirical back-testing. Training data volume (D) is a necessary condition but not sufficient to explain or validate the behavior change.