An AI model for supply chain disruption prediction is retrained quarterly using recent data. After the latest retraining, the model's predictions become significantly more conservative, flagging 40% fewer disruptions than the previous version. Before promoting the new version, what validation step is MOST important?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — conduct back-testing on historical periods with known disruptions to assess whether the new version would have correctly flagged those events. A 40% drop in flags could mean fewer false positives or missed real disruptions, and back-testing tells you which one you've got.
Full Explanation
Conducting back-testing on historical periods with known disruptions to assess whether the new version would have correctly flagged those events is correct because a 40% reduction in disruption flags could indicate either a genuine improvement (fewer false positives) or a dangerous regression (missing real disruptions). Back-testing on historical periods with known disruptions is the definitive test: if the new model fails to flag events that clearly occurred, it has regressed and should not be promoted. Verifying that the model was retrained using the organization's approved cloud computing infrastructure is an operational compliance check, not a model quality check. Surveying supply chain managers to assess their qualitative confidence in the new model version provides qualitative confidence but cannot replace empirical back-testing. Comparing the new version's training data volume to the previous version to ensure more data was used is a necessary condition but not sufficient to explain or validate the behavior change.