A bank deploys an AI fraud detection model. After six months, the fraud team notes that the model's flagging rate has dropped by 40% while actual fraud losses have increased by 15%. Model performance metrics (AUC, precision) appear stable on the monitoring dashboard. How should this apparent contradiction be explained and addressed?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because this pattern is consistent with adversarial concept drift: fraudsters have studied the model's detection patterns and adapted their behavior to evade detection. Standard monitoring metrics (AUC, precision) may not detect this if the evaluation labels still come from the same (now-evaded) detection process.
Full explanation below image
Full Explanation
B is correct because this pattern is consistent with adversarial concept drift: fraudsters have studied the model's detection patterns and adapted their behavior to evade detection. Standard monitoring metrics (AUC, precision) may not detect this if the evaluation labels still come from the same (now-evaded) detection process. This creates a 'blind spot' where the model's apparent performance is stable while its real-world effectiveness is declining. This requires active adversarial testing, not just threshold adjustment (C) or data augmentation (D). Simply accepting the dashboard metrics (A) ignores the business evidence of increasing losses.