A major insurer uses an AI model to detect claims fraud. The model's recall for actual fraudulent claims is 72%, meaning 28% of real fraud goes undetected. The insurer's leadership argues this is acceptable because 'no model is perfect.' What should the AI risk manager consider when evaluating this performance level?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because risk management of a fraud detection model requires evaluating three dimensions: the materiality of the missed fraud (financial exposure from 28% undetected fraud), whether model improvements are feasible, and whether the miss rate is consistent across demographics (some groups being missed at higher rates would be a fairness issue). Simply accepting 72% recall as 'good enough' (A) without this analysis is insufficient.
Full explanation below image
Full Explanation
B is correct because risk management of a fraud detection model requires evaluating three dimensions: the materiality of the missed fraud (financial exposure from 28% undetected fraud), whether model improvements are feasible, and whether the miss rate is consistent across demographics (some groups being missed at higher rates would be a fairness issue). Simply accepting 72% recall as 'good enough' (A) without this analysis is insufficient. Targeting 100% recall (C) would cause operationally unacceptable false positives. Regulatory reporting (D) may be appropriate for material known issues but is not the first step.