A bank builds a model to flag transactions as fraudulent, and every falsely flagged legitimate transaction locks out a real customer and triggers an expensive manual review. Which metric should the team prioritize to keep those false alarms as low as possible?
Select an answer to reveal the explanation.
Short Explanation
This is a false-positives-are-expensive situation, plain and simple. Every time a legit customer gets flagged as fraud, that's a false positive, and the bank pays for it in customer annoyance and manual review costs. The metric built exactly for this concern is precision — it asks, of everything I flagged as fraud, how much of it was actually fraud? High precision means very few of your fraud alerts are false alarms. That's answer D. Recall is the wrong tool here because recall is about the opposite worry, missing actual fraud cases (false negatives), which matters too but isn't the pain point described. Overall accuracy would just get inflated by the huge number of correctly-classified legitimate transactions and wouldn't tell you anything about the false alarm rate specifically. And mean squared error is a regression metric, it doesn't even apply to a flag-or-don't-flag classification problem like this one.
Full Explanation
Precision is defined as true positives divided by the sum of true positives and false positives, meaning it measures, out of everything the model labeled positive (in this case, flagged as fraudulent), what fraction was genuinely positive. When false positives carry a high cost — such as locking out legitimate customers and triggering unnecessary manual reviews, as described here — precision is the metric that directly measures and rewards minimizing exactly that kind of error. A model with high precision rarely raises a false alarm, even if it occasionally misses some fraud cases in the process.
Recall is incorrect as the priority metric here because it measures true positives divided by true positives plus false negatives, meaning it captures how many actual fraud cases were caught, which is the relevant concern when missing a positive case (a false negative) is the expensive mistake, such as in medical diagnosis where a missed disease case can be catastrophic. That is a different cost profile from the one described in this question, where the pain point is specifically false alarms on legitimate transactions, not missed fraud. Overall accuracy is incorrect because it aggregates correct predictions across both classes and, especially since legitimate transactions vastly outnumber fraudulent ones in most real-world datasets, it would remain high regardless of how many false fraud alerts were generated, masking the exact problem the bank cares about. Mean squared error is incorrect because it is a regression loss metric that measures the average squared difference between predicted and actual continuous values; it has no direct meaning for a binary fraud/not-fraud classification decision.
The general principle worth remembering: choose precision when false positives are the costly mistake (fraud false alarms, spam filters wrongly blocking real email), and choose recall when false negatives are the costly mistake (missed disease diagnoses, missed actual fraud). Many production systems track both together via the F1 score or examine the full precision-recall tradeoff curve to choose an operating threshold that matches the real-world cost asymmetry.