A fraud detection model is trained on data where 99% of transactions are legitimate. The model predicts 'not fraud' for every transaction and achieves 99% accuracy. Why is accuracy a misleading metric here?
Select an answer to reveal the explanation.
Short Explanation and Infographic
A model that always says 'not fraud' beats 99% accuracy without catching a single fraudster — that's why we use precision, recall, and F1 for imbalanced problems. Accuracy just tells you the majority class won.
Full explanation below image
Full Explanation
On imbalanced datasets, accuracy = (TP + TN) / (TP + TN + FP + FN) is dominated by the majority class (legitimate transactions). A naive classifier that always predicts the majority class achieves accuracy equal to the majority class proportion — 99% here. But its recall for fraud = TP / (TP + FN) = 0 / (0 + all_frauds) = 0%. Better metrics for imbalanced classification: Precision = TP / (TP + FP) — of predicted fraud, how many are actual fraud. Recall (Sensitivity) = TP / (TP + FN) — of actual fraud, how many did we catch. F1 Score = 2 (Precision Recall) / (Precision + Recall) — harmonic mean. AUC-ROC or AUC-PR — threshold-independent measures. In Databricks/Spark, use BinaryClassificationEvaluator with metricName='areaUnderROC' for imbalanced problems.