A data science team builds a model to flag fraudulent credit card transactions. Only about half a percent of transactions in the historical dataset are actually fraudulent. A junior analyst reports that the model is excellent because it is 99.4 percent accurate, but closer inspection shows the model labels every single transaction as legitimate and never flags fraud at all. Which evaluation approach would have revealed the problem that overall accuracy hid?
Select an answer to reveal the explanation.
Short Explanation
Here's the trap the junior analyst fell into: when almost every transaction in the dataset is legitimate, a model can rack up a great-looking accuracy score just by guessing the majority class every single time, even if it catches zero fraud. The fix isn't piling on more data of the same imbalanced mix, since that keeps the trap intact. It also isn't swapping in a different kind of model built for predicting numbers instead of yes-or-no outcomes, since the task here is fundamentally a category decision, not a numeric one. And trimming down the number of inputs the model learns from doesn't touch how the result gets judged at all. What actually exposes the problem is looking specifically at how many of the real fraud cases got caught, rather than trusting one overall percentage that can be dominated by the easy, common case.
Full Explanation
The correct answer is B. Because fraudulent transactions make up only about half a percent of the dataset, a model that never predicts fraud can still score 99.4 percent accuracy simply by matching the dominant legitimate class, which is a classic symptom of class imbalance hiding a model's real failure. Reviewing recall, the share of actual fraud cases the model correctly catches, would immediately show a value of zero and expose that the model is not detecting fraud at all, which overall accuracy cannot reveal on such a skewed dataset. Option A is incorrect because simply adding more transactions without changing the underlying ratio of fraud to legitimate cases would still leave fraud a tiny minority, so accuracy would remain misleadingly high and the same blind spot would persist. Option C is incorrect because the task is a yes-or-no decision about whether a transaction is fraudulent, which is a classification problem by nature, and regression is designed to predict continuous numeric values rather than category labels, so it is not a suited technique for this task, let alone a fix for the evaluation blind spot. Option D is incorrect because reducing the number of features affects what information the model has available to learn from, but it does not change how the model is being evaluated, so a model trained on fewer features could still be judged solely by an accuracy score that hides its complete failure to catch fraud. Only examining a metric like recall exposes what accuracy conceals.