A data science team builds a model to flag fraudulent credit card transactions. Only about half a percent of transactions in the historical dataset are actually fraudulent. A junior analyst reports that the model is excellent because it is 99.4 percent accurate, but closer inspection shows the model labels every single transaction as legitimate and never flags fraud at all. Which evaluation approach would have revealed the problem that overall accuracy hid?
Select an answer to reveal the explanation.
Short Explanation
You can get a 99.4 percent accuracy score by calling every transaction legitimate when fraud is rare. Look at recall, the share of real fraud cases caught, and you'll see it's zero. That exposes what overall accuracy hides.
Full Explanation
The correct approach is reviewing recall, the proportion of actual fraud cases the model correctly identifies. Because fraudulent transactions make up only about half a percent of the dataset, a model that never predicts fraud can still score 99.4 percent accuracy simply by matching the dominant legitimate class, which is a classic symptom of class imbalance hiding a model's real failure. Reviewing recall would immediately show a value of zero and expose that the model is not detecting fraud at all, which overall accuracy cannot reveal on such a skewed dataset. Increasing the size of the training dataset without changing its class balance is incorrect because simply adding more transactions without changing the underlying ratio of fraud to legitimate cases would still leave fraud a tiny minority, so accuracy would remain misleadingly high and the same blind spot would persist. Switching from a classification model to a regression model is incorrect because the task is a yes-or-no decision about whether a transaction is fraudulent, which is a classification problem by nature, and regression is designed to predict continuous numeric values rather than category labels, so it is not a suited technique for this task, let alone a fix for the evaluation blind spot. Reducing the number of features used to train the model is incorrect because reducing the number of features affects what information the model has available to learn from, but it does not change how the model is being evaluated, so a model trained on fewer features could still be judged solely by an accuracy score that hides its complete failure to catch fraud. Only examining a metric like recall exposes what accuracy conceals.