A team builds a classifier for a rare disease where only 1% of patients in the dataset actually have the condition. The model predicts 'no disease' for every single patient and reports a very high score on one metric. Which metric is dangerously misleading in this scenario if used as the only measure of performance?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's a classic trap: a model that just says 'no disease' for every patient, where 99% of patients genuinely don't have the disease, scores about 99% accuracy without learning a single useful thing. Accuracy is 'correct predictions over total predictions,' and on a severely imbalanced dataset, always predicting the majority class racks up a huge score for free, while missing every actual disease case. That's answer B — accuracy alone is the misleading one here. Recall would expose this instantly, since it measures how many true positives you actually caught, and this model catches zero. Precision for the positive class would also expose it, since the model never predicts positive. And the false negative count would show every case missed — the opposite of hiding the problem.
Full explanation below image
Full Explanation
Accuracy is calculated as the number of correct predictions (true positives plus true negatives) divided by the total number of predictions. On a severely imbalanced dataset, such as one where 99% of examples belong to the negative class, a trivial model that predicts the majority class for every single input can achieve accuracy near 99% without learning any genuine discriminative pattern, simply because the negative class dominates the denominator. This makes accuracy alone dangerously misleading as a sole performance metric on imbalanced problems, since a model can appear excellent by this measure while being completely useless at its actual job of identifying the minority (and often more clinically or business-critical) class.
Recall is incorrect as the answer because it measures the proportion of actual positive cases that were correctly identified (true positives divided by true positives plus false negatives); far from hiding the problem, recall would immediately reveal it, dropping to zero or near zero for a model that never predicts the positive class, making recall one of the metrics you'd actually want to check specifically because accuracy can't be trusted here. Precision for the positive class is incorrect for a similar reason: it measures how many of the model's positive predictions were actually correct, and since a majority-class-only model makes no positive predictions at all, precision would be undefined or trivially reveal the model is not identifying any positive cases, again exposing rather than concealing the issue. The false negative count from the confusion matrix is incorrect because it is a direct, transparent count of exactly how many true disease cases were missed; in this scenario, it would show the maximum possible number of false negatives, making the failure obvious rather than hidden.
The underlying principle: on imbalanced classification problems, always pair accuracy with class-aware metrics like precision, recall, F1 score, or a full confusion matrix breakdown, because accuracy treats every correct prediction as equally valuable and cannot distinguish between a model that meaningfully separates classes and one that simply exploits class imbalance to inflate its score.