After training a logistic regression classifier, you observe much higher metrics on the training set than on a held-out test set. What is the most likely explanation?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Check this out: train score through the roof, test score in the basement—that's overfitting waving at you. Logistic regression can overfit too, especially with tons of features, polynomial expansions, or weak regularization. Underfitting would look bad on training as well. Low learning rate? You'd struggle to fit training, not ace it. Tiny datasets make overfit easier, but the diagnosis name for that gap is still overfit. Exam trap: picking "underfitted" because performance is "bad" somewhere. Bad on test only, great on train → overfit. Fix with regularization, fewer features, more data, or simpler model. Got it?
Full explanation below image
Full Explanation
Overfitting arises when a model captures idiosyncrasies of the training sample that do not reflect the broader data-generating process. Empirically, the clearest red flag is a substantial gap: strong training metrics paired with markedly weaker validation or test metrics. Logistic regression is linear in its parameters and often thought of as a relatively high-bias model, yet it can still overfit when the feature space is high-dimensional relative to sample size, when features are noisy or collinear, when strong interactions or many one-hot categories are encoded explicitly, or when regularization such as L1, L2, or elastic net is too weak. In those regimes the decision boundary can hug training quirks and fail on fresh cases.
Underfitting is the opposite pattern: the model is too constrained to fit even the training data, so both training and test performance are poor and often similar. A learning rate that is too low typically produces slow convergence or incomplete optimization, which tends to hurt training fit rather than create a train-test chasm with excellent training scores. A small dataset increases variance and overfitting risk, but calling the dataset too small is a possible root contributor, not a competing definition of the observed symptom; the symptom itself is overfitting and poor generalization. Leakage can also inflate training metrics, so pipelines must keep test rows completely out of fitting and feature selection.
Practical responses include stronger regularization, feature selection or dimensionality reduction, collecting more representative labeled data, checking for leakage that inflates training scores, simplifying the hypothesis class, and using proper cross-validation with a final held-out test. Always evaluate on data the training procedure never touched, and compare learning curves for train versus validation as capacity or regularization changes.
For exam items that describe much better training than test performance, prefer overfitting as the most likely cause. Associate underfitting with uniformly weak fit on both sets, optimization-rate issues with training dynamics that fail to fit the training data well, and data size as contextual risk—not as a replacement for the generalization diagnosis named overfitting.