When diagnosing why a learner underperforms, practitioners separate systematic wrongness from sensitivity to training quirks. How do bias and variance differ in this sense?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Okay, let's dive in. Bias vs variance is the classic “why is my model bad?” split. High bias: you're consistently wrong the same way—like a broken bathroom scale that always reads five pounds light. That's underfitting; the model is too rigid. High variance: tiny changes in the training set flip your predictions around—like a gossip network that retells the story differently every time. That's overfitting; you memorized noise. Exam trap: tying bias to “the loss function” or variance to “learning rate,” or saying bias means complex and variance means simple. Nope—those are related knobs, not the definitions. Remember: bias = systematic miss; variance = sensitivity to the sample. Got it? Sweet. You'll see this again next to the tradeoff question.
Full explanation below image
Full Explanation
In statistical learning theory, expected prediction error can be decomposed (under standard squared-error settings) into bias, variance, and irreducible noise. Bias quantifies error from erroneous assumptions—how far the average model (across training sets) sits from the true target function. A high-bias learner systematically misses patterns (underfitting): for example, fitting a straight line to strongly nonlinear data. Variance quantifies how much the fitted model changes when trained on different samples drawn from the same distribution. A high-variance learner tracks idiosyncrasies of each sample (overfitting) and may perform well on training data yet poorly on new data.
Understanding the difference guides remediation. High bias suggests richer features, more flexible models, or fewer overly strong constraints. High variance suggests more data, stronger regularization, simpler hypotheses, bagging, or other variance-reduction techniques. Learning curves and train/validation gaps help diagnose which regime dominates.
Distractors mislabel unrelated concepts. The loss function and learning rate influence optimization but are not definitions of bias and variance. Calling bias “complexity” and variance “simplicity” muddles cause and effect: complexity often trades one for the other, yet the terms refer to error behaviors. Training versus inference latency are engineering metrics outside the bias–variance decomposition. Underlying principle: bias and variance describe different error behaviors that suggest different remedies. Best practice diagnoses which regime dominates using learning curves and the train–validation gap before changing architecture or collecting more data. Memory aid: bias means wrong on average in a stable way; variance means unstable when the dataset jitters. Exam stems that contrast consistent systematic error with sensitivity to training fluctuations are asking for this distinction.