In a logistic regression model, what is the role of the log-likelihood (log-loss) function used during training?
Select an answer to reveal the explanation.
Short Explanation
Logistic regression doesn't just predict yes-or-no, it predicts a probability, so you need a loss that grades probabilities, not just labels. That's exactly what log-likelihood (log-loss) does: it measures how probable the actual outcomes were given the model's predicted probabilities, and it's brutal on confident wrong answers — predict 99% chance of 'yes' when the true answer is 'no' and you get slammed with a huge loss. That's answer C. Option A describes something like Euclidean distance, which is a regression-style metric for continuous values, not a probability-based classification loss. Option B is just plain accuracy, a simple count of right versus wrong, but it doesn't care how confident the model was, so it can't guide gradient-based training the way a smooth loss function can. Option D is describing feature selection or correlation analysis, an entirely different task from scoring a trained model's predictions.
Full Explanation
In logistic regression, the model outputs a predicted probability (via the sigmoid function) that a given input belongs to the positive class. The log-likelihood function, typically used in its negative form as the log-loss (or binary cross-entropy) during training, quantifies how well these predicted probabilities align with the actual observed class labels across the training set. Mathematically, for each example, it takes the log of the predicted probability assigned to the true class; summing (or averaging) these across all examples and negating the result yields a loss that is minimized when predicted probabilities are pushed close to 1 for true positives and close to 0 for true negatives. Because it operates on the log of probabilities, it disproportionately penalizes predictions that are both confident and wrong (e.g., predicting 0.99 probability for the wrong class produces a very large loss), which strongly encourages well-calibrated probability estimates rather than just correct hard classifications.
The first distractor describes something akin to a distance-based loss (such as squared error), which is appropriate for continuous-valued regression targets, not for scoring probabilistic classification outputs. Logistic regression's loss is derived from probability theory (maximum likelihood estimation), not geometric distance.
The second distractor describes classification accuracy, a simple ratio of correct predictions to total predictions. Accuracy is a useful evaluation metric, but it is not differentiable in a way that is useful for gradient-based optimization, and it does not use the actual predicted probability values, only whether the final hard classification was correct, so it cannot serve as the training loss itself.
The third distractor describes feature ranking or correlation analysis, which belongs to feature selection or exploratory data analysis, not to the loss function used to fit model parameters.
Memory aid: log-loss = 'maximum likelihood estimation for classification' — it is the negative log of the probability the model assigned to the correct answer, which is why confident-but-wrong predictions are punished so severely, pushing the model toward well-calibrated, accurate probability estimates.