How does logistic regression fundamentally differ from linear regression in what it outputs?
Select an answer to reveal the explanation.
Short Explanation
Let's keep this simple: linear regression is for predicting a number that can be pretty much anything — a house price, a temperature, a salary. Logistic regression is for predicting a probability that something belongs to a category, like 'will this customer churn: yes or no.' It squashes its output through a sigmoid function so the result always lands between 0 and 1, which you then interpret as a probability. The correct answer nails that distinction: continuous value versus categorical probability. The first option has it backwards — swap those two and you'd be right. The 'one feature only' option is just false; both models happily handle many input features. And skipping gradient-based optimization isn't true either — logistic regression absolutely uses gradient descent (or similar) to fit its coefficients, it's just optimizing a different loss function (log loss instead of squared error).
Full Explanation
Linear regression models the relationship between input features and a continuous dependent variable, producing an output that can, in principle, range across all real numbers (e.g., predicting price, temperature, or age). It is fit by minimizing a loss function such as mean squared error. Logistic regression, despite sharing 'regression' in its name, is used for classification: it models the probability that an observation belongs to a particular category (commonly binary, though it extends to multiclass via softmax). It achieves this by passing a linear combination of inputs through the sigmoid (logistic) function, which compresses any real-valued input into the range (0, 1), making the output interpretable as a probability. The model is typically fit by minimizing log loss (cross-entropy) rather than squared error.
The first distractor reverses the true relationship: it claims logistic regression is the unbounded one and linear regression is bounded, which is exactly backwards from how the two algorithms actually behave.
The 'single feature only' distractor is incorrect because both linear and logistic regression naturally extend to multiple input features (multivariate/multiple regression); there is no inherent restriction limiting logistic regression to one predictor.
The 'no gradient-based optimization' distractor is incorrect because, unlike linear regression (which has a closed-form normal-equation solution), logistic regression generally has no closed-form solution and is fit using iterative optimization methods such as gradient descent, Newton's method, or IRLS (iteratively reweighted least squares) to minimize the log-loss objective.
A helpful memory aid: linear regression draws a best-fit line through numbers; logistic regression draws a best-fit boundary between classes and reports how confident it is with a probability between 0 and 1. The name is a historical artifact — logistic regression is a classification algorithm, not a numeric-value regression algorithm.