A data scientist is building a network to predict house prices, a continuous dollar value with no fixed range. Which activation function should sit on the output layer's single neuron?
Select an answer to reveal the explanation.
Short Explanation
For predicting something like house price, you need the network to output any number, big or small, positive or huge. That's exactly what a linear activation gives you: the raw weighted sum passes straight through, unconstrained. Sigmoid and tanh both squash the output into a tight little range, 0 to 1 or -1 to 1, which is great for probabilities but terrible for a value like $450,000. Softmax is even further off since that's built for choosing between multiple categories, not producing a single continuous number. For regression, keep it simple: linear out, let the number be whatever it needs to be.
Full Explanation
Regression tasks require an output layer that can represent any real-valued number, since the target (price, temperature, distance, and so on) is continuous and unbounded. A linear activation function, essentially f(x) = x, passes the neuron's weighted sum straight through without any squashing, so the network's prediction can land anywhere on the real number line, matching the nature of the target variable.
Softmax is wrong because it converts a vector of outputs into a probability distribution that sums to 1 across multiple classes; it is designed for multi-class classification, not for producing a single unbounded continuous value. Applying softmax to a regression problem would force the output into an artificial probability-like range that has no meaningful relationship to a price in dollars.
Sigmoid is wrong because it compresses any input into the range (0, 1), which is appropriate for binary classification or for outputs that are naturally bounded probabilities, but it cannot represent a house price of $450,000 without severe distortion; the gradient would also saturate for large input magnitudes, making learning difficult.
Tanh is wrong for the same underlying reason as sigmoid: it squashes output into (-1, 1). While tanh is zero-centered and sometimes preferred in hidden layers for that property, it is still a bounded function unsuitable for an open-ended regression target.
A reliable memory aid: match the output activation to the shape of your target. Continuous, unbounded targets pair with linear; a single yes/no probability pairs with sigmoid; mutually exclusive multi-class probabilities pair with softmax. Choosing the wrong output activation is a common beginner mistake that silently caps model performance because the network becomes mathematically incapable of producing the correct range of answers, no matter how well the hidden layers are trained.