A model's final layer needs to output something that can be interpreted as the probability of belonging to the positive class in a two-class problem. Which activation function is the classic choice for that output neuron, and why?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Sigmoid is the go-to here because it takes whatever number comes out of that last neuron — could be -50, could be 800 — and squashes it into a tidy value between 0 and 1. That's exactly the shape you want when the output is supposed to represent 'probability the answer is yes.' Feed it into a threshold (usually 0.5) and you've got your binary decision. ReLU is wrong because it just clips negatives to zero and lets positives blow up unbounded — nothing like a probability. Softmax on a single neuron doesn't even make sense in the way described, and it definitely doesn't eliminate vanishing gradients, which sigmoid is actually somewhat prone to. Linear activation leaves raw, unbounded scores, which is useful for regression, not for something you want to read as a probability.
Full explanation below image
Full Explanation
The correct choice is sigmoid, because its defining mathematical property is mapping any real number to the open interval (0, 1) via the formula 1/(1+e^-x). That range maps naturally onto a probability interpretation, which is exactly what's needed on the output neuron of a binary classifier: a value close to 1 signals strong confidence in the positive class, a value close to 0 signals strong confidence in the negative class, and values near 0.5 signal uncertainty. Paired with binary cross-entropy loss, sigmoid output gives a principled probabilistic framework for two-class problems. The first distractor, ReLU, is incorrect because ReLU outputs zero for all negative inputs and the raw input value for positive inputs, producing an unbounded range from 0 to infinity — it's excellent for hidden layers because it avoids saturation and is cheap to compute, but it cannot represent probability because its outputs aren't confined to a 0–1 range, and 'letting negative values pass through' is also factually backwards since ReLU actually zeroes out negatives. The second distractor invokes softmax on a single neuron; softmax is designed for multi-class problems where it normalizes a vector of scores across multiple neurons so they sum to 1, and applying it to just one neuron would trivially always output 1, which is meaningless — it also is not a technique for solving vanishing gradients, a problem that's actually associated with sigmoid and tanh due to their flat regions at extreme inputs. The fourth distractor, linear activation, correctly notes that it preserves the raw, unbounded score, but that's precisely why it's unsuitable here: without squashing, the output value could be any real number, making a probability interpretation invalid unless further transformation is manually applied — linear output is appropriate for regression tasks predicting continuous quantities, not for a bounded probability estimate. A useful memory aid: sigmoid's 'S-curve' shape is the same reason it's used for binary logistic regression, and deep learning simply borrowed that idea for the final layer of binary classifiers.