A data scientist is building a model that outputs the probability a customer will click an advertisement. Which activation is appropriate for the final layer?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Pay close attention—this one bites people on exams. If you need a probability someone clicks, your last layer has to speak probability language. Sigmoid squeezes one logit into 0–1 for yes/no click. Softmax if you've got a multi-class “which outcome?” setup that must sum to one. Tanh goes negative—weird for probs. Linear can spit out 7.3. ReLU can go to infinity. Think of it like a volume knob that must stay between silent and full for odds, not a raw amp. Boss wants CTR predictions for bidding? Probability head, then log loss. Takeaway: sigmoid/softmax for click probability outputs—not ReLU/linear/tanh as the final answer.
Full explanation below image
Full Explanation
Click-through modeling is typically framed as probabilistic classification: estimate P(click | features). Neural networks produce raw scores (logits) that must be mapped into a valid probability interpretation for training with log loss / cross-entropy and for decision thresholds in ads ranking. Without a probability-compatible final activation (or an equivalent loss that includes that mapping), the network’s last number is just an unbounded score, not a click probability the business can threshold or calibrate.
For binary click versus no-click, a single unit with a sigmoid activation converts a real-valued logit z into σ(z) ∈ (0,1), which is the standard Bernoulli probability parameterization. When the problem is cast as multi-class (for example, mutually exclusive outcomes), softmax produces a distribution over classes that sums to one. Either choice aligns the output geometry with classification likelihoods; the stem’s “probability of a click” most often implies sigmoid, with softmax as the multi-class counterpart—hence “sigmoid or softmax” as the correct family. Both pair naturally with cross-entropy objectives used throughout industry CTR systems.
Tanh saturates in (−1,1). While it is useful in hidden layers of some architectures, negative outputs are not probabilities without an ad hoc remap, so it is a poor default output for CTR. A linear output is unbounded and appropriate for regression or as pre-sigmoid logits—but the question asks for the activation used to represent probability at the output, not the unbounded logit itself. ReLU outputs [0, ∞) and zeros out negatives; it neither upper-bounds at 1 nor forms a normalized distribution, so it is unsuitable as a probability head even though it dominates hidden layers in modern deep nets.
Best practices: train with binary cross-entropy (or focal loss variants) for imbalanced clicks, monitor calibration (reliability diagrams, Platt/isotonic adjustments if needed), separate ranking metrics from probability quality, and keep ReLU/tanh in hidden layers where their optimization properties help representation learning. Memory aid: “probabilities need sigmoid/softmax; ReLU is for hidden feature maps.” On the exam, match the output range to the prediction type before you pick the activation name.