You are constructing a deep neural network to classify network traffic into one of five mutually exclusive protocols. Which activation function should you apply to the output layer of this network to obtain a valid probability distribution?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal: when you're classifying traffic into five mutually exclusive protocols, your output layer produces logits, not probabilities. Softmax squashes those raw scores into values between 0 and 1 that add up to exactly 1.0, so you get a valid probability distribution. Sigmoid is for binary yes/no decisions, not multi-class classification.
Full Explanation
In neural network design, the activation function chosen for the output layer is determined by the type of machine learning task being performed:
1. Multi-class Classification (Softmax): When a model must classify an input into one of three or more mutually exclusive classes—such as five protocols—the output layer uses the Softmax activation function. Softmax normalizes the output values (logits) of the final layer into a probability distribution over the classes. Mathematically, it exponentiates each output and divides it by the sum of all exponentiated outputs. This ensures that: - Each output value is a probability between $0$ and $1$. - The sum of all output probabilities equals exactly $1$. 2. Binary Classification (Sigmoid): When there are only two possible classes, a Sigmoid function is used, outputting a single probability value between $0$ and $1$. 3. Hidden Layers (ReLU, Tanh): Functions like ReLU (Rectified Linear Unit) and Tanh (Hyperbolic Tangent) are used in hidden layers to introduce non-linearity, but they are not suitable for output layers of multi-class classifiers because they do not output normalized probabilities that sum to 1.
The Rectified Linear Unit (ReLU) choice is incorrect because it outputs values from 0 to infinity, which cannot represent a probability distribution. The Hyperbolic Tangent (Tanh) choice is incorrect because it outputs values between -1 and 1, which cannot represent probabilities. The Sigmoid choice is incorrect because it is designed for binary classification, mapping a single output to a range between 0 and 1, and does not coordinate multiple output nodes to sum to 1.