During backpropagation, why does the algorithm need to compute the derivative of each neuron's activation function at its current input value?
Select an answer to reveal the explanation.
Short Explanation
Think of the activation function as a dimmer switch between a neuron's input and its output. Backprop needs to know: if I nudge the input a tiny bit, how much does the output actually move? That sensitivity is exactly what the derivative gives you, and it's the piece that lets the chain rule pass gradient information backward from the loss, through each layer, to the weights that need updating — that's answer B. Option A is describing what an activation function like sigmoid does on the forward pass, not what its derivative is for. Option C is talking about pruning, a completely separate model-compression technique that has nothing to do with backprop's core math. And option D, weight initialization, happens before training even starts and doesn't involve activation derivatives at all — it's a different step in the pipeline entirely.
Full Explanation
Backpropagation computes how the loss changes with respect to every weight in the network by repeatedly applying the chain rule, working backward from the output layer to the input layer. At each neuron, the chain rule requires multiplying the gradient flowing in from the layers ahead by the local derivative of that neuron's own activation function, evaluated at the input value it received during the forward pass. This local derivative tells you exactly how much the neuron's output would change for a small change in its input — essentially the neuron's local 'gain' or sensitivity. Without this term, there would be no way to correctly propagate the gradient signal from the loss back through a nonlinear activation to the pre-activation values and, ultimately, to the weights that produced them.
The first distractor confuses the activation function itself with its derivative. Functions like sigmoid do squash outputs into a bounded range (e.g., 0 to 1), but that squashing behavior is a property of the forward-pass function, not the reason its derivative is computed during backprop.
The second distractor, about deciding how many neurons to prune, describes network pruning, a separate model-compression or regularization technique usually applied after or during training based on weight magnitude or other importance criteria. It is unrelated to the chain-rule mechanics of standard backpropagation.
The third distractor, weight initialization, refers to how starting weight values are chosen (e.g., Xavier or He initialization) before any training or backpropagation occurs. It does not involve computing activation derivatives during the backward pass.
Memory aid: 'forward pass uses the function; backward pass uses its slope.' The activation derivative is the local multiplier that lets error signals flow correctly through each nonlinearity via the chain rule — which is also why activations like sigmoid or tanh, whose derivatives shrink toward zero at extreme inputs, are linked to the vanishing gradient problem in deep networks.