What does the term "forward propagation" refer to during the execution or training of a neural network?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of forward propagation like a relay race. The input data is the baton. It starts at the input layer, gets passed to the first hidden layer where some math is done, then to the next layer, and finally reaches the output layer to make a prediction. It's a one-way street from left to right. Every node does its linear math and applies an activation function, passing the result forward. That's it! It doesn't update weights or calculate errors; it just generates the prediction. So, Option D is the correct answer.
Full explanation below image
Full Explanation
Forward propagation (or the forward pass) is the process by which a neural network processes input data to generate a prediction or output. During forward propagation: 1. The input layer receives the raw feature vector. 2. For each hidden layer, the inputs are multiplied by a weight matrix, a bias vector is added, and the resulting linear combination is passed through a non-linear activation function (such as ReLU). 3. This activation output is then passed as input to the next layer. 4. This sequence continues until the final output layer is reached, producing the network's prediction. During training, the activations computed during the forward pass are stored in memory because they are required during backpropagation to calculate the gradients of the loss function. Let's look at the incorrect options: Option A describes weight initialization, which occurs once before training starts. Option B describes the parameter update step (gradient descent), which occurs after backpropagation. Option C describes loss computation, which happens immediately after forward propagation completes. For the exam, make sure you associate forward propagation with the sequential, layer-by-step calculation of outputs from inputs.