What does the term "forward propagation" refer to during the execution or training of a neural network?
Select an answer to reveal the explanation.
Short Explanation
You can think of forward propagation like a relay race. The input data is the baton, passed from the input layer through hidden layers to the output layer to make a prediction. It doesn't update weights or calculate errors; it just generates the prediction.
Full Explanation
Forward propagation, or the forward pass, is the sequential process of passing input data through the network's layers to compute activations and generate an output. During forward propagation, the input layer receives the raw feature vector. For each hidden layer, the inputs are multiplied by a weight matrix, a bias vector is added, and the resulting linear combination is passed through a non-linear activation function, such as ReLU. This activation output is then passed as input to the next layer. This sequence continues until the final output layer is reached, producing the network's prediction. During training, the activations computed during the forward pass are stored in memory because they are required during backpropagation to calculate the gradients of the loss function. The description of the initial setup where random weights and biases are assigned to neurons is weight initialization, which occurs once before training starts. The description of modifying weight values based on the calculated gradient and learning rate is the parameter update step, such as gradient descent, which occurs after backpropagation. The description of computing the loss value by comparing the network's predictions to the target label is loss computation, which happens immediately after forward propagation completes. For the exam, make sure you associate forward propagation with the sequential, layer-by-step calculation of outputs from inputs.