Neural Networks & Deep Learning

Certified AI Developer (CAID) · 3 questions

• During the training of a deep neural network, after the input has passed forward through the layers and the prediction error has been calculated, how does the network determine exactly how to adjust the millions of weights and biases inside its hidden layers to reduce that error?
• You're monitoring the training of a new deep neural network on a GPU cluster, and you notice the loss curve is completely flat—basically a straight line from epoch one. You check your configurations and see the learning rate is set to an extremely tiny number, like 0.000000001. Why is this tiny learning rate preventing the model from learning?
• When reviewing training logs for a neural network, you see the term 'Epoch' listed next to the loss values. What does a single epoch represent in the training process?