When reviewing training logs for a neural network, you see the term 'Epoch' listed next to the loss values. What does a single epoch represent in the training process?
Select an answer to reveal the explanation.
Short Explanation
Okay, let's get our terminology straight. When you train a model, one epoch means you've fed your entire training dataset through the network once, forward and backward. You usually do that in batches, so don't confuse an epoch with a batch size or a single step.
Full Explanation
In the context of machine learning and neural network training, an epoch is defined as one full pass of the entire training dataset through the model, including both forward and backward propagation. Because datasets are often too large to fit into memory at once, they are divided into smaller subsets called batches. The definition describing the number of training samples grouped together in a single batch for gradient calculation is batch size, not an epoch. An iteration or step is the completion of one forward and backward pass on a single batch; if the batch contains only one sample, it resembles the definition describing processing a single training sample through one forward and backward pass, but it still is not a full dataset pass. If a dataset has 10,000 samples and the batch size is set to 100, it will take 100 iterations to complete one epoch. Typically, neural networks require training over multiple epochs, often dozens, hundreds, or thousands, to adequately learn the underlying patterns in the data. Monitoring metrics like training and validation loss epoch-by-epoch is standard practice to determine when a model has converged and to prevent overfitting, which occurs when training for too many epochs. The definition describing the depth of the network model, representing the total count of weight-bearing layers, is unrelated to training time cycles.