When reviewing training logs for a neural network, you see the term 'Epoch' listed next to the loss values. What does a single epoch represent in the training process?
Select an answer to reveal the explanation.
Short Explanation
Okay, let's get our terminology straight. When you are training a model, you don't just show it one picture and call it a day. You have to feed it your entire dataset—let's say 10,000 images. One epoch means the network has looked at and processed every single one of those 10,000 images once. Now, you usually don't feed all 10,000 at once; you break them into smaller bites called 'batches'. If your batch size is 100, it takes 100 steps (iterations) to complete one epoch. Don't confuse an epoch with a batch size or a single step, because that's an easy point to lose on the exam!
Full Explanation
In the context of machine learning and neural network training, an epoch is defined as one full pass of the entire training dataset through the model (both forward and backward propagation). Because datasets are often too large to fit into memory at once, they are divided into smaller subsets called batches. Batch Size (Option A) is the number of training examples processed in a single batch. An Iteration or Step (similar to Option C) is the completion of one forward and backward pass on a single batch. If a dataset has 10,000 samples and the batch size is set to 100, it will take 100 iterations to complete one epoch. Typically, neural networks require training over multiple epochs (often dozens, hundreds, or thousands) to adequately learn the underlying patterns in the data. Monitoring metrics like training and validation loss epoch-by-epoch is standard practice to determine when a model has converged and to prevent overfitting (which occurs when training for too many epochs). Option D describes network depth, which is unrelated to training time cycles.