In the context of neural network training, what precisely is one epoch?
Select an answer to reveal the explanation.
Short Explanation
An epoch is the big-picture unit of training: it's one full lap through the entire training dataset, every single example seen once. If you've got 10,000 training examples and a batch size of 100, that's 100 mini-batches, and 100 weight updates (iterations), all bundled together to make up just one epoch. That's answer B. The first option is actually describing an iteration or a single training step, not an epoch — one weight update per mini-batch happens many times within a single epoch, it's a smaller unit, not the same thing. The third option confuses epoch with evaluation — running the model against a test set is a completely separate action from a training pass, and it's not what defines an epoch at all. And the fourth option is describing something like a hyperparameter sweep or grid search cycle, which is a totally different level of the workflow, way outside what 'epoch' actually refers to.
Full Explanation
An epoch refers to one complete pass of the training algorithm through the entirety of the training dataset, meaning every training example has been used exactly once to compute gradients and update the model's weights (across however many mini-batches that requires). Training deep networks typically requires many epochs, often dozens to hundreds, because a single pass through the data is rarely enough for the weights to converge to a good solution; the same examples are revisited epoch after epoch, with the optimizer refining the weights a bit further each time.
The first distractor describes an iteration (sometimes called a training step), which is one single weight update performed after processing one mini-batch of data; if the training set contains N examples and mini-batches of size B are used, one epoch consists of N/B iterations, so an iteration is a much smaller unit nested inside an epoch, not equivalent to it. The third distractor confuses epoch with evaluation: running the model against a held-out test set (or validation set) to measure performance is an assessment step, typically performed periodically (often once per epoch) but conceptually and mechanically distinct from the epoch itself, which refers specifically to the training pass over the training data, not to any evaluation step that might follow it. The fourth distractor describes a hyperparameter search cycle, such as one full pass of a grid search or random search evaluating a set of candidate configurations; this operates at an entirely different level of the machine learning workflow, potentially running many complete multi-epoch training jobs, and has no direct equivalence to the single-pass-through-data meaning of an epoch.
A simple way to keep the hierarchy straight: an iteration (or step) is one weight update from one mini-batch, an epoch is a full pass through all mini-batches covering the entire training set exactly once, and a full training run typically spans many epochs. Choosing the right number of epochs is itself a balancing act tied closely to overfitting: too few epochs and the model may underfit, having not yet learned enough from the data, while too many epochs, especially without early stopping or adequate regularization, risks the model overfitting to the training set as it continues revisiting the same examples again and again.