In the context of neural network training, what precisely is one epoch?
Select an answer to reveal the explanation.
Short Explanation
An epoch is the big-picture unit of training: it's one full lap through the entire training dataset, every single example seen once. If you've got 10,000 training examples and a batch size of 100, that's 100 mini-batches, and 100 weight updates (iterations), all bundled together to make up just one epoch. That's the correct answer. The single-mini-batch-update option is actually describing an iteration or a single training step, not an epoch — one weight update per mini-batch happens many times within a single epoch, it's a smaller unit, not the same thing. The test-set option confuses epoch with evaluation — running the model against a test set is a completely separate action from a training pass, and it's not what defines an epoch at all. And the hyperparameter-search option is describing something like a hyperparameter sweep or grid search cycle, which is a totally different level of the workflow, way outside what 'epoch' actually refers to.
Full Explanation
An epoch refers to one complete pass of the training algorithm through the entirety of the training dataset, meaning every training example has been used exactly once to compute gradients and update the model's weights (across however many mini-batches that requires). Training deep networks typically requires many epochs, often dozens to hundreds, because a single pass through the data is rarely enough for the weights to converge to a good solution; the same examples are revisited epoch after epoch, with the optimizer refining the weights a bit further each time.
An epoch is not one single weight update from a mini-batch; that describes an iteration or training step, a much smaller unit nested inside an epoch. Confusing an epoch with running the model against a held-out test set is incorrect; test-set evaluation is an assessment step distinct from an epoch, which refers to training over the data. Describing an epoch as one full cycle of hyperparameter search is incorrect; that describes a hyperparameter search at a different level of the workflow.
A simple way to keep the hierarchy straight: an iteration (or step) is one weight update from one mini-batch, an epoch is a full pass through all mini-batches covering the entire training set exactly once, and a full training run typically spans many epochs. Choosing the right number of epochs is itself a balancing act tied closely to overfitting: too few epochs and the model may underfit, having not yet learned enough from the data, while too many epochs, especially without early stopping or adequate regularization, risks the model overfitting to the training set as it continues revisiting the same examples again and again.