A Keras training run sometimes overfits in later epochs. Which built-in callback should be used to automatically save only the model weights from the epoch with the best validation loss?
Select an answer to reveal the explanation.
Short Explanation
You want the model that performed best on validation data, not whatever weights happen to exist when training finally stops — and that's exactly what ModelCheckpoint is built for. Set it up with monitor='val_loss' and save_best_only=True, and it'll watch validation loss every epoch and only overwrite the saved file when it actually improves. That's your answer. EarlyStopping is a close cousin — it stops training when things stall — but it doesn't save weights by itself. ReduceLROnPlateau adjusts the learning rate when progress stalls, which is about training dynamics, not saving files. And TensorBoard is a visualization dashboard for watching metrics over time — great for monitoring, but it doesn't save your model either. When the question is specifically about saving the best weights, ModelCheckpoint is the one.
Full Explanation
ModelCheckpoint is the Keras callback designed specifically to save a model (or just its weights) during training, typically configured with monitor='val_loss' and save_best_only=True so that it writes a new checkpoint file only when the monitored metric improves compared to all previous epochs. This directly addresses the scenario of overfitting in later epochs: even if validation loss starts climbing after some point, ModelCheckpoint ensures the saved artifact corresponds to the epoch with the best (lowest) validation loss rather than the final epoch's potentially overfit weights.
EarlyStopping is a related but distinct callback that monitors a metric (often val_loss) and halts training once it fails to improve for a specified number of epochs (the 'patience' parameter). While EarlyStopping can optionally restore the best weights seen during training via its restore_best_weights=True argument, it does not persist a checkpoint file to disk on its own; its core function is to stop the training loop early, not to save the model. In many production pipelines, EarlyStopping and ModelCheckpoint are used together but serve different purposes.
ReduceLROnPlateau is a callback that reduces the learning rate when a monitored metric stops improving, helping the optimizer take smaller, more careful steps as training plateaus. It affects the optimizer's hyperparameters mid-training but does not save the model or its weights at all.
TensorBoard is a callback (and companion visualization tool) that logs metrics, graphs, and other training diagnostics for visualization in the TensorBoard dashboard. It is used for monitoring and debugging training progress visually, but it does not save model weights to be reloaded later for inference or deployment.
Memory aid: EarlyStopping decides when to stop; ModelCheckpoint decides what to keep; ReduceLROnPlateau adjusts how the optimizer behaves; TensorBoard shows you what happened. When the specific requirement is 'save the best weights based on validation performance,' ModelCheckpoint is the tool built exactly for that job.