A team training a GAN on a dataset of diverse handwritten digits notices that the generator has started producing only a handful of nearly identical-looking digit images, no matter what noise vector is fed in. What is this phenomenon called?
Select an answer to reveal the explanation.
Short Explanation
Here's what's happening: the generator found a small handful of outputs that reliably fool the discriminator, and instead of exploring the full variety of digits in the real data, it just keeps cranking out those same safe bets. That's mode collapse — the generator collapses onto one or a few 'modes' of the real data distribution and ignores the rest. Gradient clipping is actually a fix for a totally different problem (exploding gradients in RNNs), not something that causes repetitive output. Overfitting to a validation set is a supervised-learning concept about memorizing labeled examples, and GANs don't really have a validation-set overfitting story in the same sense. Catastrophic forgetting is about a network losing previously learned skills when trained on new tasks sequentially — a continual-learning problem, not a diversity problem in a GAN's output.
Full Explanation
Mode collapse is a well-known failure mode in GAN training where the generator learns to produce a limited variety of outputs — sometimes even a single output — regardless of variation in the input noise vector. This happens because the generator's objective is simply to fool the discriminator, and if it discovers a small set of outputs that reliably achieve this, gradient signals may not sufficiently push it to explore the rest of the real data distribution's diversity, especially since the minimax game between generator and discriminator does not explicitly reward output diversity. The result is repetitive, low-diversity samples that fail to capture the full richness of the training data (e.g., producing only a couple of digit styles instead of all ten digits with natural variation). Common mitigations include minibatch discrimination, unrolled GANs, Wasserstein GAN with gradient penalty, and diversity-promoting loss terms, all aimed at encouraging the generator to cover more of the true data distribution. Gradient clipping is an unrelated technique that caps the magnitude of gradients during backpropagation, most commonly applied to recurrent neural networks to prevent exploding gradients during training; it addresses a numerical-stability issue, not a lack of output diversity, and is not the term for the phenomenon described. Overfitting to a validation set describes a supervised-learning failure where a model performs well on data it has effectively memorized (directly or through excessive hyperparameter tuning against that split) but poorly on unseen data; GAN training dynamics and evaluation do not map cleanly onto this concept, and it does not describe repetitive generator output. Catastrophic forgetting refers to a neural network abruptly losing performance on previously learned tasks when it is subsequently trained on new tasks, a central challenge in continual and lifelong learning; it concerns forgetting old knowledge across sequential tasks, not a lack of diversity within a single generative model's outputs. Recognizing mode collapse and knowing it is specific to generative adversarial dynamics is essential for diagnosing and addressing GAN training instability.