A team training a GAN to generate images of handwritten digits notices the generator keeps producing only a handful of nearly identical-looking digit images, regardless of the random noise input. What phenomenon is this?
Select an answer to reveal the explanation.
Short Explanation and Infographic
This is a classic, well-known GAN failure mode called mode collapse. It happens when the generator finds a small handful of outputs that reliably fool the discriminator, and rather than keep exploring the full diversity of the real data distribution, it just leans on those same few 'winning' outputs over and over, regardless of what random noise vector you feed it. That narrow, repetitive output is the signature of mode collapse, so that's the answer. Vanishing gradients would show up as training stalling out and weights barely updating, not repetitive-but-still-working outputs. There's no 'validation set' concept driving the discriminator's behavior here — GANs aren't typically evaluated that way, and this symptom isn't overfitting to held-out data. And exploding gradients would typically cause unstable, erratic, or diverging training, not a generator converging on a narrow set of repeated samples.
Full explanation below image
Full Explanation
Mode collapse is a well-documented failure mode specific to generative adversarial network training. It occurs when the generator discovers that producing a small number of particular outputs (or even a single output) is sufficient to reliably fool the current discriminator, and as a result the generator collapses to producing only that limited set of outputs regardless of variation in its input noise vector, rather than learning to model the full diversity of the real data distribution. This happens because the minimax adversarial game between generator and discriminator does not guarantee convergence to a solution that captures the entire data distribution; the generator is only incentivized to fool the discriminator, and if it finds an easy, narrow way to do so, gradient signals may not push it toward broader diversity. Symptoms include noticeably repetitive or near-identical generated samples across many different random input seeds, which matches the scenario described (a handful of near-identical digit images regardless of the input noise).
The first distractor, vanishing gradients in the discriminator's final layer, describes a different training pathology in which gradients become too small to meaningfully update weights, typically causing training to stall or the generator to stop improving altogether, rather than causing it to repeatedly output a narrow set of samples; vanishing gradients are more associated with poor discriminator feedback quality than with output diversity collapse.
The second distractor, overfitting of the discriminator to a validation set, is a mismatched concept because standard GAN training does not typically involve a validation set in the way supervised learning does; the discriminator is trained directly against real training samples and generator outputs, not evaluated against a held-out validation split, so this framing does not fit typical GAN training dynamics.
The third distractor, exploding gradients from too high a learning rate, describes a scenario where weight updates become excessively large, generally causing unstable or divergent training (e.g., NaN losses, wildly fluctuating outputs), which does not match the described symptom of a generator converging on a small, repetitive set of consistent outputs.
Memory aid: think of mode collapse as the generator finding one or two 'cheat codes' that reliably fool the discriminator and then refusing to try anything else, producing a narrow, repetitive set of outputs instead of the rich variety present in the real training data.