In a variational autoencoder (VAE), what does the encoder learn to output for a given input, as opposed to a single fixed latent vector?
Select an answer to reveal the explanation.
Short Explanation
This is the one big structural difference between a VAE and a plain autoencoder. A standard autoencoder's encoder squashes the input down to one fixed point in latent space, end of story. A VAE's encoder instead outputs parameters of a probability distribution, typically a mean and a variance, describing a range of possible latent values for that input. You then sample a point from that distribution before handing it to the decoder. That distributional output is exactly what gives a VAE its smooth, structured latent space useful for generating new data, so it's the correct answer. The encoder isn't producing a classification label — that's a supervised classification task, unrelated to a VAE's unsupervised generative goal. It's also not literally producing the final reconstructed output; that's the decoder's job, which comes after sampling. And it's definitely not outputting raw gradient values — gradients are computed during backpropagation by the training process, not emitted as the encoder's forward-pass output.
Full Explanation
In a standard (vanilla) autoencoder, the encoder maps a given input deterministically to a single fixed point in the latent space, essentially just compressing the data. A variational autoencoder changes this behavior: instead of outputting one fixed latent vector, the VAE's encoder outputs the parameters, typically the mean and variance (or log-variance for numerical stability), of a probability distribution over the latent space for that particular input, commonly assumed to be a multivariate Gaussian. A specific latent vector is then obtained by sampling from this distribution, using the reparameterization trick so that gradients can still flow through the sampling step during training. This distributional formulation, combined with a regularization term (KL divergence) that encourages the learned distributions to stay close to a standard normal prior, produces a smooth, continuous, well-structured latent space. That structure is what allows the decoder to generate plausible new outputs by sampling from anywhere in the prior distribution after training, which is the generative capability that distinguishes a VAE from a standard autoencoder.
The first distractor, producing a discrete class label, describes a supervised classification output, which is unrelated to what a VAE's encoder does; VAEs are typically trained in an unsupervised or self-supervised manner to reconstruct and generate data, not to predict categorical labels.
The second distractor, producing a reconstructed version identical to the decoder's final output, conflates the encoder's role with the decoder's role; the encoder's output is a distribution over the compressed latent representation, and it is the decoder, operating on a sample drawn from that distribution, that produces the reconstructed data, not the encoder itself.
The third distractor, producing gradient values used to update the decoder's weights, misunderstands how training works; gradients are computed via backpropagation of the loss function with respect to the network's parameters, a separate computational process from the encoder's forward-pass output, which describes a probability distribution over the latent code, not a set of gradients.
Memory aid: standard autoencoder's encoder outputs a point; VAE's encoder outputs a cloud (a distribution) that you sample a point from, and that added randomness plus the KL regularization is precisely what turns an autoencoder into a generative model.