A product team wants to build a system that generates long, coherent, human-sounding paragraphs of text from a prompt. Which type of model is best suited to this task?
Select an answer to reveal the explanation.
Short Explanation
For generating long, fluent, coherent text, you want a Transformer-based language model — think GPT-style architectures. Why? Self-attention lets the model weigh every earlier word against every other word when predicting the next one, so it keeps track of long-range context — who the subject was three sentences ago, what tone it's in, all of it — far better than older approaches. A CNN over embeddings can pick up local word patterns but struggles with long-range coherence. A single perceptron is way too shallow to model language structure. And k-nearest-neighbors just retrieves similar existing text — it doesn't generate anything new. Transformers are the standard answer here for a reason.
Full Explanation
The correct answer is a Transformer-based language model. Transformers use self-attention mechanisms that allow every token in a sequence to attend to every other token, regardless of distance, which gives them a strong ability to model long-range dependencies and maintain coherence across long passages of generated text. This is precisely why architectures like GPT (built on stacked Transformer decoder blocks) are the standard choice for open-ended, high-quality text generation, and why they have largely superseded older sequence models for this task. (A GAN can also generate data, but GANs are notoriously difficult to train for discrete text output and are far less common for this purpose than Transformer language models.)
A convolutional neural network over word embeddings can capture local n-gram-like patterns efficiently and is used in some NLP tasks (e.g., text classification), but its fixed, local receptive field makes it a poor fit for tracking dependencies across long spans of generated text, so paragraph-level coherence suffers. A single-layer perceptron with a softmax output is far too shallow and lacks any mechanism for sequential or contextual modeling at all — it can only make a one-shot mapping from a fixed input to an output distribution, nothing resembling autoregressive generation over a sequence. A k-nearest-neighbors classifier over a text corpus isn't a generative model in the deep-learning sense; it retrieves the most similar existing examples rather than synthesizing new, original sequences, so it cannot produce novel coherent text beyond recombining stored snippets.
Memory aid: 'Attention is all you need' (the title of the original Transformer paper) — self-attention is the mechanism that lets these models track relationships across an entire passage, which is exactly what long-form text generation demands.