What is a key advantage of a Transformer architecture over a recurrent neural network (RNN) for natural language processing tasks?
Select an answer to reveal the explanation.
Short Explanation
Here's the core win of a Transformer: it drops the strict sequential bottleneck that RNNs are stuck with. An RNN has to process a sentence one token at a time, in order, because each step depends on the hidden state from the step before it, which is slow and hard to parallelize. A Transformer instead uses self-attention to look at every token in the sequence at once, letting the whole thing run in parallel on modern hardware, which is a huge speed and scalability win, especially on long sequences. That's the correct answer. Training-data efficiency isn't the real distinguishing advantage here, and it's not universally true anyway. Transformers absolutely can handle long sequences, they don't refuse to. And they still need positional encoding added in explicitly, because attention on its own has no built-in sense of token order.
Full Explanation
One of the defining advantages of the Transformer architecture over RNNs (and their LSTM/GRU variants) for NLP tasks is parallelism. An RNN processes a sequence strictly one token at a time, since computing the hidden state at time step t requires the hidden state from time step t-1; this inherent sequential dependency means RNN computation cannot be fully parallelized across the time dimension, which slows down both training and inference, particularly on long sequences. A Transformer instead relies on the self-attention mechanism, which computes relationships between all pairs of tokens in a sequence simultaneously, allowing the entire sequence to be processed in parallel rather than step by step. This parallelism translates into substantially faster training on modern hardware (GPUs/TPUs) and better scalability to longer sequences and larger datasets, which is a major reason Transformers have become the dominant architecture for large language models.
The first distractor, requiring far fewer training examples, is not an inherent architectural advantage of Transformers; in fact, Transformers are often noted for needing large amounts of training data to reach their full potential, and data efficiency depends heavily on factors like pretraining strategy, model size, and task, not simply the choice between Transformer and RNN.
The second distractor, that Transformers cannot handle sequences beyond a fixed maximum length, is incorrect in the sense implied; while a given Transformer configuration does have a fixed context window (maximum sequence length) determined by its positional encoding scheme and computational budget, this is a practical/engineering limit, and various techniques (e.g., longer context windows, relative position encodings, sparse attention) allow Transformers to handle very long sequences, so the claim that they categorically cannot is false, and it does not describe an 'advantage' of Transformers at all.
The third distractor is incorrect because Transformers do not eliminate the need for positional information, they require it. Because self-attention itself has no built-in notion of token order (it treats the input as a set), Transformers must add explicit positional encodings to the input embeddings so the model can distinguish where in the sequence each token occurs.
Memory aid: RNN equals sequential (one token waits for the last), Transformer equals parallel (attention looks at all tokens at once), and that parallelism is the headline advantage driving Transformer adoption for large-scale NLP.