Why did the Transformer architecture become so influential for natural language processing tasks?
Select an answer to reveal the explanation.
Short Explanation
Here's the big idea behind the Transformer: attention. Instead of marching through a sentence one word at a time like older recurrent networks did, attention lets every token look directly at every other token in a single pass. That gives you two huge wins — the whole sequence can be crunched in parallel instead of step-by-step, and the model captures long-range relationships between words no matter how far apart they sit. That's why the attention option is correct here. The 'one token at a time with a bigger hidden state' option describes the old RNN/LSTM approach the Transformer was built to move past. Swapping matrix multiplications for lookup tables isn't a real Transformer feature either — attention still leans heavily on matrix math under the hood. And Transformers don't need labeled part-of-speech tags; they typically train on raw text using self-supervised objectives like predicting masked or next tokens.
Full Explanation
The Transformer architecture became foundational to modern NLP because of its self-attention mechanism, which allows every token in an input sequence to directly attend to every other token in a single layer, computing a weighted combination of all other tokens' representations based on learned relevance scores. This design delivers two major advantages over the recurrent architectures (RNNs, LSTMs) that preceded it: first, because attention computes relationships across the whole sequence simultaneously rather than token-by-token, Transformers can be parallelized efficiently on modern hardware, dramatically speeding up training compared to the inherently sequential nature of recurrence; second, attention connects distant tokens directly, with a path length of one between any two positions in the sequence, which lets the model capture long-range dependencies (such as subject-verb agreement across a long clause) far more effectively than RNNs, which had to propagate information step-by-step and often suffered from vanishing gradients over long sequences. The correct option is therefore the one describing attention enabling parallel computation and long-range dependency capture. The option describing the Transformer as processing tokens strictly one at a time with a larger hidden state is incorrect because that describes the sequential, recurrent processing style of RNNs and LSTMs, which the Transformer was specifically designed to move away from; the Transformer instead processes the whole sequence at once using positional encodings to retain order information, since attention alone has no inherent sense of sequence order. The option describing the Transformer as replacing matrix multiplications with lookup tables is incorrect because the core operations of self-attention (computing queries, keys, and values, and their dot-product similarities) are fundamentally matrix multiplication-heavy; there is no lookup-table substitution in the standard architecture. The option describing a requirement for labeled part-of-speech tags is incorrect because Transformers, especially in modern pretraining regimes like masked language modeling or next-token prediction, are trained in a self-supervised manner directly on raw text, with no need for explicit linguistic annotations such as part-of-speech tags. A useful memory aid: think of self-attention as everyone in a large meeting being able to speak to and hear from everyone else at once, rather than passing a note around the room one person at a time — that direct, all-at-once communication is what gives Transformers both their speed and their ability to connect distant ideas.