What key property most clearly separates Transformer architectures from classic RNNs when processing sequences?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Okay, let's dive in. Classic RNNs read token one, then two, then three—like a single-file line at the coffee shop. Transformers let tokens talk through attention all at once, so GPUs can crunch the sequence in parallel during training. That's a huge deal for speed on long corpora. Boss wants faster language-model training? Parallel sequence processing is a big reason Transformers won. Exam trap: saying Transformers "can't do long range" is backwards. Land it: RNN = sequential steps; Transformer = parallelizable attention over positions. Got it? Keep that contrast locked in.
Full explanation below image
Full Explanation
A hallmark distinction of Transformer models relative to classic recurrent neural networks is parallel processing of sequence elements during the forward and backward passes of training. Self-attention computes pairwise interactions using matrix multiplications over the full sequence, with masking as needed, so hardware accelerators can utilize parallelism across positions. Recurrent networks, by contrast, update a hidden state token by token; later steps depend on earlier ones, which limits parallelization along the time axis even though examples within a batch can still be processed together. That sequential dependence is a major reason large-scale language pretraining shifted toward attention-based stacks.
Transformers are not weak at long-range modeling. Attention provides direct paths between distant tokens, often improving over vanilla recurrent networks that suffer vanishing gradients across many steps. Transformers also do not compress history into one fixed recurrent state as their sole memory mechanism; they maintain and attend over representations of many positions. While feedforward subnetworks sit inside each Transformer block, the architecture is defined by multi-head attention plus those position-wise multilayer perceptrons, residual connections, and normalization layers. It is therefore incorrect to describe Transformers as mere shallow single-layer feedforward classifiers without attention.
Positional encodings or relative position biases supply order information because attention alone does not assume sequence order. Autoregressive decoding at inference may still generate tokens step by step for language modeling, but that generation pattern does not erase the training-time parallelism advantage over recurrent unrolling. Memory and compute trade-offs remain important because attention cost grows with sequence length, yet the conceptual exam answer stays stable. A durable memory aid is that a recurrent network walks the timeline while a Transformer attends across the timeline in parallel. Questions that ask for the key differentiator almost always want parallel sequence processing via attention rather than misconceptions about long-range failure, fixed hidden states, or shallow feedforward-only designs.