What capability is a transformer-based model best known for in sequence modeling?
Select an answer to reveal the explanation.
Short Explanation
Here's where transformers changed the game for you. Old-school RNNs read a sentence basically one token after another—slow to train and painful for long context. Transformers say: let's look at all positions with attention and score how much each token should care about every other token. The cool thing is training can blast through the sequence in parallel on GPUs. Pay close attention: "parallel via attention" is the headline on the exam—not "cheap and tiny," not "images only," not "pure CNN classifier." Boss wants a language model that scales? You're talking transformers. Remember the slogan: attention + parallelism beats strict sequential recurrence for modern large models.
Full Explanation
Transformer architectures center on attention mechanisms—especially multi-head self-attention—combined with position-wise feedforward layers, residual connections, and normalization. Self-attention lets each position build a representation by weighting information from other positions in the sequence according to learned compatibility scores. Unlike recurrent networks that update a hidden state step by step, transformers compute interactions across the sequence in a form that maps efficiently onto parallel hardware. That parallel, attention-driven processing of text (and other sequences) is the hallmark property highlighted in the correct answer.
Parallelism does not imply trivial compute cost. Full self-attention is quadratic in sequence length for naive implementations, and large transformer language models demand substantial memory and FLOPs. Therefore options claiming inherent simplicity and low cost misrepresent practice. Similarly, while related ideas appear in vision (ViT) and multimodal systems, defining transformers primarily as image generators or as pure convolutional classifiers misses their defining sequence modeling role and their attention backbone. Convolutional networks excel at local spatial filters; transformers excel at content-based global mixing via attention, though hybrids exist.
Historically, transformers unlocked large-scale pretraining for machine translation, language modeling, and transfer learning, forming the backbone of modern LLMs. Positional encodings or relative position biases restore order information that pure set-like attention would otherwise lack. Encoder-only, decoder-only, and encoder–decoder variants trade off bidirectional context versus autoregressive generation.
For exams, associate transformers with: attention for dependency modeling, parallel sequence processing versus RNN serialism, and scalability for NLP and generative AI—not with being the cheapest model, not exclusively with GANs/image synthesis, and not with convolution-only classification pipelines. That conceptual map distinguishes the architecture cleanly among neighboring deep learning families.