For translating long documents, why is a Transformer generally preferred over an LSTM-based encoder-decoder?
Select an answer to reveal the explanation.
Short Explanation
Long documents are exactly where an LSTM's biggest weakness shows up: it has to process tokens one after another, in strict sequence, so a long input means a long chain of sequential steps, and information from way back in the sequence can degrade by the time it's needed. A Transformer sidesteps this by using self-attention to look at every token in the sequence at once, in parallel, and directly relate distant tokens to each other without a long chain of sequential steps in between. That parallel, direct-connection advantage on long sequences is why it's preferred here, making it the correct answer. Memory usage actually tends to grow with sequence length for standard self-attention, so 'strictly less memory regardless of length' isn't accurate. Attention is actually central to how Transformers work, not something they avoid needing. And Transformer decoders generate variable-length output sequences just fine, they're not locked into one fixed output length.
Full Explanation
For translating long documents, Transformers are generally preferred over LSTM-based encoder-decoders primarily because of how they handle long-range dependencies and computational parallelism. An LSTM processes a sequence strictly step by step, since the hidden state at each time step depends on the hidden state from the previous step; this means information about tokens early in a long sequence must be carried forward through many sequential updates to remain relevant late in the sequence, and despite LSTM's gating mechanisms designed to preserve long-term dependencies, very long sequences can still degrade this information and are also expensive to process because computation cannot be parallelized across time steps. A Transformer, by contrast, uses self-attention to directly compute relationships between every pair of tokens in the sequence in a single parallel operation, regardless of how far apart those tokens are positioned. This means a token near the end of a long document can directly attend to a token near the beginning without that information needing to pass through many sequential intermediate steps, and because the sequence is processed in parallel rather than one token at a time, training and inference can be substantially faster on modern parallel hardware.
The first distractor, requiring strictly less memory regardless of sequence length, is incorrect because standard self-attention's memory and computation requirements scale quadratically with sequence length (since it computes relationships between every pair of tokens), which can actually make vanilla Transformers more memory-intensive than RNNs on very long sequences; various efficient-attention variants exist specifically to address this, but it is not accurate to say Transformers unconditionally use less memory than LSTMs.
The second distractor, that Transformers never need any attention mechanism for long inputs, is incorrect and contradictory, since self-attention is the core mechanism that gives Transformers their key advantages; removing attention would remove the very feature that makes Transformers effective at capturing long-range dependencies.
The third distractor, that the decoder can only generate one fixed-length output, is incorrect because Transformer decoders, like RNN decoders, generate output sequences autoregressively (one token at a time) and can produce variable-length outputs depending on the input and an end-of-sequence signal, not a single fixed length.
Memory aid: LSTM equals a long sequential relay race passing information step by step (which can lose fidelity over long distances); Transformer equals every token getting a direct line to every other token at once via attention, which is why it scales better to long documents.