What is a key feature of a classic RNN-based encoder-decoder architecture used for machine translation?
Select an answer to reveal the explanation.
Short Explanation
The classic encoder-decoder setup for translation works in two stages. First, the encoder reads through the entire source sentence, one token at a time, and by the end produces a single summary representation, often called a context vector, that's meant to capture the gist of the whole sentence. Then the decoder takes that context vector and uses it to generate the translated sentence, one word at a time, in the target language. That handoff of a context vector from encoder to decoder is the key feature here, so it's the right answer. The encoder and decoder are separate networks with separate learned weights, not shared ones. The decoder never sees the raw input text directly — it only has access to what the encoder passed along. And it's the decoder, not the encoder, that eventually produces a probability distribution over the target vocabulary, and only when generating output tokens, not while just reading the input.
Full Explanation
A classic RNN-based (or LSTM/GRU-based) encoder-decoder architecture for machine translation operates in two distinct stages. The encoder reads the entire source-language input sequence, one token at a time, updating its hidden state at each step, and after processing the final token, its resulting hidden state (or a summary derived from it) serves as a context vector meant to represent the overall meaning of the input sentence. This context vector is then passed to the decoder, which is a separate recurrent network that generates the target-language output sequence one token at a time, conditioning each generated token on the context vector and on the tokens it has already produced. Many practical systems augment this with an attention mechanism, allowing the decoder to look back at all of the encoder's intermediate hidden states rather than relying solely on a single fixed context vector, which helps address the information bottleneck of compressing long sentences into one vector, but the fundamental encoder-then-decoder-conditioned-on-context structure remains the defining feature of this architecture.
The first distractor, that the encoder and decoder share identical weights so only one network is trained, is incorrect; the encoder and decoder are structurally and functionally distinct networks (one reads input, the other generates output) with their own separate sets of learned parameters, even though both may be trained jointly end-to-end.
The second distractor, that the decoder receives the raw input text directly and bypasses the encoder, is incorrect because the whole purpose of the encoder-decoder design is that the decoder never sees the raw source text; it only has access to the encoded representation (the context vector, and optionally the encoder's intermediate states via attention), which is precisely what allows the architecture to separate the 'understanding the input' step from the 'generating the output' step.
The third distractor, that the encoder outputs a probability distribution over the target vocabulary at every input token, describes what the decoder does during generation, not the encoder; the encoder's job is to build an internal representation of the input sequence, not to produce vocabulary-level output probabilities, which only happens on the decoder side when generating the translated tokens.
Memory aid: think of the encoder as 'reading and summarizing' the source sentence into a context vector, and the decoder as 'writing' the translation based on that summary, two distinct jobs performed by two distinct, separately-trained networks.