A team wants to build a model that predicts the most likely next word given the words that came before it in a sentence. Which type of architecture is best suited to this task?
Select an answer to reveal the explanation.
Short Explanation
Predicting the next word is all about context: what came before in the sentence directly shapes what's likely to come next. That's a sequential modeling problem through and through, and it's exactly what RNNs (and their LSTM/GRU cousins) and Transformers are built for, since both are designed to capture dependencies across a sequence of tokens, whether through a recurrent hidden state or self-attention. That's why 'RNN or Transformer' is the correct answer. A CNN built for fixed-size image input is designed for spatial, grid-like data, not sequential text prediction. A discriminator's whole job is a binary real-vs-fake classification, nothing to do with generating the next word in a sequence. And a plain autoencoder just reconstructs its own input, it isn't set up to predict something new like an upcoming word that wasn't part of its input at all.
Full Explanation
Predicting the next word in a sequence, given the preceding words, is a classic example of language modeling, a task fundamentally about capturing sequential dependencies: the probability of the next word depends heavily on the words that came before it. Recurrent neural networks (including LSTM and GRU variants) are naturally suited to this task because they process a sequence step by step, maintaining a hidden state that carries forward information about earlier words as context for predicting what comes next. Transformer-based language models, which have become the dominant approach for this task, instead use self-attention to let each position in the sequence attend to all preceding tokens (in autoregressive/causal language models) to build a rich contextual representation used to predict the next token. Both architecture families share the key property that they are designed to model dependencies across positions in a sequence, which is precisely what next-word prediction requires; the choice between them in practice often comes down to considerations like training efficiency, parallelism, and the scale of data, with Transformers currently favored for large-scale language modeling.
The first distractor, a CNN applied only to a fixed-size image, describes an architecture designed for spatial, grid-structured data like pixels in an image, and while CNNs can be adapted for certain sequence tasks using 1D convolutions, an architecture 'applied only to a fixed-size image input' is fundamentally mismatched to the variable-length, sequential nature of language modeling.
The second distractor, a discriminator trained solely to classify text as human-written versus machine-generated, describes a binary classification task; a discriminator in this framing only outputs a real/fake judgment about a complete piece of text and has no mechanism for generating or predicting an upcoming word, which is a fundamentally different, generative task.
The third distractor, a standalone autoencoder trained only to compress and reconstruct a single sentence, describes a self-reconstruction task where the output is meant to match the same input it was given; this does not naturally support predicting a genuinely new next word that extends beyond the given input, which is the core requirement of next-word prediction.
Memory aid: whenever the task is 'what comes next, given everything so far,' whether it's the next word, the next frame in a video, or the next value in a time series, think sequence-modeling architectures: RNN/LSTM/GRU historically, and Transformers as the modern standard for large-scale language modeling.