An engineering group is designing a model whose job is to predict the next token in a running sentence for autocomplete. Which neural approach is the most appropriate primary choice?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Boss wants autocomplete by Friday—what do you reach for? Next-word prediction is language modeling: use what came before to guess what comes next. RNNs did this for years; transformers own it now with attention over context. A CNN is great for pixels, not your first call for sentences. A GAN fights itself to fake samples—not the usual next-token trainer. A plain fully connected net that forgets order? That's how you get word salad. Exam tip: if the stem says "next word in a sentence," think sequence LM—RNN or transformer. Land it: ordered text prediction → sequential/transformer models.
Full explanation below image
Full Explanation
Predicting the next word (or subword token) is the canonical language modeling objective: maximize the probability of each token given preceding context. Architectures that respect sequence order and long-range dependencies are therefore the natural fit. Recurrent neural networks maintain a hidden state across time steps and were historically standard for this task, including LSTM and GRU variants that improved gradient flow over plain RNNs. Transformers, via self-attention and positional information, model dependencies in parallel and scale more effectively; modern autocomplete and large language models are overwhelmingly transformer-based, though the conceptual family remains "sequence models for ordered text," which includes both RNNs and transformers as appropriate answers on many exams.
Convolutional neural networks aggregate local neighborhoods on grids and dominate vision. While 1D convolutions can process text windows, a vision-style CNN is not the most appropriate primary choice when the requirement is next-token language modeling with flexible context. Generative adversarial networks optimize a min-max game between generator and discriminator and are awkward for discrete token sequences; mainstream language models train with maximum likelihood / cross-entropy (and related objectives), not GAN training as the default. A generic fully connected network that treats inputs without sequence structure fails to encode order—"dog bites man" versus "man bites dog"—and cannot implement a proper causal context window without additional sequential design such as recurrence or attention with masking.
Engineering practice includes causal masking so the model cannot peek at future tokens, subword tokenization (BPE, WordPiece, SentencePiece), context length limits, temperature and top-k/top-p sampling at inference, and evaluation with perplexity or downstream task metrics. Memory aid: "next token = language model = RNN or transformer." Exam questions contrasting CNN, GAN, plain MLP, and sequence models for next-word prediction should resolve to the sequence language-model family.