A transit authority diagrams why its schedule chatbot can relate a stop name mentioned early in the chat to a later transfer question. Which transformer mechanism best explains that long-range link?
Select an answer to reveal the explanation.
Short Explanation
Self-attention is like giving every word in the rider chat a highlighter that can mark any earlier stop name, not just the last few words. That is how a transfer question late in the thread still hooks the stop mentioned up front. Tiny windows and bag-of-words averages cannot stretch that far with the same flexibility.
Full Explanation
A transit authority chatbot that can connect a stop name mentioned early in a rider chat to a later transfer question depends on modeling long-range dependencies across the municipal conversation. Self-attention is the transformer mechanism that lets each token compute relationships to other tokens anywhere in the allowed context, so a late transfer query can still attend to an early stop mention without being limited to a fixed local window. That pairwise, content-based linking is exactly what explains the long-range civic dialogue behavior in the stem. A fixed bag-of-words average collapses order and pairwise structure, so early stop names and late questions become an unordered multiset with no principled link. A three-token convolution never revisits distant tokens outside its receptive field unless the stack is made impractically deep or dilated for that purpose, so it is a poor primary explanation for chat-scale stop-to-transfer links. Strict left-to-right n-gram tables lack learned attention weights over arbitrary distances and cannot dynamically reweight an early stop when a later transfer intent appears. Exam caveat: positional encodings, memory, or retrieval can assist long context, but the core transformer explanation for relating distant tokens in-context remains self-attention. Operational check: probe attention maps or ablation on early stop tokens when answering transfer questions, measure accuracy as chat length grows, and confirm context windows actually include the early stop mention in production transit logs.