The notices “crew relieved captain” and “captain relieved crew” would look the same if the model only saw an unordered bag of token vectors. What restores order?
Select an answer to reveal the explanation.
Short Explanation
A bag of token vectors cannot tell “crew relieved captain” from “captain relieved crew.” Self-attention is order-blind unless you inject a position signal. Learning rate, tree importance, and a job queue do not put words back in line.
Full Explanation
Self-attention does not inherently know sequence order, so transformers inject a position signal. Learning rate, tree importance, and job queues are not that signal.