A mustard-mill note says “add seed then grind,” which means the opposite if the words swap. Why does a transformer need positional information?
Select an answer to reveal the explanation.
Short Explanation
Without a sense of place, “seed then grind” and “grind then seed” look like the same bag of words. Positional information tells attention who came first. Order matters for meaning.
Full Explanation
Self-attention alone is permutation-sensitive only if order is encoded somehow; positional information supplies that order signal. Transformers therefore combine token embeddings with positional encodings or similar mechanisms. Without them, word-order distinctions such as recipe steps can be lost.