A mustard-mill note says “add seed then grind,” which means the opposite if the words swap. Why does a transformer need positional information?
Select an answer to reveal the explanation.
Short Explanation
Without a sense of place, “seed then grind” and “grind then seed” look like the same bag of words. Positional information tells attention who came first. Order matters for meaning.
Full Explanation
A mustard-mill instruction such as “add seed then grind” means the opposite if word order swaps, so a transformer needs positional information so attention is not blind to order when reading the token sequence. Positional encodings or equivalent order cues distinguish sequences that share the same bag of tokens. Positional information does not authorize deleting embeddings in favor of pixel averages. Learning-rate schedules do not replace token embeddings. Multimodal stills do not bypass tokenization without a text path when text meaning depends on order. Exam caveat: order sensitivity is the reason for positions, not a vision workaround. Operational check: compare model behavior on swapped civic instruction variants; if both collapse to the same prediction without positions, restore positional signals and retest. Add swapped-order regression tests for civic instructions that reverse critical verbs. Positional signals make “seed then grind” differ from the reverse; schedules and pixel averages are not substitutes for order. Add swapped-order regressions for critical civic instructions. Close q100 with a municipal evidence note that records why the correct mechanism fits this civic multimodal scenario, why each of the three distractors fails by a distinct concept, and which operational check was observed in logs or dataset cards before promotion. File that note with the run packet so later maintenance cannot silently revive a rejected framing.