A canal-archive team used to feed tokens one after another through a recurrent net and lost track of early tokens. Why did transformers become the building block of modern LLMs?
Select an answer to reveal the explanation.
Short Explanation
A recurrent net lost track of early tokens. A transformer can weigh relationships across the sequence and train more readily in parallel. It is not a boosted tree, and not a job scheduler.
Full Explanation
Transformers model pairwise token relationships and avoid the strict step-by-step bottleneck of classic RNNs, which also makes training easier to parallelize. They are not limited to the previous token, not boosted trees, and not a job scheduler.