A canal-history project cannot afford librarians to tag every word in a century of minutes, yet it can hide some words and ask the model to recover them from the rest. What is that approach?
Select an answer to reveal the explanation.
Short Explanation
Nobody can tag a century of minutes word by word. Hide some words and make the model recover them from the rest, and the text itself becomes the teacher. That is self-supervised pretraining, not a librarian on every token, not dock clustering, and not a CUDA homework.
Full Explanation
Self-supervision builds the objective from the text (for example, recover a hidden word). That is what unlocks web-scale pretraining without a librarian on every token. Clustering and CUDA kernels are not that signal.