Training blanks out a few tokens in a lock log (“The ____ gate failed”) and scores whether the model fills the blank using both sides. Which objective is that?
Select an answer to reveal the explanation.
Short Explanation
Blank out “The ____ gate failed” and let the model use both sides to fill it. That is masked language modeling, the BERT-style objective. Causal next-token training never looks right of the blank, and a harvest number or NCCL reduce is not a fill-in-the-blank.
Full Explanation
Masked language modeling fills blanks from both sides and is the classic BERT-style objective. Causal next-token modeling hides the future. A numeric harvest and NCCL are not that task.