A historic brickworks archive has thousands of unlabeled kiln diaries and a small labeled set of caption this photo pairs. Which application choice matches each data type?
Select an answer to reveal the explanation.
Short Explanation
Think of thousands of unlabeled kiln diaries versus a small stack of caption-this-photo pairs. Continued pre-training on the diaries, fine-tuning on the pairs. Do not swap those, and do not start from random weights.
Full Explanation
Unlabeled kiln diaries and labeled caption-this-photo pairs call for different adaptation stages. Continued pre-training (domain-adaptive pre-training) on the large unlabeled diary corpus teaches the model the brickworks language, spellings, and kiln vocabulary without needing captions. Fine-tuning on the smaller labeled caption pairs then teaches the supervised mapping from photo context to caption text. That order matches how foundation-model customization usually progresses from broad domain adaptation to task-specific supervision.
Fine-tuning on unlabeled diaries fails because fine-tuning expects labeled task supervision; unlabeled text is the wrong fuel for that step. Continued pre-training on the tiny labeled pair set wastes the wrong objective on scarce labels and skips using the abundant unlabeled diaries. Treating both jobs as a dashboard refresh ignores that labels and unsupervised corpora drive different training objectives. Starting from random weights discards the value of a pretrained foundation model and is especially wrong for a comparatively small archive.
Exam caveat: continued pre-training still needs compute and careful evaluation for forgetting; it is not free, and it is not a substitute for labeled fine-tuning when the end task is caption generation.
Operational check: run continued pre-training on a diary subset, evaluate domain perplexity or cloze probes on held-out kiln language, then fine-tune on caption pairs and score caption quality on a labeled holdout before full archive training.