A public housing authority wants a foundation model to broadly understand its internal housing-policy terminology, procedures, and case history before it is used for a casework assistant. The team runs continuous pre-training on its internal policy corpus rather than fine-tuning on a small labeled task dataset. What distinguishes this choice from fine-tuning?
Select an answer to reveal the explanation.
Short Explanation
Think of continuous pre-training as sending the model back to school to read the agency's whole policy library cover to cover, versus fine-tuning, which is more like coaching it on one specific job interview. Continuous pre-training deepens broad domain knowledge from unlabeled text, so the model just generally knows housing terminology better. That broad grounding is different from tuning it to nail one narrow task.
Full Explanation
Continuous pre-training extends a foundation model's original pre-training by exposing it to additional unlabeled domain text, in this case an internal housing-policy corpus, so the model absorbs broader vocabulary, concepts, and context specific to that domain without being optimized toward any single downstream task's exact input-output behavior. Fine-tuning, by contrast, typically uses labeled examples tailored to a specific task to sharpen the model's behavior on that task, which is a narrower and more targeted adjustment. The option claiming continuous pre-training requires labeled prompt-response pairs describes fine-tuning's typical data requirement, not continuous pre-training's use of unlabeled corpus text. The option limiting the effect to formatting and tone undersells what continuous pre-training changes; it can shift the model's underlying knowledge and terminology handling, not just surface style. The option claiming no foundation-model equivalent exists is simply incorrect, since continuous pre-training is a recognized customization technique for foundation models specifically, distinct from the fine-tuning and prompt-engineering techniques also available. Scope note: continuous pre-training on a large corpus is typically more compute-intensive than task-specific fine-tuning, and the two techniques are often combined in sequence. Operational check: after continuous pre-training, evaluate the model on general domain-comprehension prompts, not just the casework assistant's specific task format, to confirm the broader knowledge actually transferred.