A team wants to adapt a foundation model to municipal terminology by continuing its pre-training on a large volume of raw 311-service text, rather than teaching it to follow a specific instruction format. Which data-preparation choice correctly reflects continuous pre-training rather than supervised fine-tuning?

Mechanism: continuous pre-training extends a foundation model's underlying language understanding by exposing it to large volumes of raw, domain-relevant text using the same self-supervised objective as the original pre-training run, no instruction-response structure is imposed, and the goal is shifting the model's general knowledge and vocabulary toward the municipal domain rather than teaching a specific task format. Why the wrong options fail by concept: strictly formatting the corpus as labeled prompt-response pairs describes supervised fine-tuning data prep, that approach teaches a specific input-output behavior, which is a different objective from broad domain adaptation. Limiting the corpus to a small curated set undersizes what continuous pre-training needs, that scale of tight curation fits fine-tuning, while domain adaptation benefits from breadth and volume of raw text. Applying preference reward labels describes data preparation for alignment techniques that compare candidate responses, an entirely different training objective from extending a model's base language understanding. Scope caveat: continuous pre-training on a narrow domain corpus risks catastrophic forgetting of the base model's general capabilities if the domain text dominates training without any general-purpose text mixed in. Operational check: hold out a set of general-domain evaluation prompts unrelated to municipal topics and confirm the model's performance on them hasn't regressed after the continued pre-training run.