A city is deploying a lobby kiosk chatbot to answer simple resident questions and wants a cheaper, faster model than the large foundation model currently used for more complex casework elsewhere. The team trains a smaller student model to reproduce the larger model's behavior on this narrow set of simple questions. What technique does this describe?

Distillation is the technique of training a smaller student model to reproduce the outputs or behavior of a larger teacher model, typically by training the student on the teacher's predictions or generated responses for a defined task scope, which yields a model with lower inference cost and latency that still performs well within that narrower scope. This fits the kiosk scenario precisely: the city needs a lightweight model for simple resident questions and is using the large FM's behavior as the training signal for a smaller model rather than deploying the large model everywhere. Continuous pre-training describes a different process, exposing a model to broad unlabeled text to expand general knowledge, not compressing an existing model's task-specific behavior into a smaller one. Hyperparameter optimization refers to tuning settings like learning rate or batch size to improve a training run, which is unrelated to creating a smaller model from a larger one's outputs. Model ensembling combines multiple models' outputs at inference time to improve accuracy or robustness, which is the opposite goal of distillation's single lightweight deployed model, and no ensembling is described in the scenario. Scope note: a distilled model's quality is bounded by the scope of examples used from the teacher, so it should be validated only against the task range it was distilled for. Operational check: compare the student model's response accuracy against the teacher's on a held-out sample of simple resident questions before replacing the larger model in the kiosk.