A county skate-park info bot will run all summer. The team can pay once to shrink a large teacher into a smaller student that keeps most quality at lower serving cost. Which technique is that?
Select an answer to reveal the explanation.
Short Explanation
Think of shrinking a large teacher into a smaller student you can serve all summer. That one-time pay is distillation. Raising temperature or stuffing PDFs does not make the model smaller.
Full Explanation
Distillation pays once to shrink a model for cheaper or faster inference while keeping most quality. Temperature and stuffing PDFs do not distill a teacher into a student. Daily from-scratch pre-training is the opposite of a cheaper serving path.