Neighborhood kiosks cannot host a 70B city-code teacher. How should the city create a deployable model?
Select an answer to reveal the explanation.
Short Explanation
Kiosks can’t swallow a 70B city-code brain. Distill that teacher into a lean ~7B student that still knows the neighborhood rules.
Full Explanation
Neighborhood kiosks cannot host a 70B city-code teacher; knowledge distillation compresses teacher behavior into a smaller student suitable for edge inference. Distilling the 70B teacher into a smaller ~7B student for kiosk inference is the standard municipal pattern: train with teacher signals centrally on capable GPUs, then deploy the compact student where Wi-Fi, power, and GPU budgets are limited. Shipping the full 70B teacher to every kiosk over consumer Wi-Fi fails size, bandwidth, and on-device memory constraints and is operationally fragile for neighborhood sites. Deleting the teacher and training a random 7B model with no teacher signals abandons distillation—the student never receives soft targets or rationales that carry city-code knowledge. Quantizing only the kiosk display firmware and leaving the LLM unchanged does not reduce model footprint at all and leaves the 70B problem untouched. Exam caveat: distillation is not guaranteed parity on rare ordinances—evaluate long-tail section IDs and bilingual code Q&A before declaring the student kiosk-ready, and budget for periodic redistillation when code amendments land. Operational check: compare teacher versus student exact-match on a frozen city-code quiz, measure student VRAM/latency on the kiosk GPU, and promote only if both quality and fit gates pass under concurrent kiosk sessions.