County IT fine-tunes a permits FAQ model, then prunes and quantizes it so building-inspector tablets can run it offline. What is the primary goal of pruning and quantization here?

After fine-tuning a permits FAQ model, pruning and quantization shrink and simplify weights so building-inspector tablets can run the assistant offline. The primary goal is edge-friendly deployment: fewer parameters and lower-precision weights reduce memory, latency, and power so municipal field devices can host the FAQ without constant cloud streaming. Speeding up offline tablet inference using speculative decoding draft models fails because pruning and quantization compress weights for edge deployment, not implement a draft-and-verify decoding strategy; speculative decoding needs a separate draft model to propose tokens for the target model to verify and does not shrink or simplify the fine-tuned model for tablets. Extending offline tablet dialogue memory using rotary position interpolation fails because pruning and quantization compress model weights for edge deployment, not adjust position embeddings to expand the input window; longer dialogue memory would increase memory and compute demands on tablets, undermining the goal of running the FAQ offline on constrained field devices. Retaining full-precision permit weights using activation checkpointing on tablets fails by concept because pruning and quantization intentionally reduce precision and parameter count for edge deployment; activation checkpointing is a training-time memory technique that recomputes activations during backpropagation, not a method to keep full-precision weights for on-device inference, and it does not shrink or simplify the model for tablets. Exam caveat: aggressive compression can degrade rare permit edge-case answers; offline models still need secure update channels and offline evaluation before inspectors rely on them in the field. Operational check: benchmark compressed versus full model on the permits FAQ gold set, measure tablet RAM/CPU budgets, sign model packages for update integrity, and provide a fallback path when the on-device answer confidence is low.