School grading-rubric assistants stay on FP16 after INT8 calibration sets prove too small for reliable PTQ. Why is that decision sound?
Select an answer to reveal the explanation.
Short Explanation
PTQ without enough calibration examples is guessing with a half-empty ruler. Keep the rubric assistant in FP16 until the civic calibration set is trustworthy.
Full Explanation
Post-training INT8 quantization depends on a representative calibration set; when school grading-rubric assistants find civic calibration sets too small for reliable PTQ, remaining at FP16 is an appropriate accuracy and reliability decision. Insufficient calibration makes scale factors unstable, so refusing INT8 until better civic calibration exists protects student-facing rubric quality. Claiming FP16 is required by law for every school district model invents a legal constraint the stem does not establish and misses the calibration rationale. Claiming INT8 never works on any GPU so calibration size is irrelevant is false—INT8 often works when calibration is adequate. Asserting FP16 always scores higher BLEU than any calibrated INT8 by definition overstates a universal metric ranking and ignores well-calibrated PTQ successes; the sound reason here is insufficient calibration data, not metaphysical FP16 supremacy. Exam caveat: “remain at FP16” is temporary engineering judgment—once a larger, stratified rubric calibration set exists (subjects, grade bands, bilingual comments), revisit PTQ with measured deltas. Operational check: document calibration set size/coverage, show PTQ variance across repeated calibrations, and keep FP16 in production until variance and rubric exact-match gates stabilize under INT8.