Before routing field-level confidence for municipal forms, what should the team do?

Before routing field-level confidence on municipal forms, the team must calibrate those scores on a labeled municipal validation set. Raw model confidence is not a probability until it is checked against city documents—permits, affidavits, utility forms—with known field values.

Calibration on municipal labels works because form layouts, handwriting, stamps, and civic jargon differ from generic corpora. Field-level curves (name vs. parcel ID vs. fee amount) often need different thresholds before auto-accept versus clerk review.

Using raw model confidence without any validation set fails because uncalibrated scores mis-rank risk and mis-route review. Calibrating once on private SaaS receipts and ignoring city forms fails because receipt domains do not transfer to municipal paperwork. Skipping calibration and routing randomly to clerks fails because it wastes clerk time and provides no principled threshold policy.

Exam caveat: recalibrate when forms, models, or scanners change; a one-time private-sector curve is not permanent civic truth. Operational check: build a labeled city validation set, plot confidence versus empirical accuracy by field, set routing thresholds from that curve, then apply them in production.