Permit reviewers have gold “complete application” letters ready. What alignment order should the city follow?
Select an answer to reveal the explanation.
Short Explanation
Teach the permitting bot to write a solid “complete application” letter with supervised fine-tuning first. Preference tuning works better once that civic gold tone is already in the model’s bones.
Full Explanation
Permit reviewers’ gold “complete application” letters define the civic tone, structure, and required fields the assistant must first learn to imitate. Running supervised fine-tuning on those gold letters before any preference optimization stage establishes format competence so later DPO/GRPO stages refine relative quality instead of teaching letter-writing from a cold base. That alignment order—SFT warm start, then preferences—is standard for municipal correspondence bots. Starting with GRPO on empty rankings and skipping supervised examples entirely asks preference methods to invent style without demonstrations. Applying DPO first on unlabeled permit PDFs with no chosen or rejected letters lacks the pairwise signal DPO requires. Deploying the base model to residents and collecting preferences with no SFT warm start burdens the public with incoherent drafts and gathers noisy comparisons. On the exam, SFT on gold civic letters precedes preference optimization; distractors skip SFT, run DPO without pairs, or preference-collect from an unadapted base. Caveat: SFT on a narrow letter template can overfit boilerplate—keep a validation set of varied permit types and only then layer preference data for tone and helpfulness. Operational check: SFT on the gold complete-application corpus, verify template and field coverage on a held-out permit set, then introduce preference pairs for ranking among already-competent drafts.