Housing counselors ranked groups of resource-referral answers from best to worst. Which method can use those group preferences?

Housing counselors ranked groups of resource-referral answers from best to worst, producing relative ordering within each group rather than isolated binary labels. Group relative policy optimization (GRPO) style training samples a group of responses per prompt, scores each with a reward (for example, derived from the counselors' rankings), and uses each response's reward relative to the group to compute its advantage; that consumes the ranked referral groups to optimize relative preferences—learning to promote better civic referrals over weaker ones inside the same comparison set. That method matches the preference structure counselors already provided. Discarding rankings and training only on the single worst answer in each group throws away most of the ordinal signal and oddly focuses on negatives alone. Converting rankings into random binary labels unrelated to counselor judgment destroys the human preference information. Fine-tuning exclusively with masked language modeling and ignoring preference ranks reverts to unsupervised token prediction and never uses the referral quality judgments. On the exam, GRPO-style methods fit ranked municipal answer groups; distractors keep only the worst item, randomize labels, or ignore ranks for MLM. Caveat: small counselor panels can embed local resource bias—audit referral fairness across neighborhoods and languages even after GRPO improves average rank metrics. Operational check: format counselor rankings into GRPO group batches, train, and measure rank correlation / top-1 counselor agreement on held-out housing referral groups before publishing the assistant.