A housing AI program needs scalable assessment across automatic scores, judges, humans, and error reviews. Which design fits?
Select an answer to reveal the explanation.
Short Explanation
Housing AI needs a calendar, not a one-off vibe check: ROUGE on the clock, a judge LLM, HITL samples, and a weekly error huddle. That framework is how quality stays visible at scale.
Full Explanation
Housing AI programs need an evaluation framework that schedules automatic metrics, LLM-as-a-judge scoring, human-in-the-loop sampling, and recurring error reviews so quality stays visible as volume scales. ROUGE (or similar automatic scores) catches regression cheaply and continuously; a judge LLM adds semantic coverage beyond n-gram overlap; HITL sampling grounds scores in resident-facing truth; weekly error reviews feed taxonomy updates and prompt fixes. ROUGE alone without human or error-analysis loops overfits to surface similarity and misses harmful housing advice that still looks fluent. Judge LLM scores without HITL or taxonomy updates drift with the judge model and never convert failure modes into durable civic governance. Ad-hoc spot checks with no scheduled metrics suite leave gaps between audits and cannot support scalable assessment across automatic scores, judges, humans, and error reviews as the stem requires. Exam caveat: NCP-GENL treats evaluation as a designed system of complementary signals, not a single metric or occasional vibe check. Operational check: put ROUGE and judge jobs on a calendar, sample HITL cases with clear rubrics, run a weekly error huddle that updates an error taxonomy, and gate housing-model promotions on completion of that scheduled loop.