A housing team sees eligibility edge cases shift after a Claude model release. What response fits?
Select an answer to reveal the explanation.
Short Explanation
Models move—when a release tweaks edge cases, retest those housing eligibility prompts before residents feel it. Old eval scores, a temperature tweak, or a generic benchmark won't tell you whether the new model still gets eligibility right.
Full Explanation
Retesting housing eligibility prompts after a Claude model release that changes edge-case behavior is the required response because eligibility logic is high-stakes for residents and model upgrades can silently shift borderline answers. Civic teams must treat releases as potential breaking changes: re-run the eligibility eval suite, compare edge-case diffs, and gate production promotion on acceptance criteria rather than assuming behavioral continuity. That discipline protects applicants from incorrect denials or approvals introduced by an upstream model change.
Reusing prior release eval scores as a proxy for current edge-case behavior during triage fails because model providers routinely alter edge-case behavior even within a family, and housing decisions are sensitive to those shifts, so stale scores cannot represent the new release. Tuning sampling temperature to recover prior eligibility decision patterns fails because it papers over behavior changes instead of measuring them, leaving the housing authority without a reliable signal about whether the new model handles protected edge cases correctly. Running a general capability benchmark in place of targeted eligibility checks fails because broad benchmark scores do not verify housing-specific eligibility logic, so untested edge-case shifts still convert model release notes into live resident impact.
Exam caveat: prompt retesting must cover protected edge cases (mixed household composition, income documentation gaps, preference codes) and not only happy-path fixtures that always passed. Operational check: pin the prior model in staging, deploy the candidate release beside it, diff eligibility outcomes on the frozen edge-case set, and require housing program owners to sign off before the new model receives production traffic.