Courts want to detect whether self-help answers change unfairly with resident identity cues. Which bias detection method fits?

Courts want to detect whether self-help answers change unfairly with resident identity cues; the fitting bias detection method is to run counterfactual tests that swap resident names and genders in court self-help prompts. Counterfactual testing holds the legal-self-help ask constant while varying identity markers; systematic answer drift (different rights explanations, harsher tone, fewer resources, more refusals) indicates the model is conditioning on identity rather than on the legal question. That is a standard, targeted probe for disparate treatment in generative systems. Asking only one volunteer once and declaring the model unbiased is anecdotal and underpowered. Comparing answers only when prompts are identical and never varying names refuses to run the counterfactual and cannot detect identity-conditioned drift. Disabling logging of disparate outcomes to avoid finding bias is concealment, not detection. Exam caveat: choose name/gender swaps carefully to avoid introducing unintended confounds (for example culturally specific legal contexts), use multiple pairs, and combine automated difference metrics with human legal-aid review for court domains. Operational check: build paired court self-help prompts differing only in name/gender cues, score answer equivalence on substance and tone, flag systematic gaps, and remediate before asserting the self-help LLM treats residents equitably.