After a housing-authority prompt edit, staff worry answers drifted even though classic uptime metrics look fine. Which GenAI-specific observability practice best detects that consistency problem?

Running golden datasets through the housing assistant and using output diffing to spot response drift after prompt changes is the GenAI-specific observability practice that best detects consistency problems when classic uptime metrics still look fine. Prompt edits can change tone, policy citations, and eligibility language without crashing the service; only comparing new answers to known-good golden responses reveals silent drift that housing caseworkers feel as “the bot changed.” Diff-based evaluation turns consistency into an observable signal rather than hallway rumor. Relying solely on network ping success as proof that housing answers remain accurate fails by concept: connectivity proves reachability, not semantic correctness of housing guidance. Increasing provisioned throughput without comparing new outputs to known-good answers fails by concept—more capacity does not detect or prevent answer drift after a prompt edit. Turning off logging so housing caseworkers cannot see answer variance fails by concept and governance: hiding variance removes the evidence needed to roll back bad prompt changes. Exam caveat: GenAI quality regressions often leave infrastructure green; the exam privileges golden-set output comparison over uptime, capacity, or log suppression. Operational check: maintain a golden Q&A set covering eligibility, waitlist, and maintenance topics; run it on every prompt change; diff answers for citation and decision language; gate promotion on acceptable drift thresholds; and keep before/after samples for housing leadership review.