After a model update, the parks department wants assurance the chatbot still follows policy. Which practice best fits response monitoring?

Response monitoring after a parks chatbot model update means systematically reviewing sampled outputs for policy violations, unsafe advice, and drift from approved corpus-backed answers. Model upgrades can change tone, refusal behavior, and factual grounding even when prompts look unchanged; inspecting what the bot actually returns is the only way to validate that parks policy still holds for residents. Sampling across common trail, hours, and closure questions—plus known edge cases—detects regressions before they become widespread public misinformation. This practice treats outputs as production behavior subject to quality and compliance review, not as opaque vendor magic.

Reviewing sampled chatbot prompts for policy violations after the model update fails because prompts are inputs, not outputs; they cannot reveal whether the responses actually violated parks policy or drifted from approved answers.

Reviewing user-reported chatbot responses for policy violations after the model update fails because waiting for residents to notice and report is reactive and incomplete, not systematic response monitoring; unreported violations and quiet drift can persist even when few reports arrive.

Reviewing response log retention settings for policy violations after the model update fails because reviewing how long logs are kept is not reviewing the responses themselves; it cannot show whether sampled outputs actually violated parks policy or drifted from approved answers.

Exam caveat: Response monitoring complements—not replaces—prompt sampling review, user-report triage, or log-retention-settings upkeep. A model with clean prompts, tidy retention settings, and few user reports can still drift on the ordinary questions nobody is systematically sampling after the update. Also distinguish automated toxicity/policy classifiers from human review of civic domain accuracy (trail closures, fees, ordinances), which often needs staff judgment.

Operational check: After each parks model update, run a fixed evaluation set and sample live responses for policy violations; retain samples per records policy; do not substitute prompt sampling for response sampling because inputs do not verify post-update policy compliance; alert on rising refusal failures or hallucination flags; and ensure reviewers can see outputs, not only inbound prompts.