A courts-clerk agent that has reliably answered filing-status questions for months starts giving subtly outdated procedural guidance, even though no code or prompt changes were deployed to it. What is the most likely explanation, and how should the team confirm it?
Select an answer to reveal the explanation.
Short Explanation
Think of a courts-clerk agent like a paralegal who hasn't cracked open a new manual in months: the desk hasn't moved, but the rules on it quietly have. That's drift — the world around the agent changed even though nobody touched its code. You catch it by re-running the same golden questions you used at launch and comparing today's answers to yesterday's baseline.
Full Explanation
Model behavior and the source procedures it reflects both shift gradually — an underlying model update, a changed filing procedure, or evolving entity behavior — while the agent's code and prompts stay untouched. Because nothing in the deployment itself changed, the failure never shows up in a code diff; it only surfaces when current output is compared against a fixed reference over time, which is exactly what a periodic evaluation run against a golden dataset detects. A latency spike changes how fast an answer arrives, not whether its content is stale, so it doesn't explain subtly wrong guidance. A reasoning loop produces no final answer at all rather than a confidently wrong one, and a regional outage is an availability failure, not a content-quality failure, so neither matches this symptom. Scope caveat: rule out a recent deploy or upstream data-source change first, since drift specifically describes decay without a corresponding change event. Operational check: schedule a recurring evaluation run against the launch-time golden dataset and alert when a domain's pass rate falls below its established baseline, rather than waiting for a resident complaint to surface the problem.