Meridian Line's maintenance-triage agent has been running in production for months. The platform team wants to detect, without waiting for a broker or engineer to complain, that the mix of equipment types being asked about has started shifting toward a crane model the retrieval corpus has very little documentation for. Where should this be detected?

Detecting a shift in what users are actually asking about, as opposed to what the system was built and evaluated to handle, requires monitoring the live query distribution over time, which is exactly what analyzing logged production requests, via inference tables or similar endpoint monitoring, provides, surfacing a drift toward equipment types the corpus has thin coverage for before it shows up as a wave of complaints. Reviewing MLflow experiment history from initial development looks backward at what was evaluated at build time, it has no visibility into what real users have been asking in production since then, and cannot detect a change in traffic composition that emerged after deployment. Re-running the original offline evaluation set on a schedule tests the system against a fixed, unchanging set of queries, which by definition cannot reveal that the live query mix has shifted toward something that set never covered in the first place, the offline set is frozen while production traffic is not. Checking a model signature only describes the declared input and output schema, a structural contract, it says nothing about the actual topical distribution of the queries being sent to the endpoint over time. Exam framing: detecting drift in what production traffic actually looks like requires monitoring live logged requests, not relying on a static offline evaluation set or build-time artifacts.