A county procurement agent uses a large proprietary LLM to reason about high-stakes vendor-compliance determinations, but routes routine vendor FAQ questions to a smaller, cheaper model. What is the underlying selection principle?
Select an answer to reveal the explanation.
Short Explanation
Think of it like a law office: the senior partner reviews the contract that could sink the county in litigation, while a paralegal handles the routine office-hours question. A procurement agent works the same way - the stakes of a vendor-compliance call justify the bigger model, while a simple FAQ doesn't need that firepower. It's about matching capability to what the task demands, not running one model for everything.
Full Explanation
Tiering models by task stakes lets an architect reserve the most capable - and most expensive - reasoning for decisions where getting it wrong carries real consequences, like a vendor-compliance determination that could affect a contract award, while routing lower-stakes, well-bounded questions to a smaller model that answers just as well for less cost and lower latency. This is a deliberate architectural pattern, not an accident of which model happened to be handy. Running the large model for everything and treating the small model as a mere outage fallback misreads the pattern as reactive rather than a proactive cost-and-risk design choice. Assuming a newer model automatically outperforms an older one on every task ignores that recency isn't a proxy for fitness - a smaller, well-scoped model can outperform a larger general model on a narrow, repetitive task like FAQ answering. And assuming an SLM is better suited to compliance work because it's domain-specific inverts the actual tradeoff: SLMs are chosen for speed and cost on simpler tasks, not because they out-reason a larger model on complex, high-stakes judgment calls. The scope caveat: stakes-based routing needs a reliable way to classify incoming requests by risk, or a genuinely high-stakes question can slip through to the smaller model. A concrete check: audit a sample of FAQ-routed requests to confirm none of them were actually compliance-sensitive questions in disguise.