An enterprise deploys Claude as an internal knowledge assistant for 5,000 employees. The system must handle questions across HR policy (low stakes), IT troubleshooting (medium stakes), and financial regulatory guidance (high stakes). Average query complexity varies widely. The architecture team debates whether to use a single model tier or a dynamic routing strategy. Load testing shows 80% of queries are HR/IT, 20% are financial. The company's API spend is currently $120K/month with a single Sonnet deployment. Which routing architecture provides the best accuracy/cost tradeoff?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — an embedding-based router (not a Claude call) is architecturally superior for this multi-tier routing problem. It adds zero token cost to classification, introduces minimal latency, and cleanly separates domain/complexity detection from answering.
Full explanation below image
Full Explanation
An embedding-based router (not a Claude call) is architecturally superior for this multi-tier routing problem. It adds zero token cost to classification, introduces minimal latency, and cleanly separates domain/complexity detection from answering. Dispatching Haiku for HR, Sonnet for IT, and Opus for financial regulatory aligns model capability with task stakes — the 80/20 split means most queries hit Haiku, dramatically cutting cost from the $120K baseline. Option B uses a Claude Haiku classifier, which adds an entire API call's cost and latency to every query before routing — wasteful when embeddings can achieve the same classification. Option D's confidence-score approach is flawed because model-reported confidence is unreliable for routing decisions; the model may be confidently wrong. Option A (no routing) ignores the clear cost optimization opportunity — 80% of queries could be served by Haiku at a fraction of Sonnet cost.