A legal technology firm benchmarks Claude models for contract review. Their tests show that on standard commercial contracts under 50 pages, Claude Haiku and Claude Sonnet achieve nearly identical precision on clause extraction (within 1%). On complex M&A agreements over 200 pages with cross-references, Sonnet outperforms Haiku by 12% precision. The firm handles 10,000 contracts per month: 8,500 are standard commercial, 1,500 are complex M&A. Current spend with Sonnet on all contracts is $18,000/month. Haiku pricing is approximately one-fifth of Sonnet. What is the expected monthly cost with optimally routed model selection?
Select an answer to reveal the explanation.
Short Explanation
You route the 8,500 standard contracts to Haiku and keep the 1,500 complex M&A contracts on Sonnet. That captures Haiku's lower cost where precision is nearly identical while preserving Sonnet's precision on complex deals. The expected monthly cost lands around $7,560.
Full Explanation
Optimal routing sends 8,500 standard contracts to Haiku, priced at about one-fifth of Sonnet, and 1,500 complex M&A contracts to Sonnet. Current Sonnet-all spend is $18,000 for 10,000 contracts, so uniform Sonnet cost is $1.80 per contract and Haiku is $0.36. Uniformly, standard contracts cost 8,500 × $0.36 = $3,060, complex contracts cost 1,500 × $1.80 = $2,700, totaling $5,760. That estimate is too low because complex M&A agreements are longer, contain cross-references, consume more tokens, and carry higher per-contract cost. The approximately $7,560/month answer is expected because it accounts for higher token usage on complex documents while still routing standard contracts to Haiku. The approximately $3,600/month all-Haiku option is wrong because it accepts the 12% precision drop on complex M&A, unacceptable for high-value deals. The approximately $9,000/month option adding 50% cost to Haiku is wrong because the facts support pricing and workload mix, not an unsupported 50% routing overhead. The approximately $14,400/month option routing only the simplest 20% to Haiku is wrong because it underutilizes Haiku's cost advantage and leaves too many standard contracts on Sonnet.