A real-time voice assistant application requires responses within 800ms end-to-end (including network and TTS). The assistant handles both simple acknowledgments and complex scheduling tasks. Streaming is enabled. An architect measures that the Sonnet-tier model averages 1200ms time-to-first-token for complex queries. The Haiku-tier model measures 350ms P95 time-to-first-token and passes the team's scheduling-accuracy eval. What is the best architectural strategy?
Select an answer to reveal the explanation.
Short Explanation
When the fast option already passes the test, there's nothing left to trade. Your measurements show the Haiku-tier model hits 350ms and clears the scheduling eval, so you run everything on it and your 800ms budget holds.
Full Explanation
Using the Haiku-tier model for every request is the best strategy because the stem supplies both pieces of evidence that decision needs. Its measured P95 time to first token of 350ms leaves room inside the 800ms end-to-end budget for network time and speech synthesis, and it passes the team's scheduling-accuracy eval, so there is no quality reason to reach for a slower model. Anthropic's model overview lists the Haiku tier as the fastest in the lineup, but tier descriptions are relative; the decision rests on the team's own latency and eval measurements, not on a label. Routing scheduling to the Sonnet-tier model deliberately accepts SLA misses on exactly the queries users care about, for a quality gain the eval says is unnecessary. The speculative pattern that switches from one model's stream to another mid-utterance adds complex state handling and can produce jarring audio in text-to-speech. Relaxing the SLA to 1500ms ignores that the requirement comes from conversational user experience. On the exam, choose the option backed by measured latency and a passing eval rather than an assumed model characteristic. Operationally, keep the scheduling eval in CI and alert when P95 time to first token drifts toward the budget.