A real-time fraud detection system uses Claude to analyze transaction patterns. The system must evaluate each transaction within 800ms end-to-end (including network round-trips). Input context per transaction is approximately 3,000 tokens including 30-day transaction history. The fraud team wants Claude to explain its reasoning for every flagged transaction. Current architecture uses Claude 3 Sonnet and achieves 1,400ms P95. Which architectural change most directly addresses the latency requirement without sacrificing explainability?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — option B is the architecturally superior solution because it addresses both requirements simultaneously. Switching to Haiku dramatically reduces TTFT and generation speed for 3,000-token inputs — meeting the 800ms SLA.
Full Explanation
Option B is the architecturally superior solution because it addresses both requirements simultaneously. Switching to Haiku dramatically reduces TTFT and generation speed for 3,000-token inputs — meeting the 800ms SLA. Critically, streaming with partial-result rendering means the fraud flag decision (the first substantive output token) can be emitted and acted on immediately while the explanation continues streaming to the analyst interface — preserving explainability without blocking on full response completion. Option A switches to Haiku but ignores that the 800ms requirement includes full response generation for the explanation — streaming is the key architectural insight for decoupling the decision gate from explanation delivery. Option C reduces context window, potentially harming fraud detection accuracy by losing older pattern signals — a capability tradeoff that should be tested carefully, and the latency saving may be insufficient. Option D (prompt caching) helps with repeated system prompts but transaction data is unique per call — cache hit rates on the variable transaction history will be low, providing marginal latency improvement.