A real-time fraud detection system uses Claude to analyze transaction patterns. The system must evaluate each transaction within 800ms end-to-end (including network round-trips). Input context per transaction is approximately 3,000 tokens including 30-day transaction history. The fraud team wants Claude to explain its reasoning for every flagged transaction. Current architecture uses Claude 3 Sonnet and achieves 1,400ms P95. Which architectural change most directly addresses the latency requirement without sacrificing explainability?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — option B is the architecturally superior solution because it addresses both requirements simultaneously. Switching to Haiku dramatically reduces TTFT and generation speed for 3,000-token inputs — meeting the 800ms SLA.
Full explanation below image
Full Explanation
Option B is the architecturally superior solution because it addresses both requirements simultaneously. Switching to Haiku dramatically reduces TTFT and generation speed for 3,000-token inputs — meeting the 800ms SLA. Critically, streaming with partial-result rendering means the fraud flag decision (the first substantive output token) can be emitted and acted on immediately while the explanation continues streaming to the analyst interface — preserving explainability without blocking on full response completion. Option A switches to Haiku but ignores that the 800ms requirement includes full response generation for the explanation — streaming is the key architectural insight for decoupling the decision gate from explanation delivery. Option C reduces context window, potentially harming fraud detection accuracy by losing older pattern signals — a capability tradeoff that should be tested carefully, and the latency saving may be insufficient. Option D (prompt caching) helps with repeated system prompts but transaction data is unique per call — cache hit rates on the variable transaction history will be low, providing marginal latency improvement.