A media company runs a real-time content moderation system that evaluates user-submitted posts before publication. The system must respond within 200ms P99. Current Claude Sonnet latency for their prompt is 350ms median, 620ms P99. They are considering switching to Claude Haiku. Haiku median latency is 90ms with P99 of 180ms. Moderation accuracy drops from 96% (Sonnet) to 91% (Haiku). The false-negative rate (harmful content passing) increases from 2% to 6%. Their acceptable false-negative rate is 4%. What is the best architectural response?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — a two-tier async architecture solves both constraints simultaneously. Haiku's 180ms P99 meets the 200ms SLA for clear-pass content, while borderline or Haiku-flagged content goes through Sonnet asynchronously before publication — the async queue handles the cases where 6% false negatives would otherwise be problematic.
Full explanation below image
Full Explanation
A two-tier async architecture solves both constraints simultaneously. Haiku's 180ms P99 meets the 200ms SLA for clear-pass content, while borderline or Haiku-flagged content goes through Sonnet asynchronously before publication — the async queue handles the cases where 6% false negatives would otherwise be problematic. This brings effective false-negative rate well below 4% for the combined system. Option A fails because a 6% false-negative rate exceeds the 4% threshold, making it non-compliant regardless of latency. Option B (shorter prompts) may help but cannot close a 430ms P99 gap through prompt optimization alone. Option D (parallel Sonnet calls) violates economics and doesn't reliably hit P99 since the slowest-instance problem remains.