A media company runs a real-time content moderation system that evaluates user-submitted posts before publication. The system must respond within 200ms P99. Current Claude Sonnet latency for their prompt is 350ms median, 620ms P99. They are considering switching to Claude Haiku. Haiku median latency is 90ms with P99 of 180ms. Moderation accuracy drops from 96% (Sonnet) to 91% (Haiku). The false-negative rate (harmful content passing) increases from 2% to 6%. Their acceptable false-negative rate is 4%. What is the best architectural response?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — use Haiku for a fast first pass, publish clear-pass posts immediately, and send borderline cases to Sonnet asynchronously. That meets the 200ms P99 SLA while keeping harmful-content misses below your 4% threshold.
Full Explanation
A two-tier async architecture solves both constraints simultaneously. Haiku's 180ms P99 meets the 200ms SLA for clear-pass content, while borderline or Haiku-flagged content goes through Sonnet asynchronously before publication — the async queue handles the cases where 6% false negatives would otherwise be problematic. This brings effective false-negative rate well below 4% for the combined system. Switching entirely to Claude Haiku fails because a 6% false-negative rate exceeds the 4% threshold, making it non-compliant regardless of latency. Keeping Sonnet and optimizing the prompt to reduce token count may help but cannot close a 430ms P99 gap through prompt optimization alone. Deploying multiple parallel Sonnet calls, returning the first response, and averaging latency across instances violates economics and doesn't reliably hit P99 since the slowest-instance problem remains.