A SaaS company is evaluating model selection for a code review assistant that processes pull requests. PRs range from 500 to 45,000 lines of diff. The assistant must: identify security vulnerabilities, suggest performance improvements, enforce style guidelines, and explain its reasoning. The company's P99 PR size is 8,000 lines. A product manager argues that Claude 3 Haiku is sufficient 'because code review is just pattern matching.' The engineering team disagrees. Which argument most precisely captures why the engineering team is correct for the P99 case?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — the engineering team's strongest argument is cross-file reasoning coherence. Security vulnerabilities in production code are rarely simple single-file patterns — they emerge from interactions between components: a variable tainted in module A, passed through module B, unsanitized into module C.
Full explanation below image
Full Explanation
The engineering team's strongest argument is cross-file reasoning coherence. Security vulnerabilities in production code are rarely simple single-file patterns — they emerge from interactions between components: a variable tainted in module A, passed through module B, unsanitized into module C. This requires the model to maintain a mental model of the entire diff's semantics while evaluating each component. This is exactly where capability tier differentiates: higher-capability models maintain more coherent cross-context reasoning across large code diffs. Option A is factually incorrect — all Claude 3 models support 200K context, so an 8,000-line diff (even at 10 tokens/line = 80,000 tokens) is within Haiku's context window. Option C is partially correct about the performance reasoning point but incorrectly implies style enforcement is 'adequate' for Haiku — the question asks for the most precise argument about P99 case specifically. Option D makes an unsupported assertion about training data volume differences; the real differentiator is reasoning capability, not code training volume.