A team's agent produces excellent code review output but takes 45 seconds per review, making it too slow for developer workflows. Which evaluation and optimization strategy should be applied?
Select an answer to reveal the explanation.
Short Explanation and Infographic
A sports car that's stuck in traffic isn't useful despite its speed capability. Before changing anything, profile where the 45 seconds actually goes — it might be one slow tool call, a bloated prompt, or an oversized model for the task. Fix the real bottleneck, not just anything that might help.
Full explanation below image
Full Explanation
Optimizing agent latency requires profiling before optimizing — a common mistake is to reduce quality arbitrarily rather than targeting actual bottlenecks.
Profiling an agent pipeline: 1. Instrument each step (LLM call, tool call, context preparation) with timing. 2. Identify which step(s) consume the most time. 3. Apply targeted optimizations based on findings: - LLM call latency: Try a smaller, faster model; evaluate whether quality drops are acceptable. - Tool call latency: Parallelize independent tool calls; cache frequently accessed results. - Context size: Compress or trim unnecessary context that inflates prompt processing time. - Architecture: Break sequential chains into parallel calls where possible.
Why B is correct: Profiling + targeted optimization preserves quality where possible while addressing actual bottlenecks. Quality impact is measured empirically at each step, not assumed.
Why A is wrong: Documenting a 45-second latency as a known limitation ignores that it renders the tool impractical. Developer workflows require real-time or near-real-time feedback to remain in flow.
Why C is wrong: Rule-based systems cannot match the nuanced reasoning quality of LLM agents for code review. Switching to rules trades away the core value proposition.
Why D is wrong: 'Reduce quality until it's faster' is not a principled optimization strategy. It conflates the result (faster) with the method (lower quality) without evidence that the quality reduction is necessary or sufficient.