A SaaS company is migrating from Claude 3 Opus to Claude 3.5 Sonnet for their enterprise knowledge base assistant. Internal evaluations show Claude 3.5 Sonnet outperforms Claude 3 Opus on their specific tasks and is significantly cheaper. However, during shadow testing, they observe that 3.5 Sonnet occasionally produces more concise answers where Opus was more exhaustive, and enterprise customers have complained about missing detail. What is the correct architectural response before completing the migration?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — model migrations often require prompt recalibration because newer models have different default behaviors, including default verbosity levels. The correct approach is to update the system prompt to explicitly request the level of detail required by enterprise customers, then re-evaluate.
Full explanation below image
Full Explanation
Model migrations often require prompt recalibration because newer models have different default behaviors, including default verbosity levels. The correct approach is to update the system prompt to explicitly request the level of detail required by enterprise customers, then re-evaluate. Claude 3.5 Sonnet is capable of producing exhaustive responses when instructed to do so — the behavior difference is a prompt tuning issue, not a model capability ceiling. Option A abandons a cost-effective model prematurely based on a correctable behavioral difference. Option B degrades product quality without customer consent. Option D (post-processing expansion) is an antipattern: it adds latency, cost, and complexity while producing stitched-together responses that lack the coherence of a single well-prompted generation.