A financial analytics application submits complex multi-step calculations to Claude. Occasionally, responses are cut off mid-calculation due to reaching max_tokens. The team sets max_tokens to 4,096 to prevent truncation, but this creates over-provisioning cost for short responses. What is the correct approach?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — d is correct because dynamic max_tokens sizing with a truncation-detection retry is the correct engineering pattern. Pre-classifying requests by expected output length allows appropriate provisioning (short calculations get 512 tokens, complex multi-step calculations get 4,096).
Full explanation below image
Full Explanation
D is correct because dynamic max_tokens sizing with a truncation-detection retry is the correct engineering pattern. Pre-classifying requests by expected output length allows appropriate provisioning (short calculations get 512 tokens, complex multi-step calculations get 4,096). Checking stop_reason == max_tokens after each response provides a reliable truncation signal that triggers a retry with a higher limit. This approach optimizes cost by not over-provisioning short requests while ensuring long calculations complete. A is wrong because while Anthropic bills for generated tokens not the ceiling, a high max_tokens ceiling can cause issues with certain response patterns and the principle of appropriate resource provisioning applies even when costs are output-proportional. The more important flaw is that this answer does not address the structural problem. C is wrong because detecting 'completion' by parsing streaming output for a final result pattern is brittle for mathematical calculations — partial results can appear to be complete, and stream cancellation adds operational complexity without the reliability of the retry-on-truncation approach. B is wrong because instructing Claude to 'limit to 500 tokens' for complex multi-step calculations would produce incomplete results rather than concise ones — the content genuinely requires more tokens, and truncating it causes incorrect outputs.