A multi-turn conversational assistant stores conversation history and sends it with each Claude API call. After 6 months in production, the team notices that average token counts per API call have grown from 2,000 to 18,000 tokens as users accumulate longer conversation histories. API costs have increased 9x. The team considers either truncating old messages or summarizing them. For a customer support context where customers frequently reference issues discussed 10+ turns ago, which strategy preserves the most relevant context while controlling cost?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — rolling summarization maintains semantic continuity across long conversations while bounding token growth. When the context window fills, the oldest turns are condensed into a running summary that preserves key decisions, stated preferences, and unresolved issues — exactly the information customers reference when they say 'like I mentioned earlier.' The summary grows slowly (logarithmically relative to conversation length) compared to raw message storage (linear growth).
Full explanation below image
Full Explanation
Rolling summarization maintains semantic continuity across long conversations while bounding token growth. When the context window fills, the oldest turns are condensed into a running summary that preserves key decisions, stated preferences, and unresolved issues — exactly the information customers reference when they say 'like I mentioned earlier.' The summary grows slowly (logarithmically relative to conversation length) compared to raw message storage (linear growth). Option A (truncate to last 10) is simple but directly breaks the use case — customers referencing issues from turn 5 in turn 30 will find Claude has no memory of the earlier discussion. Option C (RAG over messages) is clever but introduces retrieval failure modes — relevant context might not be retrieved if query framing differs from the original message content. Option D (session termination) actively harms the customer experience in a support context where issue continuity matters.