A city CIO asks security why the 311 assistant sometimes drops the start of a resident’s long case history mid-conversation. Which GenAI foundation concept best explains that behavior?
Select an answer to reveal the explanation.
Short Explanation
Picture a sticky note pad that only holds so many words — once residents paste a novel-length case history, the oldest notes fall off the pad. That is the LLM context window at work, not a broken certificate or sticky session. When early instructions or privacy rules get truncated, leakage and odd answers get more likely.
Full Explanation
Large language models process a finite context window measured in tokens. That window holds system instructions, conversation history, tool results, and the latest resident message. When a city 311 assistant receives a novel-length case history, earlier turns can fall outside the limit and are truncated or summarized away, so the model appears to "drop" the start of the conversation even though networking remains healthy. The correct answer names that token-budget behavior as the foundation concept behind the CIO's symptom.
TLS certificate expiry can break encrypted sessions or trigger browser warnings, but it does not selectively erase the beginning of prompt history while later turns still generate. DNS caching an outdated A record can send clients to the wrong host or cause intermittent reachability failures; it does not explain mid-conversation loss of prior text inside an otherwise responding assistant. Disabling sticky sessions on chatbot pods can move a resident to a different replica without shared memory, which may feel like amnesia if session state is not externalized—but the stem describes truncation of long case history content, which maps to context-window limits rather than load-balancer affinity.
Exam caveat: sticky-session loss and context truncation can look similar in tickets. Distinguish them by checking whether history is missing because it never fit in the prompt versus because the application failed to reload prior turns from a session store. Also remember that "tokens" are not characters; multilingual text, code, and long identifiers consume budget unevenly.
Operational check: instrument prompt token counts and hard limits for the 311 stack; log when truncation or history summarization fires; keep authoritative case details in a retrieval or CRM system outside the LLM context; and validate that system instructions and privacy rules remain inside the window after long pastes. For civic assistants, design for retrieval of case IDs rather than pasting entire histories into every turn.