A swim-club FAQ bot gets expensive after members paste entire handbooks into the prompt. Replies also get slower and drop useful club rules. Which explanation is accurate?
Select an answer to reveal the explanation.
Short Explanation
Think of paying by the slice and then stuffing the whole handbook into the sandwich. Token bills go up, replies get slower, and useful club rules get squeezed out. More tokens are not cheaper, and this is not a GPU-temperature fee.
Full Explanation
Under token-based pricing, cost scales with the number of tokens read and written. When swim-club members paste entire handbooks into the FAQ prompt, the bill rises, latency often worsens, and the finite context window can crowd out the club rules that actually answer the question. That single explanation covers both the expense spike and the quality drop that staff observed after full-handbook pastes.
Billing per document so that pasting a handbook cannot change the bill misstates typical generative pricing: document count is not the unit when the meter is tokens. Claiming more tokens always lower cost reverses the economics; more input and output tokens increase spend, they do not make the model cheaper when it reads more. Calling token price a GPU-temperature fee invents a billing unit unrelated to text pieces processed by the model.
Exam caveat: tokens affect both money and quality. Extra context is not free, and stuffing the window can displace the very policy snippets the FAQ bot needs to answer correctly.
Operational check: enforce retrieval of short, relevant handbook sections instead of full-paste, set max prompt size on the FAQ bot, and alert when average tokens per turn or cost per day crosses a swim-club budget threshold.