A municipal broadband team plans nightly batch summarization of incident logs and must stay inside a fixed monthly inference budget. Which constraint should drive how they size each job?
Select an answer to reveal the explanation.
Short Explanation
Tokens are like prepaid postage for each night's mail run. More log text means more stamps—and the city only bought so many for the month. Cap what each batch sends so the bill and the API limits both stay honest.
Full Explanation
Nightly batch summarization of municipal broadband incident logs is an inference job whose cost and throughput are measured in tokens—input tokens for each log chunk and output tokens for each summary. A fixed monthly inference budget therefore makes token limits per request and across the batch the controlling capacity and cost ceiling when sizing chunk length, concurrency, max output, and retry policy. Teams that ignore those ceilings overrun spend or trip provider caps even when fiber and portal metrics look healthy.
The physical length of each fiber span feeding the network operations center is a plant-engineering attribute; it does not determine how many tokens a generative summarizer will bill for a night's log text. Whether the summarizer card is generative versus embedding-only matters for choosing the right tool class, but once a generative summarizer is in use, the budget driver for job sizing remains token quotas—not the mere presence of an embedding alternative. The number of public Wi-Fi SSIDs on the open-data portal is unrelated catalog metadata and has no bearing on batch inference allotments.
Exam caveat: token ceilings cover both prompt and completion; shrinking only input while allowing unbounded summary length can still exhaust the monthly budget. Operational check: measure average tokens per incident, set hard max_tokens and chunk sizes so nightly volume times unit cost stays under the purchase, and alert when cumulative month-to-date tokens approach the civic allotment.