A county 311 call-center intent-classification model receives sporadic, unpredictable bursts of traffic with long idle gaps between citizen contacts. The team wants to avoid paying for capacity that sits idle. Which inference option fits best?
Select an answer to reveal the explanation.
Short Explanation
Long idle gaps followed by sudden bursts is exactly the traffic pattern serverless inference was built for — it scales itself down to nothing when calls stop coming and spins back up when they do. Paying for a standing endpoint through all those quiet hours would mean paying for capacity nobody's using. Match the compute to how the traffic actually behaves.
Full Explanation
Serverless inference automatically provisions and de-provisions compute in response to incoming requests, so during long idle gaps it costs nothing, and it scales up when a burst of citizen contacts arrives — a direct match for sporadic, unpredictable traffic where a standing endpoint would sit unused most of the time. A real-time endpoint sized for the largest burst solves the availability problem but fails the cost goal the team stated: it holds that peak-sized capacity around the clock, including through every idle gap. A multi-model endpoint reduces cost by sharing infrastructure across several models, but it's still an always-on real-time endpoint underneath, so it doesn't address the idle-capacity waste this single classifier's bursty pattern creates. Batch transform processes a full backlog as one offline job, which trades away the ability to classify a call's intent as it happens — useful for retrospective analysis, not for the live call flow this workload implies. Scope note: serverless carries a cold-start latency on the first request after an idle period, which is an acceptable tradeoff here but wouldn't be for a strict sub-second SLA. Operational check: review CloudWatch invocation and cold-start metrics after a few weeks to confirm the burst pattern still fits serverless scaling limits.