An emergency chatbot deployed on a serverless SageMaker endpoint shows noticeable delay on the first request after periods of low traffic, and residents report the assistant feels unresponsive at the start of a conversation. Which endpoint configuration change best addresses this cold-start delay?
Select an answer to reveal the explanation.
Short Explanation
A serverless endpoint that's gone quiet is like a store that locks its doors between customers — the first person in after a lull waits while someone unlocks up and turns the lights on. Provisioned concurrency keeps a set of instances pre-warmed and the doors effectively open, so that first request after a quiet stretch gets served immediately instead of waiting through a cold start.
Full Explanation
Serverless inference endpoints scale compute down to zero during idle periods to save cost, and spinning a fresh instance back up to handle the next request takes measurable time — that's the cold start. Provisioned concurrency keeps a specified number of instances initialized and ready even when idle, so requests that would otherwise hit a cold instance get routed to an already-warm one instead. Lowering maximum concurrency controls how many requests can be handled in parallel once instances are running; it has no bearing on how long a fresh instance takes to initialize, so it doesn't touch the cold-start problem. A larger model artifact in S3 doesn't reduce initialization time — if anything, a larger artifact to download and load tends to lengthen a cold start, not shorten it. Switching to asynchronous inference changes the endpoint into a queue-based, non-interactive pattern meant for workloads that can tolerate delayed responses, which works against the low-latency, conversational needs of a chatbot rather than solving the intermittent-traffic cold-start issue. Scope caveat: provisioned concurrency reintroduces a baseline cost even during idle periods, so it's a tradeoff between cost and the number of residents who might otherwise hit a cold start. Operational check: measure response time for the first request after a deliberate idle period, with and without provisioned concurrency enabled, to confirm the improvement.