A B2B SaaS company hosts churn models for 180 small tenants. Each model is invoked sporadically during business hours, sits idle overnight, requires low-latency responses when called, and must avoid paying for 180 always-on endpoints. Which SageMaker inference design delivers cost-efficient low-latency scoring across tenants?
Select an answer to reveal the explanation.
Short Explanation
You get shared power without paying for idle engines. A multi-model setup keeps you fast when calls arrive and cheap when they don't, since autoscaling handles the bursts.
Full Explanation
SageMaker AI multi-model endpoint is correct because it hosts many small tenant artifacts behind one shared endpoint fleet, dynamically loading the requested model into container memory via the target-model header and caching hot models, which preserves synchronous low-latency invocation while sharing compute cost across 180 tenants and allowing instance autoscaling to follow business-hours demand instead of paying for 180 separate endpoint-hours. Optimized per-tenant real-time endpoints fail on cost efficiency by concept because even right-sized autoscaled endpoints still incur per-endpoint hourly infrastructure charges multiplied across 180 tenants, so burst handling remains good but the avoid-always-on fleet constraint is violated. Serverless inference fails on latency consistency by concept because scale-from-zero provisioning and cold starts introduce variable initialization delay and concurrency limits, so idle savings and isolation are excellent but consistent immediate response for interactive churn scoring cannot be guaranteed. Frequently scheduled batch transform fails on interaction pattern by concept because collecting queued requests and starting transform jobs every few minutes adds queueing plus startup delay and writes results to storage rather than returning a live synchronous response, which suits backfill but not sporadic on-demand scoring. For the exam, remember that scale-to-zero does not mean cheap plus fast for synchronous workloads, so always match request synchronicity to real-time versus async versus batch, and in practice verify CloudWatch metrics for model cache hits, load latency and P99 invocation latency plus autoscaling schedules for daytime ramps before consolidating tenants onto the shared fleet.