A school district trains a small eligibility-scoring model per individual school — dozens of near-identical models — and wants to avoid provisioning a separate costly endpoint for each one. What deployment pattern fits best?
Select an answer to reveal the explanation.
Short Explanation
Dozens of nearly identical little models don't each need their own standing endpoint — that's paying rent on dozens of apartments to store one box each. A multi-model endpoint keeps them all behind one door and pulls the right one off the shelf as requests come in. One endpoint, many models, a fraction of the cost.
Full Explanation
A SageMaker multi-model endpoint hosts many similar models behind a single endpoint, loading each model into memory on demand and caching frequently used ones, which lets dozens of small per-school models share infrastructure instead of each paying for its own dedicated endpoint. Running one endpoint per school technically works but multiplies fixed per-endpoint costs by the number of schools, which is precisely the waste the district wants to avoid — smaller instances reduce but don't eliminate that per-endpoint overhead. A multi-container serial pipeline solves a different problem: chaining distinct processing steps (like preprocessing then scoring) in a fixed sequence for one request, not hosting many interchangeable models that each serve a different school's traffic independently. Nightly batch transform per school trades away real-time scoring and still runs a separate job per school, so it neither restores interactivity nor meaningfully consolidates infrastructure. Scope note: multi-model endpoints work best when individual models are similar in size and framework, which fits near-identical per-school models well. Operational check: monitor per-model load latency to confirm cold-loading a less-used school's model doesn't violate response-time expectations.