A transit authority built a custom inference container for predictive-maintenance scoring and needs to run it at scale outside SageMaker, as a long-lived, always-on ML workload. What kind of service should the team use to run it?

A managed container orchestration service runs and scales containers across a cluster continuously, which is the operating model a long-lived, always-on custom inference workload actually needs — the service keeps instances of the container running and can scale them with demand, outside of SageMaker's own hosting layer. Lambda functions are built around short-lived, event-triggered execution with maximum duration limits, which conflicts directly with a workload meant to run continuously rather than spin up per request and exit. AWS Batch is designed for discrete, scheduled, or queued jobs that run to completion and stop, which fits periodic scoring runs but not a service meant to stay available and serving continuously. Looping a batch transform job to simulate always-on service misuses a tool built for one-time offline dataset scoring, forcing it into a pattern it wasn't designed for and losing the reliability guarantees a real continuously-running service would provide. Scope note: running a custom container outside SageMaker means the team also owns scaling policies, health checks, and rolling deployments that SageMaker hosting would otherwise manage. Operational check: confirm the orchestration service's auto-scaling and health-check configuration keeps the maintenance-scoring service available through instance replacements.