Within MLOps, what does model serving (model service) mean?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Model serving is the "open for business" moment: the trained model sits behind an API, queue, or batch job and answers prediction requests from real apps. Training built the brain; evaluation graded it; architecture drew the blueprint. Serving hands the brain a microphone. Think restaurant pass-through window—orders in, dishes out. Exam trap: calling training or test scoring "serving." If the boss says customers need real-time scores in the checkout app, you're talking model service—latency, scaling, versioning, health checks. Got it? Sweet. Remember: serving equals inference access for external systems, not fitting weights or sketching layers.
Full explanation below image
Full Explanation
In MLOps, model serving (sometimes called a model service) is the operational capability that makes a trained machine learning model available for inference by other software systems. Serving stacks load model artifacts, manage resources (CPU/GPU/memory), expose interfaces such as REST, gRPC, or message-driven endpoints, and return predictions with appropriate authentication, logging, and scaling. The outcome is that product applications, microservices, or batch pipelines can consume model outputs without embedding training logic themselves.
Serving is distinct from earlier lifecycle stages. Training optimizes parameters on historical data and produces an artifact; it does not inherently provide a production inference endpoint. Validating performance on a held-out test set assesses generalization before release; evaluation metrics inform go/no-go decisions but are not the serving process. Designing model architecture—depth, width, attention heads, feature modules—belongs to research and development. Once a candidate is approved, serving concerns shift to latency SLOs, throughput, autoscaling, canary releases, multi-model hosting, feature retrieval at request time, and safe rollout of new versions.
Production serving patterns include online real-time APIs, streaming inference, and offline batch scoring. Teams often pair serving with monitoring for drift, errors, and resource saturation, and with a model registry that tracks which version is live. Common pitfalls include train/serve skew in feature pipelines, insufficient load testing, and lacking rollback paths when a new version misbehaves. For certification questions, map "model service/serving" specifically to enabling external applications to obtain predictions from a trained model—not to training, test evaluation, or architecture design. Capacity planning, GPU sharing, request batching, and graceful degradation under load are everyday serving concerns. If the scenario mentions APIs, clients, latency budgets, or production inference, the answer is serving.