To dynamically adjust the replica count of an active AI inference service in Kubernetes based on fluctuations in query traffic or hardware resource consumption, which component should be configured?
Select an answer to reveal the explanation.
Short Explanation
When traffic spikes or GPU/CPU usage climbs, you need something that automatically adds or removes pods. That's the Horizontal Pod Autoscaler: it watches metrics and adjusts replica count for you. The scheduler places pods, Triton serves the model, and DCGM monitors GPUs, but they do not scale the service.
Full Explanation
The Horizontal Pod Autoscaler (HPA) is a standard Kubernetes controller that dynamically scales the number of replica pods in a deployment, replication controller, or stateful set. The scaling decisions are based on user-defined thresholds for system metrics like CPU utilization or custom metrics, such as HTTP request rates or GPU duty cycles exposed through Prometheus. This is critical for AI inference workloads where traffic patterns are volatile. The Kubernetes Scheduler is responsible for choosing the best node for a pod to run on based on resource availability, but it does not create or destroy replicas. NVIDIA Triton Inference Server is an open-source inference serving software that manages model execution but relies on orchestrators like Kubernetes for scaling. NVIDIA Data Center GPU Manager (DCGM) is a suite of tools for managing and monitoring NVIDIA GPUs in cluster environments; it provides telemetry data but does not perform scaling actions itself.