An enterprise needs to scale its machine learning pipeline to handle unpredictable surges in both model training and user inference requests. They want to avoid massive upfront hardware costs for peak capacity while ensuring the system does not crash or slow down during usage spikes. What is the most effective scaling strategy for this architecture?
Select an answer to reveal the explanation.
Short Explanation
If your boss asks for ten times the training jobs next week but you can't buy fifty GPU servers that'll sit idle, don't overbuy local hardware. Run baseline workloads on your on-prem GPUs and burst to public cloud GPUs during spikes. That gives you cost control and elastic capacity without crashing.
Full Explanation
A hybrid cloud deployment strategy is ideal for handling highly variable and unpredictable AI workloads. On-premises GPU infrastructure represents a sunk capital expenditure (CapEx) with low operating costs (OpEx) per hour of use. It is highly efficient for running steady-state, baseline workloads. However, provisioning on-premises hardware to handle peak, unpredictable spikes is financially impractical due to low average utilization. By integrating cloud-based GPU instances, an organization can implement a 'cloud bursting' strategy, dynamically scaling out to the cloud during high demand and scaling back down when the surge subsides.
Let's analyze why other strategies are less suitable: - The fixed local cluster approach that scales based strictly on host CPU utilization is ineffective for GPU-heavy workloads. A GPU training job can fully saturate GPU memory and cores while leaving the host CPU mostly idle. Furthermore, a fixed physical cluster cannot scale beyond its physical limit, regardless of the scaling policy. - The static round-robin scheduling approach merely distributes existing loads across a fixed set of resources; it does not add compute capacity or address resource exhaustion during global demand spikes. - The single massive monolithic cloud VM approach introduces a single point of failure (SPOF) and is highly inefficient. Paying for a massive, multi-GPU cloud instance 24/7 to accommodate occasional peak loads leads to extreme cost inefficiencies.
By utilizing hybrid orchestrators (like hybrid Kubernetes clusters), workloads can transition seamlessly between local and cloud nodes, maintaining performance and minimizing service disruption.