An enterprise needs to scale its machine learning pipeline to handle unpredictable surges in both model training and user inference requests. They want to avoid massive upfront hardware costs for peak capacity while ensuring the system does not crash or slow down during usage spikes. What is the most effective scaling strategy for this architecture?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Imagine your boss walks in and says we need to run ten times as many training jobs next week because of a new feature, but we can't afford to buy fifty new GPU servers that will sit empty next month. What do you do? If you buy enough local hardware for the worst-case spike, you're going to burn a hole in your budget. If you don't buy enough, your services will crash during peak hours. Here's the deal: you build a hybrid cloud. You run your everyday, predictable training workloads on your local, cost-effective on-premises GPUs. Then, when a giant spike hits, you 'burst' those workloads out into the public cloud, renting GPUs by the minute. It gives you the best of both worlds: control and lower costs for baseline stuff, and infinite scalability when things get crazy. Trust me, this is how you design a resilient architecture.
Full explanation below image
Full Explanation
A hybrid cloud deployment strategy is ideal for handling highly variable and unpredictable AI workloads. On-premises GPU infrastructure represents a sunk capital expenditure (CapEx) with low operating costs (OpEx) per hour of use. It is highly efficient for running steady-state, baseline workloads. However, provisioning on-premises hardware to handle peak, unpredictable spikes is financially impractical due to low average utilization. By integrating cloud-based GPU instances, an organization can implement a 'cloud bursting' strategy, dynamically scaling out to the cloud during high demand and scaling back down when the surge subsides.
Let's analyze why other strategies are less suitable: - Dynamic scaling based on CPU usage (Option A) is ineffective for GPU-heavy workloads. A GPU training job can fully saturate GPU memory and cores while leaving the host CPU mostly idle. Furthermore, a fixed physical cluster cannot scale beyond its physical limit, regardless of the scaling policy. - Round-robin scheduling (Option B) merely distributes existing loads across a fixed set of resources; it does not add compute capacity or address resource exhaustion during global demand spikes. - A single large monolithic VM (Option D) introduces a single point of failure (SPOF) and is highly inefficient. Paying for a massive, multi-GPU cloud instance 24/7 to accommodate occasional peak loads leads to extreme cost inefficiencies.
By utilizing hybrid orchestrators (like hybrid Kubernetes clusters), workloads can transition seamlessly between local and cloud nodes, maintaining performance and minimizing service disruption.