You are responsible for a medical AI platform that processes both critical, real-time diagnostic imaging queries and long-running genomic data analysis. The workload volume fluctuates throughout the day, but the system must maintain low latency for diagnostics and high availability overall. Which infrastructure management approach best maintains these service levels under varying demand?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal: in the real world, compute demands are never flat, so you can't slice resources down the middle or manually guess who needs what. Auto-scaling watches traffic and load, spinning up resources where needed and shrinking them when things quiet down. That keeps latency low for diagnostics and keeps your budget happy.
Full Explanation
Handling heterogeneous workloads with different latency and availability requirements requires a resource management system that can adapt to changing conditions in real time. The dynamic auto-scaling policy that allocates and redistributes compute resources based on real-time request volume and system metrics is the best approach because it dynamically adjusts compute and memory allocations based on active system load and incoming requests. This ensures that latency-sensitive tasks, like real-time diagnostics, receive immediate execution resources during traffic spikes, while batch processing tasks, like genomic modeling, can scale back or utilize spare capacity during low-demand periods.
Let's break down why the other approaches are suboptimal: - The fixed oversized slice of GPU resources permanently assigned to diagnostic tasks, with genomic analysis only during scheduled offline windows, leads to severe resource underutilization. If the diagnostic pipeline is idle, those GPUs sit wasted, while the genomic analysis remains bottlenecked. Conversely, a massive influx of diagnostics could exceed the fixed allocation, causing latency issues. - The approach that distributes the available compute clusters evenly among all active workloads to maintain scheduling parity ignores the distinct service-level agreements of different workloads. A real-time inference request requires low-latency execution and should not be throttled equally with a background batch job. - The approach that manually partitions the resource pools based on daily average execution times of the respective tasks is administrative overhead that cannot react to sudden, unexpected traffic spikes. Real-time environments require programmatic, sub-second scaling adjustments that manual intervention cannot provide.
Modern AI orchestration platforms like Kubernetes use horizontal pod autoscalers and cluster autoscalers to monitor GPU and CPU utilization, queue lengths, or custom API metrics to adjust resources dynamically. This ensures service-level agreement compliance while maintaining cost efficiency.