Your Kubernetes cluster runs a mix of AI workloads. Some tasks, like large language model training, require dedicated access to whole GPUs. Other tasks, like lightweight model inference or development notebooks, only need a fraction of a GPU's compute power. To maximize hardware utilization and dynamically allocate resources for both types of jobs, which configuration should you implement?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal: GPUs are expensive, and you don't want a simple notebook hogging a whole H100 while your training jobs wait. Deploy the NVIDIA GPU Operator on Kubernetes and enable GPU sharing with MIG or Time-Slicing so you can slice GPUs for small tasks and keep dedicated access for heavy training. That's how you maximize utilization without wasting hardware.
Full Explanation
In an enterprise AI infrastructure, managing GPU utilization efficiently is critical due to the high cost and power consumption of accelerator hardware. Workloads are often heterogeneous: training deep learning models requires massive parallel processing power and full GPU memory bandwidth, whereas inference tasks, testing, and debugging require far fewer resources. To address this, administrators utilize the NVIDIA GPU Operator in conjunction with Kubernetes. The NVIDIA GPU Operator automates the deployment of all NVIDIA software components needed to provision GPUs. To optimize utilization, the Operator can be configured to enable GPU sharing. This is achieved through two primary technologies: Multi-Instance GPU (MIG), available on NVIDIA Ampere and Hopper architectures, partitions a single GPU physical instance into up to seven independent GPU instances, and each instance has its own dedicated memory and SMs (Streaming Multiprocessors), guaranteeing hardware-level isolation and predictable performance; and Time-Slicing, which allows multiple CUDA applications to share a single GPU by interleaving their execution over time, does not offer hardware-level isolation like MIG, but is highly effective for oversubscribing GPUs for development environments and light inference. The approach that uses static node affinity to dedicate one GPU per pod, regardless of workload size, is incorrect because allocating one GPU per pod results in massive underutilization; lightweight pods will reserve an entire GPU, leaving its compute cores idle and preventing other workloads from running. The FIFO queue approach that runs one job at a time on the cluster, ensuring no two jobs ever share hardware, is incorrect because FIFO (First In, First Out) scheduling running one job at a time destroys cluster concurrency, creates massive queue delays, and leaves the rest of the cluster's GPUs idle while a single-GPU job is running. The approach that configures pod resource limits to request 100% of CPU resources for all pods to offload GPU utilization is incorrect because adjusting CPU limits does not solve GPU allocation issues; offloading workloads to the CPU is counterproductive for accelerated AI workloads and does not optimize GPU resource sharing.