A shared multi-tenant NVIDIA GPU cluster is running several machine learning workloads. Recently, non-critical background batch jobs have been consuming disproportionate GPU memory and compute cycles, preventing high-priority, real-time inference services from executing reliably. How should the infrastructure administrator configure the cluster to guarantee resource availability and priority for the critical services?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: if you have a shared GPU cluster without any traffic cops, the loudest and heaviest workloads will hog all the lanes, leaving your critical apps stuck in gridlock. Upgrading the GPUs is just a temporary band-aid—those non-critical jobs will just expand to fill the new space. You need a scheduler to enforce the rules. In the real world, we do this using Kubernetes. By setting up GPU requests, limits, and quotas, you tell the cluster exactly how much GPU horsepower each namespace or pod is allowed to grab. You can prioritize your critical real-time apps so they get first dibs on the hardware, and limit the low-priority tasks so they don't starve everyone else. Trust me, Kubernetes resource management is the way to keep the peace.
Full explanation below image
Full Explanation
In a multi-tenant or shared GPU cluster, resource starvation is a common issue when workloads are not actively governed. To prevent low-priority batch jobs from consuming excessive GPU resources and degrading high-priority, latency-sensitive services, administrators must implement container orchestration resource controls. Kubernetes, when integrated with the NVIDIA Device Plugin, allows administrators to schedule GPUs just like CPU and memory resources.
By defining: - Resource Requests: The minimum amount of GPU resources a container needs to be scheduled. - Resource Limits: The maximum amount of GPU resources a container is allowed to consume. - ResourceQuotas: Cluster-level limits per namespace to restrict total GPU consumption. - PriorityClasses: Kubernetes objects that define the scheduling queue order, allowing the orchestrator to preempt (evict) lower-priority pods to free up GPU capacity for incoming high-priority pods.
Let's look at the other options: Option A (moving all low-priority to CPUs) is inefficient and heavy-handed. Low-priority jobs may still benefit from GPU acceleration; they simply need to be throttled or preempted when high-priority tasks require resources. Option B (upgrading physical hardware) is expensive and does not solve the underlying resource-scheduling problem; larger GPUs will still be subject to starvation if there are no scheduler constraints. Option C (model optimization) reduces the footprint of the high-priority model but does not prevent a low-priority job from scaling up and consuming the remaining cluster resources.