A bank's engineering team is deploying a complex, multi-layer neural network for real-time transaction scoring. During high-traffic events, the model's memory footprint exceeds the capacity of a single GPU, resulting in out-of-memory (OOM) errors and severe latency spikes that delay fraud detection decisions. Which of the following architectural strategies is the most appropriate for resolving these resource constraints and stabilizing transaction latency?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: deep learning models can get absolutely massive. When you're trying to run real-time fraud detection under a heavy load, time is money. If your model is so big that it's choking a single GPU's memory—or worse, crashing with Out of Memory errors—you've got a bottleneck. Think of it like trying to fit a massive engine into a tiny sports car. It's not going to fit. The solution? Model parallelism. We slice that massive model up and distribute the different layers across multiple GPUs. That way, they share the heavy lifting and the memory load. Running this on a CPU cluster will only make it slower—CPUs just don't have the parallel horsepower. And adding more data to train doesn't fix a live runtime memory bottleneck. Got it? Let's keep rolling.
Full explanation below image
Full Explanation
When deep learning models grow too large to fit within the physical memory (VRAM) of a single graphics processing unit (GPU), or when their execution latency on a single device violates real-time service-level agreements (SLAs), engineers must partition the workload. Model parallelism is an infrastructure strategy where different parts of a single neural network—such as specific layers or attention heads—are allocated to different physical GPUs. As an input tensor passes through the pipeline, each GPU computes its designated portion of the network and passes the intermediate activations to the next device. This allows the system to support massive models and maintain low-latency inference during peak traffic loads without suffering from device out-of-memory errors.
Let's analyze why other strategies are inappropriate: - Option A is incorrect because migrating complex deep learning inference workloads to CPU clusters typically degrades performance. CPUs lack the high-performance tensor cores and massive memory bandwidth required for high-throughput, low-latency execution of deep neural networks. - Option B is incorrect because increasing the training dataset size addresses model accuracy and generalization, but does not resolve runtime memory constraints or compute bottlenecks during live inference. - Option C is incorrect because pruning dataset features or reverting to simple linear classification alters the core algorithm, potentially sacrificing the high accuracy needed for fraud detection. The goal is to optimize the infrastructure to run the desired model, not compromise model capabilities.
Therefore, splitting the model parameters across multiple GPUs is the only viable infrastructure-level solution to resource exhaustion.