Your boss walks in and announces that your team has been tasked with fine-tuning a 70-billion parameter Large Language Model (LLM) on-premises. As you design the hardware architecture, which two of the following infrastructure components are most critical to prevent performance bottlenecks during the heavy tensor parallel calculations? (Choose two)
Select all correct answers, then click Submit.
Short Explanation and Infographic
Let's dive into this one because it's a classic infrastructure puzzle. LLMs are absolute beasts when it comes to memory and data transfer. During training, the GPUs need to talk to each other constantly and share parameter updates at lightning-fast speeds. If you try to run this over standard 1Gbps Ethernet or legacy storage interfaces, your GPUs will just sit there idling, waiting for data—what a waste of money! You need two things to keep those processors fed: massive, high-speed memory like High Bandwidth Memory (HBM) to hold the model, and ultra-fast interconnects like NVLink for GPU-to-GPU talk, or InfiniBand between servers. Without these, your project is dead in the water.
Full explanation below image
Full Explanation
Training and fine-tuning Large Language Models (LLMs) place extreme demands on memory bandwidth and inter-device communication. Because these models contain tens or hundreds of billions of parameters, they cannot fit on a single GPU's memory. Therefore, they must be split across multiple GPUs using tensor or pipeline parallelism. To prevent severe communication bottlenecks, high-bandwidth interconnects like NVIDIA NVLink (for intra-node GPU communication) and InfiniBand (for inter-node cluster networking) are critical, as they allow high-throughput, low-latency data transfers. Furthermore, high-capacity and high-speed accelerator memory (such as High Bandwidth Memory, or HBM) is vital to sustain the massive computational throughput required by tensor cores during forward and backward passes. Options A and F (traditional hard drives, legacy SSDs, and standard Gigabit Ethernet) do not provide the throughput or latency characteristics required to keep high-performance GPUs utilized during training. Option B (consumer-grade routers) lacks the reliability, latency profile, and throughput of enterprise interconnects. Option C (CPUs without hardware acceleration) is incapable of processing the matrix multiplication workloads of LLM training at a practical scale.