Why do data center operators deploy specialized hardware accelerators—such as GPUs, TPUs, or NPUs—instead of relying exclusively on general-purpose CPUs to execute deep learning workloads?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of a CPU as a genius mathematician. It can solve incredibly complex problems, but it does them one at a time, sequentially. Now, think of a GPU or a TPU as thousands of elementary school kids doing simple addition all at the same time. If you have to do millions of simple calculations—like multiplying matrices for neural networks—the army of kids will finish way before the single genius. That is parallel processing in a nutshell, and it's why specialized accelerators run circles around general CPUs when it comes to training and running AI models.
Full explanation below image
Full Explanation
Deep learning workloads are computationally unique because they are composed of massive volumes of simple, independent mathematical operations, primarily matrix multiplications and accumulations.
Traditional Central Processing Units (CPUs) are designed for general-purpose, sequential computing. A standard enterprise CPU contains a relatively small number of highly optimized, powerful execution cores (typically 8 to 64 cores per socket) accompanied by large caches. These cores are built to switch tasks rapidly, handle complex branching logic, and run single instruction streams with minimal latency.
In contrast, specialized AI accelerators like Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), and Neural Processing Units (NPUs) are built for throughput. They contain thousands of simpler, smaller execution cores designed to operate in parallel. Under a SIMT (Single Instruction, Multiple Threads) or SIMD architecture, these cores can execute the same mathematical operation on large blocks of data simultaneously. This parallel architecture allows accelerators to process the massive matrices of neural networks much faster and with greater energy efficiency per FLOP (Floating Point Operation) than a general-purpose CPU.
Let's look at why the other options are incorrect: - Option A (Central control hubs): AI accelerators do not manage data center facilities, power distribution, or cooling units. Those systems are controlled by Building Management Systems (BMS) and Data Center Infrastructure Management (DCIM) software. - Option B (Data replication and snapshots): Accelerators do not manage backup or storage disaster recovery. These functions are handled by storage controller software, file systems, and database management engines. - Option D (Near-zero power consumption): While accelerators are more energy-efficient per computation than CPUs, they still consume massive amounts of electricity (often 300W to 700W+ per chip) and do not reduce server power to near-zero levels.