Your team is benchmarking a transformer model across a cluster of eight NVIDIA H100 GPUs. Some of the GPUs are suspected of thermal throttling, leading to highly variable response times. You need to present a single visualization that clearly displays the distribution of inference latency for each GPU, specifically highlighting the median latency, variability, and any statistical outliers (extreme latency spikes). Which graphical representation is best suited for this task?
Select an answer to reveal the explanation.
Short Explanation
Check this out: when you're benchmarking eight H100s, averages can hide thermal-throttling spikes. Side-by-side box plots showing each GPU's latency distribution give you the median, the middle 50%, and outlier dots in one view. If one GPU's box is taller or has sky-high outliers, you know where to dig.
Full Explanation
In performance profiling and systems monitoring, understanding the statistical distribution of performance metrics (like inference latency) is critical. Latency data is rarely normally distributed and often contains significant tail latency or outliers caused by resource contention, thermal throttling, or hardware faults.
The box plot (box-and-whisker plot) is a standardized way of displaying the dataset based on a five-number summary: minimum, first quartile, median, third quartile, and maximum. Outliers are typically defined as data points that fall more than 1.5 times the interquartile range below the first quartile or above the third quartile, and they are plotted individually as points. When analyzing multiple GPUs side-by-side, box plots allow an engineer to instantly compare the spread (variability), skewness, and median latency of each GPU, making it trivial to spot anomalous hardware that exhibits high tail latency or outlier spikes.
The incorrect choices don't meet the requirement. The dual-axis scatter plot mapping GPU core temperature against average latency might show correlation, but it does not cleanly isolate the distribution profile or easily highlight statistical outliers across multiple distinct categorical entities (different GPUs). The stacked area chart detailing total cumulative inference time per card shows cumulative values over time, which obscures individual GPU performance profiles and hides outliers within the aggregated totals. The temporal heatmap showing real-time latency fluctuations on a continuous timeline is useful for representing density or patterns over time, but it does not provide a clear mathematical definition or clean visualization of statistical outliers and quartiles for comparative analysis between discrete hardware nodes.
Thus, side-by-side box plots are the optimal choice for visualizing performance distribution and identifying hardware-specific latency outliers.