When comparing k-means clustering and hierarchical clustering for a dataset with 500,000 observations in SAS, which statement is most accurate?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — k-means scales better on 500,000 observations because it needs only O(nkiterations) work per run, while hierarchical clustering needs O(n^2) to O(n^3) time and O(n^2) space. You'll hit memory and runtime walls with hierarchical clustering at that size.
Full Explanation
The accurate statement is the one saying k-means scales better to large datasets because it is O(n) per iteration, while hierarchical clustering is O(n^2) to O(n^3). Agglomerative hierarchical clustering needs a pairwise distance matrix, O(n^2) space and O(n^2) to O(n^3) time, making 500,000 observations infeasible. K-means reassigns points to nearest centroids, O(nkiterations), so it scales far better. The claim that hierarchical clustering is faster because it uses one pass is wrong; it needs multiple passes and huge memory. The claim that complete linkage always gives better quality is wrong; linkage choice does not guarantee quality. The claim that both have identical complexity over 100,000 rows is wrong; their complexities differ significantly.