Public-health aggregations stall because a few giant clinic facilities dominate shuffle. What problem should they diagnose?

Public-health aggregations that stall because a few giant clinic facilities dominate shuffle should be diagnosed as data skew: a small set of clinic keys owns disproportionate partition work while other keys finish early. Skew shows up as straggler tasks, uneven shuffle read/write sizes, and stage time dominated by a handful of keys—exactly the pattern mega-clinics create in municipal health workloads. Shuffle spill where large clinic blocks overflow shuffle partitions to disk fails as a diagnosis because spill reflects executor memory pressure spilling shuffle data to disk and can affect any partition, so it cannot explain why a few clinic keys own disproportionate partition work; the stall is uneven shuffle partition work in the aggregation by clinic key, not memory spill to disk. Fetch failure where remote clinic keys overload shuffle partitions on retry fails because fetch failures are retry-driven errors from lost executors or unavailable shuffle blocks and do not explain why a few clinic keys own disproportionate partition work; the stall is driven by skewed key distribution, not by shuffle fetch retry. Small-file skew where tiny clinic keys fragment shuffle partitions on write fails because small-file fragmentation comes from many tiny clinic keys creating excessive small writes for all partitions and does not make a few clinic keys dominate; the imbalance is in shuffle task distribution across clinic keys, not in fragmented write count. Exam caveat: DP-750 performance diagnosis distinguishes data skew from generic slowness and from unrelated shuffle-spill, fetch-failure, or small-file-skew concerns. Operational check: open Spark UI stage metrics for the aggregation, confirm a few clinic keys dominate shuffle bytes and task duration, then apply skew mitigation (salting, adaptive skew handling, or pre-aggregation) and re-measure stage balance.