Before distributed training on a skewed dataset, repartitioning can help by:

Short Explanation

Repartition to balance work — skew makes one executor the straggler.

Full Explanation

Partition balance affects distributed training/featurization throughput.

×