When splitting an imbalanced dataset into train and test sets in Python, which approach ensures each split has the same class proportion?
Select an answer to reveal the explanation.
Short Explanation and Infographic
The stratify=y parameter is the magic word — it tells train_test_split to maintain the class proportions from the original dataset in both the train and test splits.
Full explanation below image
Full Explanation
train_test_split(X, y, stratify=y) performs stratified sampling: it splits each class independently according to test_size and then combines them. For a dataset with 99% class 0 and 1% class 1, stratification ensures both train (80%) and test (20%) sets also have approximately 99%/1% class ratios. Without stratify=y, random sampling might put all (or most) minority class examples in the training set, leaving the test set with no (or few) positive examples — making evaluation meaningless. The balance=True parameter doesn't exist in scikit-learn's train_test_split. shuffle=False disables shuffling entirely (no randomness, useful for time series). For Spark, use df.sampleBy(col, fractions={0: 0.8, 1: 0.8}) or implement stratified split using groupBy/randomSplit per stratum.