A data scientist applies StandardScaler to all data before using CrossValidator with 5 folds. What error does this introduce?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Pre-scaling before CV is the classic leakage mistake — the scaler sees all 5 folds including whichever one is the validation set, so evaluation metrics look better than they'll be in production.
Full explanation below image
Full Explanation
If you StandardScaler.fit(all_data) before CrossValidator, the scaler's mean/std statistics include information from whichever fold is used as the validation fold in each CV iteration. When the model is then evaluated on the validation fold, the features were scaled using that fold's own statistics — giving the scaler prior knowledge of the validation data. The validation metrics are optimistically biased because the scaler implicitly leaks validation information. Correct approach: put StandardScaler inside the Pipeline before the estimator, then pass the Pipeline to CrossValidator. CrossValidator.fit(train_data) will correctly call scaler.fit_transform(train_fold) and scaler.transform(val_fold) separately for each fold. This is one of the most common and subtle ML bugs in practice. In Spark MLlib, the CrossValidator + Pipeline pattern handles this correctly automatically.