A junior engineer splits a dataset into train and test sets simply by taking the first 80% of rows as train and the last 20% as test, without shuffling first. What common pitfall does this risk?
Select an answer to reveal the explanation.
Short Explanation
Real-world datasets are sneaky about hidden order — maybe rows are sorted by date, by class label, by customer ID, or by whatever order they were collected in. If you just slice the first 80% and last 20% without shuffling first, you can easily end up with a test set that's missing entire classes, is all from a different time period, or is systematically different from training in some way you didn't intend. That's data leakage or bias risk from non-random splitting, which is exactly the correct answer. It has nothing to do with training speed — order of rows barely affects how fast training runs. It doesn't guarantee any particular size split either — an 80/20 split just means 80% and 20%, regardless of whether you shuffled. And Keras has no idea how you split your data beforehand; it happily trains on whatever train/test arrays you hand it, shuffled or not.
Full Explanation
When a dataset is split into training and test sets without first shuffling the rows, the split can inadvertently introduce data leakage or bias if the original ordering of the data is not random. Many real-world datasets carry implicit structure — sorted by date, grouped by class label, ordered by customer or session ID, or batched by collection source — and a simple positional slice (first 80% as train, last 20% as test) risks systematically separating the data along that hidden structure rather than sampling representatively across it. If rows are sorted chronologically, a non-shuffled split could put all older data in training and all newer data in test, which is usually an unintended and misleading bias for a general classification task, since the test set would no longer represent the same distribution as the training data. Similarly, if rows are grouped by class, a non-shuffled split could leave the test set with very few or zero examples of certain classes, harming both evaluation and any downstream cross-validation. Best practice is to shuffle the dataset with a fixed random seed before splitting, or to use a stratified split when class imbalance is a concern, ensuring both sets are representative random samples of the overall distribution.
Training speed is essentially unaffected by whether the data was shuffled before splitting; computational cost depends on batch size, architecture, and hardware, not on row order prior to the split.
The relative sizes of the train and test sets are determined by the split ratio the engineer specifies (here, 80/20), not by whether shuffling occurred. Shuffling changes which rows land in which set, not how many rows end up in each set.
Keras, or any framework, has no built-in check related to how the data was split before being handed to it; it simply trains on whatever train and test arrays are provided, regardless of methodology. Any quality problems from an unrepresentative split are silent — no error is thrown, the split just quietly produces misleading evaluation results, which is exactly why this pitfall is dangerous.