A rum rickhouse intern wants to tune every knob on the same barrel-end photos that will later be called “the test.” Why keep training, validation, and a final untouched split separate?
Select an answer to reveal the explanation.
Short Explanation
Training is practice, validation is a coach’s clipboard, and the final test is the sealed exam booklet. If you tune on that sealed booklet, the last score is no longer honest. Separate jobs keep the final number clean.
Full Explanation
A rum rickhouse (or any municipal barrel archive) must keep training, validation, and a final untouched test split conceptually separate. Training data fit the weights. Validation data guide model selection, early stopping, and hyperparameter choices without being used for gradient updates. The final untouched test estimates performance after selection is frozen, so the reported number is not contaminated by repeated peeking. Making all three splits identical so every knob sees the same photos twice destroys independence. Using the test set for every gradient update turns the sealed exam into training data. Replacing the training set with the untouched split after the first epoch also contaminates the final estimate and starves learning. Exam caveat: civic wording about barrels and interns does not change the statistical roles of the three splits. Operational check: freeze a final test tranche before tuning; tune only on training plus validation; score the sealed test once at the end and record that the test was not used for selection. Document which barrel photos sit in which split and refuse to move sealed-test images back into training when leadership asks for a quick score bump. Rickhouse governance should require a written split manifest; moving sealed barrels into training voids the final performance claim. Retire contaminated test splits rather than arguing about a single peeked score.