A county floodplain desk tunes on last spring’s permit notes, peeks at a reserved autumn set after every epoch to pick a winner, and then reports that autumn score as the official result. Why is that score no longer a valid unseen test?
Select an answer to reveal the explanation.
Short Explanation
If you peek at the reserved autumn set every epoch to pick a winner, that set is no longer unseen. Use validation for mid-run decisions and score the sealed test once after the recipe is frozen. GPU utilization is not the official result.
Full Explanation
Validation is the set used to make stop and select decisions while a recipe is still moving. A reserved test envelope stays sealed until those decisions are frozen; scoring it once is then a genuine unseen readout. Peeking after every epoch to pick a winner turns that envelope into another validation set, so the published autumn number is no longer an unseen test. Never-opening the set, NCCL winner-picking, and GPU utilization are the wrong hygiene rules.