Transit delay-explainer training needs reliable early stopping. How should validation be scheduled?
Select an answer to reveal the explanation.
Short Explanation
Early stopping only works if you check the map on a steady route. Eval every N steps on a frozen holiday-schedule slice so the transit team isn’t chasing a moving target.
Full Explanation
Transit delay-explainer training needs reliable early stopping, which means a fixed validation cadence on a frozen slice—not a moving target. Logging evaluation every N steps on a frozen holiday-schedule validation slice gives comparable metrics over time: holiday service patterns stay constant, so score changes reflect model updates rather than label churn or live noise. That stability is what makes patience counters and best-checkpoint selection trustworthy for municipal ops.
Evaluating only once after the final epoch with no intermediate checks fails by concept: you miss the best mid-training checkpoint and cannot stop early when generalization peaks. Reshuffling and rewriting the validation labels every training step fails because the metric then tracks label noise, not model progress—early stopping becomes meaningless. Using live rider complaints as the only validation signal mid-batch fails because live streams are non-stationary, biased, and unsafe to bind into step-level training decisions.
Exam caveat: the frozen holiday slice must still be representative enough of the failure modes you care about; supplement with a second fixed weekday slice if holiday-only eval would miss peak-commute errors. Operational check: pin a holiday-schedule validation file and hash, eval every N steps without rewriting labels, plot the delay-explainer metric, and stop with the best frozen-slice checkpoint when patience expires.