A regional power cooperative fits a demand model until the error on the weeks used to fit it is tiny, but a later week the model never saw shows a much larger error. Which figure should decide whether the model is ready to use?
Select an answer to reveal the explanation.
Short Explanation
The score that matters is the one on weeks the model has not already used to fit itself. A tiny error on the fitting rows can be memorization, and the trap is promoting that training number or the length of the feature list. You decide with the error on weeks the model was not fit on.
Full Explanation
Training error measures how well the demand model reproduces weeks that already influenced the fit. A tiny value can mean the model memorized those weeks rather than learned a relationship that will hold later. The week the model never saw is out of sample, and that error is what later operations will experience when the model is asked to score a new week. Readiness should be judged by that out-of-sample figure, not by the training figure. Chasing a training error of zero pushes the fit toward the known weeks and can widen the gap on a new week, so the distance from training error to zero is the wrong target. A longer feature list does not by itself generalize; extra inputs can make memorization easier and are not a substitute for an error computed on weeks left out of the fit. Exam caveat: a week that was used to tune settings is not a final untouched check, so the deciding figure should come from weeks that did not influence the fit or the tuning. Operational check: report the error on the unseen week beside the training error, and treat the model as ready only when the unseen-week error is acceptable.