A hospital adds rounds to a boosted tree model that predicts which outpatient visits will be missed. Training error keeps falling, but error on a held-out week gets worse after a point. What is happening as those later rounds are added?
Select an answer to reveal the explanation.
Short Explanation
The choir learned the song, and now it is memorizing the hiss on the tape. Your training error can keep falling while the next week gets worse, because later boosting rounds start fitting noise. More rounds are not automatically a steadier vote.
Full Explanation
Boosting adds rounds that chase remaining training error. Early rounds usually capture stable structure; later rounds can fit idiosyncrasies of the training week that do not repeat. When that happens, training error continues to fall and error on a held-out week rises, which is the pattern in this clinic's curve. Extra rounds are not a bagged average that must steady the held-out week. A worse held-out result after more rounds is the opposite of needing more averaged trees. Boosting does use the residual from earlier rounds, so a claim that each round ignores the residual and cannot memorize training quirks misstates the method. A zero step size would freeze the model, but training error is still falling, so later rounds are changing the fit. Exam caveat: the round where held-out error turns up depends on this dataset and this learning rate; there is no universal round count. Operational check: plot training error and held-out error against the round number and stop adding rounds where the held-out curve turns upward.