A data scientist uses the gradient boosting gbtreetrain CAS action in SAS Viya and wants to tune the learning rate (shrinkage) parameter. What is the effect of decreasing the learning rate from 0.1 to 0.01 while keeping all other parameters constant?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — decreasing the learning rate from 0.1 to 0.01 makes each tree contribute less, so you'll need more trees to reach the same training error. That slower learning often helps the model generalize better by reducing overfitting.
Full Explanation
The correct effect is that the model will require more trees to achieve the same training error but may generalize better. The learning rate, or shrinkage, scales each tree's contribution. A lower learning rate means each tree contributes less, so more boosting iterations are needed to reach the same training loss. This slower learning often improves generalization by reducing overfitting to noise. The claim that the model will converge faster and require fewer trees is wrong because slower learning requires more trees. The claim that the model will automatically increase MAXDEPTH to compensate is wrong because learning rate and MAXDEPTH are independent hyperparameters. The claim that feature importance scores will be scaled proportionally to the learning rate is wrong because feature importance is based on split gain contributions, not scaled by the learning rate.