Training, Optimization & Regularization

Certified Deep Learning Specialist (CDLS) · 50 questions

• What does the mean absolute error (MAE) metric measure for a regression model?
• A team trains a model on a dataset with tens of millions of examples using standard batch gradient descent. What is the main drawback they will run into?
• A model's training loss barely moves epoch after epoch and stays stubbornly high. Weight updates are being computed correctly and gradients are non-zero. What is the most likely cause?
• What is the main advantage of mini-batch gradient descent over full batch gradient descent when training on a large dataset?
• What is the main downside of setting the learning rate far too low when training a neural network?
• What is learning-rate decay in the context of training a deep learning model?
• What does the mean squared error (MSE) loss function do?
• What is the main purpose of a loss function in training a deep learning model?
• What is a key defining feature of the RMSprop optimizer?
• A model's loss oscillates wildly and never settles down, even growing over time. Gradients are correct. What hyperparameter is most likely set too high?
• What does L2 regularization add to a model's loss function?
• How does stochastic gradient descent (SGD), updating on one example at a time, typically compare to full batch gradient descent?
• A bank is building a fraud detection model. Missing an actual fraudulent transaction (a false negative) is far more costly to the bank than flagging a legitimate transaction for review (a false positive). Which metric should the team prioritize when evaluating the model?
• Which optimizer gives each parameter its own learning rate but has a well-known drawback of that per-parameter rate shrinking too aggressively over long training runs?
• What is cross-validation?
• In classification evaluation, what is a false negative (type II error)?
• A classifier is trained on a dataset where 98% of examples belong to the negative class. Which pair of metrics gives a more informative picture of performance than accuracy alone?
• During training, the training loss keeps falling steadily, but the validation loss starts rising after a certain epoch. What does this pattern indicate?
• What is the benefit of using a mini-batch size greater than one, compared to updating weights after every single training example?
• What is the main goal of hyperparameter tuning in a deep learning workflow?
• What is a practical benefit of using the RMSprop optimizer over plain SGD with a single fixed learning rate?
• What is the main strength of the F1 score as an evaluation metric for a classifier trained on imbalanced classes?
• A classifier achieves a recall of 1.0 on the positive class. What does this tell you?
• When is it most appropriate to apply early stopping during training?
• What is an advantage of batch gradient descent over stochastic gradient descent (SGD)?
• A newcomer is picking an optimizer for a fresh image classification project and wants a solid default that adapts its step size per parameter using running averages of both the gradients and their squares. Which optimizer are they describing?
• In gradient-based training, what is the primary role of the learning rate hyperparameter?
• A team wants a regularization method that can push some of their model's weights all the way to exactly zero, effectively removing those features from the model. Which comparison correctly describes the difference between L1 and L2 regularization for this goal?
• A team builds a classifier for a rare disease where only 1% of patients in the dataset actually have the condition. The model predicts 'no disease' for every single patient and reports a very high score on one metric. Which metric is dangerously misleading in this scenario if used as the only measure of performance?
• In classification evaluation, what does the accuracy metric represent?
• A bank builds a model to flag transactions as fraudulent, and every falsely flagged legitimate transaction locks out a real customer and triggers an expensive manual review. Which metric should the team prioritize to keep those false alarms as low as possible?
• When inspecting a multi-class confusion matrix, what does it indicate if the values along the main diagonal are large relative to the off-diagonal values?
• A team is training a neural network to forecast next week's electricity demand as a continuous numeric value from historical usage data. Which loss function is best suited to this time-series forecasting task?
• In a binary classification confusion matrix, what does a true negative represent?
• A hospital deploys a model to screen scans for a serious tumor, and missing an actual tumor case (letting it go undetected) is far more dangerous than sending a healthy patient for an unnecessary follow-up. Which metric should this team prioritize?
• Which regularization technique works by randomly setting a fraction of neuron outputs to zero during each training pass, forcing the network to avoid relying too heavily on any single neuron?
• What is the effect of applying L1 (lasso) regularization to a model's weights during training?
• What is the main purpose of building a confusion matrix after evaluating a classifier on a test set?
• In a regression model's evaluation report, what does the R-squared value actually tell you?
• A fraud-detection model shows high precision but low recall on the test set. What does this combination actually indicate about its behavior?
• How does the Adam optimizer relate to RMSprop in terms of what it borrows from each technique?
• What is the primary objective the Adam optimizer is trying to achieve during training?
• A spam filter classifies a legitimate email as spam. In classification terminology, what is this specific kind of mistake called?
• An object detection model trains on images where the background class vastly outnumbers rare foreground objects. Which loss function was specifically designed to address this kind of imbalance?
• Why would a data scientist report the F1 score instead of just reporting precision and recall separately?
• During model development, why do teams hold out a separate validation set instead of just training and testing on the same data split?
• A researcher training a deep network for 200 epochs notices validation loss stopped improving around epoch 60 and slowly worsened afterward. What technique would have helped avoid wasting the remaining epochs and overfitting the model?
• In the context of neural network training, what precisely is one epoch?
• A team is training a very deep neural network with dozens of layers and wants an optimizer that handles this scale well. Why is Adam typically recommended for this kind of architecture?
• A team wants a regression metric that punishes large prediction errors much more severely than small ones. Which metric fits that requirement?