Math & Classical ML Foundations

Certified Deep Learning Specialist (CDLS) · 50 questions

• A data scientist plots the distribution of a feature and confirms it is a perfect normal (Gaussian) distribution. Based on this alone, what can she conclude about the relationship between the mean, median, and mode of the feature?
• In the context of the mathematics used by deep learning frameworks, what does taking the tensor product of two tensors do?
• During backpropagation, a training framework computes the gradient of the loss function with respect to each weight in the network. What is the fundamental purpose of computing this gradient?
• A team trains a deep neural network that achieves 99% accuracy on the training set, but only 61% accuracy on new, unseen data from the same problem domain. Which phenomenon best explains this gap?
• An engineer is choosing between L1 and L2 regularization for a model and wants some of the less useful weights to be driven exactly to zero, effectively performing feature selection. Which statement correctly distinguishes the two approaches for this goal?
• A layer in a neural network multiplies a 3x2 matrix by a 2x4 matrix. What are the dimensions of the resulting matrix?
• In linear algebra as applied to machine learning (for example, in principal component analysis), what is the significance of a matrix's eigenvectors and eigenvalues?
• A dataset of household incomes shows most values clustered at the lower end, with a small number of very high-income outliers stretching the distribution's tail far out to the right. How would a data scientist describe the shape of this distribution?
• A square weight matrix needs to be inverted as part of a closed-form solution. Under which condition does that inverse actually exist?
• A team trains a deep neural network and finds it scores 99% accuracy on the training set but only 68% on the held-out test set. What best explains this gap?
• How does logistic regression fundamentally differ from linear regression in what it outputs?
• Which mathematical concept allows backpropagation to compute how the loss at the output layer should adjust the weights buried deep in earlier layers of a network?
• A data scientist is choosing between several candidate decision boundaries for a support vector machine classifier. What is the SVM's primary objective when selecting among them?
• What does the statistical measure of variance actually quantify about a dataset?
• In linear algebra, what does taking the transpose of a matrix produce?
• A regression model ends up with several coefficients that are extremely large in magnitude, making the model unstable and prone to overfitting. Which technique directly addresses this?
• In the underlying math of a neural network, what role do matrices primarily play?
• An engineer is reviewing why a similarity-scoring layer multiplies two embedding vectors element-wise and sums the results. Conceptually, what does this dot product operation give you?
• Suppose h(x) = f(g(x)), where g is an inner function and f is an outer function applied to g's output. Which expression correctly gives h'(x)?
• A researcher needs to know whether a 2x2 weight transformation matrix can be inverted before using it in a normalizing-flow layer. Which property of the matrix should the researcher compute to answer this?
• A colleague shows you a training script where the loss function is defined as Mean Squared Error (MSE) between the model's predictions and the target values. Based on this loss choice alone, what kind of task is the model most likely being trained for?
• While running 5-fold cross-validation, a data scientist notices that the model's accuracy swings wildly between folds — scoring 96% on one fold and only 71% on another, despite the folds being drawn from the same overall dataset. What does this high variance across folds most likely indicate?
• A neural network's final layer applies a softmax activation to produce output for a 3-class image classification problem. Which of the following best describes what the resulting output vector looks like?
• In a fully connected neural network layer, each neuron computes a weighted sum of its inputs before applying an activation function. What role does the bias term play in this computation?
• During training, an engineer observes that the derivative of the loss function with respect to a particular weight has reached exactly zero at the current point. Which statement correctly interprets what this means on its own?
• A data scientist has a dataset with five numeric features and wants a single mathematical object that summarizes how much each feature varies and how each pair of features moves together. Which object should they compute?
• A team needs to predict the next word in a sentence, where each prediction depends on the words that came before it in order. Which model class is classically built for this kind of sequential, order-dependent prediction task?
• While profiling a numeric feature before training, an analyst finds that its mean is noticeably higher than its median. What does this gap between mean and median most directly suggest about the feature's distribution?
• The idea of finding an optimal separating hyperplane between two classes, with the widest possible margin on either side, is the central concept behind which classical algorithm?
• In a convolutional neural network, the repeated stacking of convolution layers followed by pooling layers is, at its core, a practical implementation of which broader concept?
• A retailer has purchase history for thousands of customers but no predefined labels for them, and wants to automatically group similar customers together to design targeted marketing campaigns. Which category of machine learning task is this?
• Which classical model makes a prediction for a new data point by looking at the k training examples that are closest to it in feature space and basing its output on them (majority vote for classification, or averaging for regression)?
• A subscription company wants to predict whether each customer will churn (yes/no) in the next month using account features like tenure, usage, and support tickets. Which classical algorithm is a common, natural first choice for this kind of binary outcome prediction?
• After computing the covariance between two features, X and Y, an analyst finds it is a large positive number. What does this positive covariance indicate about the relationship between X and Y?
• During training, a framework computes the gradient vector of the loss function with respect to the model's weights. In which direction does this raw gradient vector actually point?
• Given two 2D vectors v = [v1, v2] and w = [w1, w2], which expression correctly computes their dot product v · w?
• A dataset has 200 numeric features, many of which are correlated, and a team wants to compress it into a much smaller set of features while retaining as much of the original variance as possible before feeding it into a downstream model. Which technique is the standard choice for this?
• A researcher plots the training loss curve for a new model and sees it spike up and down wildly from epoch to epoch instead of trending downward, even though the architecture and data pipeline are known to be correct. What is the most likely explanation?
• During backpropagation, why does the algorithm need to compute the derivative of each neuron's activation function at its current input value?
• In one sentence, what is gradient descent fundamentally trying to accomplish when training a model?
• In a logistic regression model, what is the role of the log-likelihood (log-loss) function used during training?
• Which core calculus concept provides the mathematical foundation that makes it possible to train a multi-layer neural network by propagating error signals backward through the network?
• A team wants an activation function that squashes any real-valued input into an output strictly between -1 and 1, with an output of 0 when the input is 0. Which activation function fits this description?
• A team trains a CNN on a dataset of images that are each labeled either "cat" or "dog," and the network learns to predict that label for new images. What type of machine learning task is this?
• You are building a network with a softmax output layer that must choose among ten mutually exclusive digit classes (0 through 9). Which loss function is the most appropriate choice for training this model?
• An engineer builds a network with a single sigmoid output neuron trained to answer one question for each input image: is this digit a "9" or not? What kind of task is this?
• In precise mathematical terms, how is the dot product of two vectors A and B, each with n components, formally defined?
• A data scientist is exploring a feature such as annual household income, where most values cluster at the lower end but a small number of values extend far into the millions. Under what condition is it generally most appropriate to apply a log scale (or log transform) to a feature like this?
• The Naive Bayes classifier, popular for tasks like spam filtering, derives its predictions from which foundational mathematical principle?
• In probability notation, what does the expression P(A|B) represent?