Which classical model makes a prediction for a new data point by looking at the k training examples that are closest to it in feature space and basing its output on them (majority vote for classification, or averaging for regression)?
Select an answer to reveal the explanation.
Short Explanation and Infographic
This one's practically defined by its name: K-Nearest Neighbors literally finds the k closest points to whatever you're trying to predict and lets them decide the answer, whether that's a majority vote among neighbors for a classification label or an average of their values for a regression target. No training phase builds a formula in advance — KNN just stores the data and does the comparison at prediction time, which is why it's called a "lazy" learner. That's why KNN is the answer. Naive Bayes instead computes class probabilities using Bayes' theorem and an independence assumption between features, no neighbor-distance calculation involved. Logistic regression fits a set of weights to model the probability of a class through a sigmoid function, and linear regression fits weights to predict a continuous value directly — both learn a fixed equation up front rather than checking nearby points at prediction time.
Full explanation below image
Full Explanation
K-Nearest Neighbors (KNN) is a non-parametric, instance-based learning algorithm. Rather than fitting a fixed set of parameters during a training phase, KNN stores the entire training dataset and, at prediction time, computes the distance (commonly Euclidean) between the new query point and every stored training point. It then identifies the k closest points and bases the prediction on them: for classification, it typically takes a majority vote of the neighbors' class labels; for regression, it typically averages the neighbors' target values. Because most of the computational work happens at prediction time rather than during training, KNN is often described as a "lazy learner," in contrast to "eager learners" that build an explicit model upfront. The choice of k and the distance metric significantly affects performance, and KNN can be sensitive to irrelevant or unscaled features, which is why feature scaling is typically applied before using it.
Naive Bayes is incorrect because it is a probabilistic classifier that applies Bayes' theorem with a simplifying assumption of conditional independence among features. It computes and compares posterior probabilities for each class rather than measuring distances to individual neighboring training points.
Logistic regression is incorrect because it is a parametric model that learns a fixed set of weights during training, applying them through a linear combination followed by a sigmoid function to estimate the probability of a class. Once trained, prediction is a direct computation using those learned weights, with no reference back to the training examples or any distance-based neighbor lookup.
Linear regression is incorrect for the same structural reason: it learns a fixed set of coefficients during training that define a line (or hyperplane) mapping inputs to a continuous output. Prediction is a direct plug-in-the-numbers calculation using those learned coefficients, not a search for nearby training instances.
Memory aid: "k" is the tell — K-Nearest Neighbors is the only classical model on this list whose core prediction step literally counts or averages over the k closest stored examples.