In a typical natural language processing (NLP) workflow that prepares raw text for modeling, which step is a standard early-pipeline operation?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Okay, let's dive in. You've got a pile of messy text—emails, tickets, reviews—and somehow that has to become something a model can chew on. Think of it like sorting mail at the post office: "running," "runs," and "ran" are the same letter with different stamps. Stemming or lemmatization peels those variants back toward a shared root so you don't treat them as totally unrelated words. Feature scaling? That's for numbers on a spreadsheet. Cross-validation? That's how you grade the model later. Dimensionality reduction? Cool trick, different job. Exam trap: don't grab every "preprocessing" buzzword you remember from tabular ML—NLP has its own early steps. Got it? Sweet. When you see "pipeline for text," look for tokenization-family moves like stem/lemma.
Full explanation below image
Full Explanation
A standard NLP pipeline turns unstructured language into model-ready inputs through linguistic and statistical preprocessing. After tokenization (and often lowercasing, punctuation handling, and stop-word decisions), stemming or lemmatization reduces inflected or derived word forms to a common base. Stemming applies rule-based truncation (for example, mapping "studies" and "studying" toward a stem), while lemmatization uses vocabulary and morphology to recover a dictionary lemma (for example, "better" → "good" in some setups). The goal is to shrink sparse vocabulary, improve feature sharing across related tokens, and stabilize bag-of-words, TF-IDF, or classic sequence features.
The correct option names this morphological normalization role. Feature scaling belongs primarily to numeric tabular workflows (min-max, z-score, and similar transforms) so gradient-based or distance-based learners are not dominated by large-magnitude columns; raw text does not start as that kind of continuous feature matrix. Cross-validation is an evaluation and model-selection protocol: it estimates out-of-sample performance by rotating train/validation folds, not by transforming word forms. Dimensionality reduction (PCA, SVD on term-document matrices, autoencoders) can follow vectorization when sparsity or multicollinearity is a problem, but it is an optional modeling step rather than a defining early NLP linguistic stage.
Underlying principle: early NLP stages normalize linguistic form before modeling; generic ML tools answer different questions. Best practice version-controls tokenization and stem/lemma choices with the model so train and serve match. Memory aid: pipeline stages answer how we clean language versus how we train or evaluate a learner. If the distractor could apply equally to house-price regression without any text, it is probably not the NLP-specific answer. On exams, associate classic NLP prep with tokenize, normalize via stem or lemma, then vectorize, and keep CV and generic numeric scaling in the general ML toolkit.