In natural language processing, what does the term word embedding refer to?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Check this out: computers don't “read” words—they need numbers. Word embeddings turn each word (or subword) into a list of floats so models can do math on language. The cool thing is similar meanings cluster—king and queen end up nearer than king and toaster. Think of it like a city map where neighborhoods are topics. Trap alert: tokenization and stop-word removal are prep steps; they aren't embeddings. And a pretty parse tree isn't a vector space. Boss wants better search or classification? Embeddings are the bridge from text to neural nets. Land it: embedding = meaning-aware numeric vectors, not just splitting or stripping text.
Full explanation below image
Full Explanation
Word embeddings are learned or constructed continuous vector representations of lexical items. Classic algorithms such as Word2Vec, GloVe, and FastText produce fixed embeddings from co-occurrence statistics; contextual models (ELMo, BERT-family) produce context-dependent vectors for each occurrence. In all cases, the operational idea is the same: replace discrete symbols with points in a real-valued space where geometric relationships reflect linguistic similarity and, often, linear analogies such as king − man + woman ≈ queen in well-trained spaces. Dimensionality is typically tens to hundreds of floats per token—far denser and more useful than enormous sparse one-hot vocabularies.
This representation is foundational for neural NLP. Dense embeddings are more informative and parameter-efficient than sparse one-hot encodings, enable transfer of distributional knowledge into downstream classifiers, and support nearest-neighbor retrieval for semantic search, recommendation of related terms, and initialization of larger sequence models. Training objectives typically encourage words that appear in similar contexts to have similar vectors (the distributional hypothesis). Downstream tasks—sentiment analysis, named-entity recognition, machine translation—almost always begin by mapping tokens into such vectors before deeper layers operate.
The incorrect options describe adjacent but distinct pipeline stages. Tokenization (or subword segmentation) decides the atomic units; it does not assign semantic geometry. Removing stop words reduces noise in bag-of-words systems but is optional and orthogonal to embedding learning. Visualizing syntactic structure helps linguists and some parsers; it is not the definition of embedding and does not place words in a continuous similarity space for gradient-based learning.
Best-practice notes: choose subword embeddings for morphology and rare words; evaluate whether static or contextual embeddings fit the task; watch for social bias encoded in co-occurrence data; retrain or fine-tune embeddings when domain language shifts; and never treat “embedding” as a synonym for any text-cleaning trick. For exams, if the stem emphasizes numeric vectors with similar meanings near each other, the answer is word embedding—not tokenization, stop-word lists, or parse diagrams.