A developer is using a pre-trained Word2Vec model for a downstream text classification task. What is the fundamental property of the static word embeddings generated by this model?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: Word2Vec is built on a simple but powerful idea called the distributional hypothesis. Basically, it means you can know the meaning of a word by the company it keeps! If two words—like "cat" and "dog"—regularly show up surrounded by similar words like "pet", "food", and "fur", Word2Vec is going to pull those two vectors close together in its multi-dimensional space. Now, pay close attention to this common trap: Word2Vec is static. That means the word "bank" has the exact same vector whether you're talking about a river bank or a money bank. It does not change dynamically. And it definitely doesn't care about spelling similarity. That's why Option C is the absolute correct answer.
Full explanation below image
Full Explanation
The core feature of the Word2Vec model is its ability to map words into a dense vector space based on context. This is rooted in linguistics—specifically J.R. Firth's distributional hypothesis: "You shall know a word by the company it keeps." During training (via Skip-gram or CBOW), the model learns vectors by predicting a target word from its context, or predicting context words from a target word. Through this optimization, words that occur in similar linguistic contexts end up with high cosine similarity (small angles between their vector representations). This allows the model to capture semantic relationships (like synonyms or analogies like "king - man + woman = queen") and syntactic relationships (like verb tenses).
Let's look at why the other options are incorrect: - Option A is incorrect because Word2Vec treats each word as a single token and does not examine its spelling or character structure. This is a feature of subword-based models like FastText. - Option B describes contextualized word embeddings (such as those from BERT, RoBERTa, or GPT models). Word2Vec is a static model, meaning a word has a single, fixed vector representation regardless of the sentence context in which it appears at inference time. - Option C is correct as it accurately describes the semantic grouping based on context. - Option D is incorrect because word embeddings are dense vectors, where every dimension is filled with a real-valued number, unlike the sparse vectors of Bag-of-Words or One-Hot Encoding.