In Natural Language Processing (NLP), how is a word represented when using the one-hot encoding technique?
Select an answer to reveal the explanation.
![A table showing four words: "cat", "dog", "bird", "fish" with their corresponding one-hot encoded vectors: [1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], and [0, 0, 0, 1]. Highlight the single "1" in each vector to illustrate the "hot" element in a sparse vector.](/media/blockchain-council/caid/CAID-e5-q218-infographic.webp)
Short Explanation
Think of it like this: computers are great at math, but they don't understand words. If you have a vocabulary of four words—say, "cat," "dog," "bird," and "fish"—you need a way to turn those words into numbers. One-hot encoding does this by creating a list (a vector) where each word gets its own slot. The vector length is the size of your vocabulary. For the word "cat," you put a "1" in the first slot and "0" everywhere else: [1, 0, 0, 0]. For "dog," it's [0, 1, 0, 0]. It's called "one-hot" because only one single element is "hot" (set to 1) while the rest are cold (0). Because most of the vector is filled with zeros, it's a very sparse vector. Option C is the correct answer!
Full Explanation
One-hot encoding is a fundamental vectorization technique used to convert categorical variables (such as words in a vocabulary) into a numerical format that machine learning algorithms can process.
In a one-hot representation, a vocabulary of size V is established. Each unique word in the vocabulary is assigned a unique index. A word is then represented as a binary vector of length V. In this vector, the element at the word's assigned index is set to 1, while all other V-1 elements are set to 0. This creates a high-dimensional and highly sparse vector. While simple to implement, one-hot encoding has significant limitations in NLP: it does not capture semantic similarity between words (the dot product of any two distinct one-hot vectors is always 0), and the vector size grows linearly with the vocabulary, leading to high memory consumption.
Let's analyze why the other options are distractors: - Option A describes dense word embeddings (such as Word2Vec, GloVe, or FastText), which represent words in a low-dimensional space (e.g., 300 dimensions) where real-valued numbers capture semantic and syntactic relationships. - Option B describes term frequency or integer mapping, not one-hot encoding. - Option D is incorrect as it refers to character encoding rather than categorical word vectorization.
For the exam, remember that one-hot encoding produces sparse vectors where a single element is 1 and all others are 0.