In natural language processing, what is the primary architectural advantage of using FastText over standard Word2Vec for generating word representations?
Select an answer to reveal the explanation.
Short Explanation
Imagine you're using a classic Word2Vec model, and it encounters a word it has never seen during training—say, a typo like "netwoork". Because Word2Vec treats every word as a single, indivisible unit, it has no idea what to do and just marks it as an "unknown" token. That's a massive bummer for your model's accuracy. FastText, which was developed by the clever folks at Facebook, solves this by breaking words down into smaller character chunks called n-grams (like "net", "etw", "two"). So, even if the model sees a misspelled word or a brand-new term, it can piece together a highly accurate vector representation based on the subwords it does know. That makes Option B the clear winner.
Full Explanation
FastText is an extension of the Word2Vec model designed to address its core limitation: the inability to handle out-of-vocabulary (OOV) words. In standard Word2Vec (both Continuous Bag-of-Words and Skip-gram architectures), each word in the vocabulary is assigned a distinct, static vector. If a word was not present in the training corpus, the model cannot generate a vector for it during inference. FastText overcomes this by representing each word as a sum of its character n-grams. For example, if we set $n=3$, the word "fast" is represented by the 3-grams: <fa, fas, ast, st> (where < and > denote the word boundaries). The word vector for "fast" is the sum of the vector representations of these n-grams. This subword approach provides several key benefits: 1. Handling OOV Words: Even if a word is missing from the training vocabulary, FastText can construct a vector by combining the vectors of its constituent n-grams. 2. Robustness to Typos: Minor misspellings (e.g., "computerr" instead of "computer") still share most of their n-grams with the correct word, resulting in a similar vector. 3. Morphologically Rich Languages: Languages like German or Finnish, which rely on compound words, are much easier to model because subwords capture prefixes and suffixes.
Let's evaluate the distractors: - Option A describes contextual embedding models (such as BERT or GPT), whereas FastText produces static embeddings. - Option C describes grammatical parsing and part-of-speech tagging, which are syntactic analysis tasks not performed by FastText. - Option D is incorrect because FastText is a lightweight CPU-friendly algorithm, and it outputs dense, not sparse, vectors.