When planning a new Natural Language Processing project, a development team gathers millions of emails, articles, and chat transcripts to train a specialized model. What term describes this broad collection of text data?
Select an answer to reveal the explanation.
Short Explanation
Okay, let's dive in. Think of a corpus like your training ground or your library. If you want a chatbot to understand how human beings talk, you can't just feed it a single document or a couple of words—it needs to read a massive pile of real-world text to learn the patterns. That giant bucket of documents, books, or transcripts is what we call a 'corpus' in the NLP world. The plural is corpora, by the way. So, Option D is the only one that fits the bill here.
Full Explanation
In Natural Language Processing (NLP), a 'corpus' (plural: 'corpora') refers to a large and structured set of texts. These texts are gathered and prepared specifically for statistical analysis, training language models, or testing linguistic hypotheses. A corpus can consist of a single language, multiple languages (a parallel corpus), or domain-specific texts (such as legal briefs, medical journals, or customer emails).
The quality and size of the corpus directly determine how well an NLP model can learn linguistic features, syntactical structures, and semantic relationships. Modern Large Language Models (LLMs) are trained on massive web-scale corpora comprising billions or trillions of tokens to build a general understanding of language before being fine-tuned on smaller, task-specific corpora.
Let's look at why the other options are incorrect: - Option A is incorrect because an NLP library (such as spaCy, NLTK, or Hugging Face Transformers) is a software package containing tools, algorithms, and code, not the raw text data itself. - Option B is incorrect because a single document or sentence is too small to be considered a corpus, although it may be a member element of a corpus. - Option C is incorrect because a single word is referred to as a word, character, or token, which is the basic unit of text analysis, not a full text collection.
For your exam, remember that a corpus is the underlying dataset of text documents used to train or evaluate NLP algorithms.