PROC TEXTMINE is run with the MINCOUNT=5 option in the PARSE statement. What effect does MINCOUNT=5 have on the resulting term-document matrix?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — MINCOUNT=5 means a term has to show up in at least 5 documents to stay in the vocabulary. You're dropping rare terms from the term-document matrix, which cuts dimensionality.
Full Explanation
MINCOUNT=5 in PROC TEXTMINE's PARSE statement sets a document-frequency threshold for terms. A term must appear in at least 5 documents to be included in the vocabulary and the resulting term-document matrix; terms appearing in fewer than 5 documents are treated as too rare to be informative and are excluded, reducing dimensionality. The option about excluding documents with fewer than 5 terms after parsing is wrong because MINCOUNT= filters terms, not documents. The option about limiting the SVD step to the top 5 singular values is wrong because singular value count is controlled by the K= option in the SVD step. The option about retaining only the 5 most frequent terms per document is wrong because MINCOUNT= is a document-frequency threshold, not a per-document term count.