PROC TEXTMINE is run with WEIGHT=IDFLOG in the PARSE statement. What weighting scheme does IDFLOG apply to terms in the term-document matrix?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — c is correct because WEIGHT=IDFLOG in PROC TEXTMINE applies log-based inverse document frequency weighting: weight = log(total_docs / docs_containing_term). Terms that appear in nearly all documents (common words like 'the', 'this') receive very low weight, while rare domain-specific terms receive high weight.
Full explanation below image
Full Explanation
C is correct because WEIGHT=IDFLOG in PROC TEXTMINE applies log-based inverse document frequency weighting: weight = log(total_docs / docs_containing_term). Terms that appear in nearly all documents (common words like 'the', 'this') receive very low weight, while rare domain-specific terms receive high weight. This is the classic IDF weighting used in text retrieval. A (raw counts) is WEIGHT=FREQ. B (TF normalized by length) is WEIGHT=ENTROPY or normalization variants. D is wrong; IDFLOG does not square term frequencies.