IDF in a Spark text pipeline down-weights terms that:
Select an answer to reveal the explanation.
Short Explanation
IDF reduces weight of ubiquitous tokens so rare discriminative terms shine.
Full Explanation
TF-IDF = term frequency adjusted by inverse document frequency.