A freight carrier cleans free-text exception notes before grouping themes. The notes say a truck ran late, was running late, or runs late, and each form should become the dictionary verb run rather than a chopped fragment. Which text step does that?
Select an answer to reveal the explanation.
Short Explanation
Chopping the end off a word is like trimming a name with scissors and hoping the nickname is still in the dictionary. Lemmatization looks up the real verb, so ran, running, and runs become run for you. The trap is stemming, which only slices the string and can leave a stub that is not a word.
Full Explanation
Lemmatization reduces a word to its dictionary form, called a lemma, using vocabulary and grammar rather than a string chop. Irregular and inflected forms such as ran, running, and runs share the verb lemma run, so the freight notes can group them as one word instead of three strings. That is the step the carrier needs. Stemming only applies chopping rules to the character string. It may strip a suffix such as ing, but it does not look up an irregular past form, and it can leave a fragment that is not a real dictionary word. Stop-word removal drops function words that carry little topic signal and does not rewrite content verbs at all. Tokenization splits the note into pieces and preserves each surface form, so ran and running stay distinct and unreduced. The carrier needs one real dictionary verb for every inflected form, including the irregular past, and suffix chopping does not promise that result. Exam caveat: a lemma depends on the word's role in the sentence, so the same spelling can map differently when it is a noun rather than a verb. Operational check: print a sample of notes before and after the step and confirm ran, running, and runs all appear as run, not as a chopped stub.