A street-sign project tags asset names. Most tokens are ordinary words, so a model that marks everything other looks accurate. Which score should they trust?
Select an answer to reveal the explanation.
Short Explanation
Most tokens are ordinary words, so a model that tags everything “other” looks accurate and still misses every asset name. Trust entity-level F1, or an equivalent span score. Summary overlap and perplexity on the ordinary words do not measure those rare tags.
Full Explanation
Token-classification tags are usually rare against a sea of ordinary words. A model that predicts other on every token can post a high per-token accuracy and still miss every asset name. Entity-level F1, or an equivalent span score, cares about those rare tags. Summary overlap and perplexity do not measure named-entity quality.