A practitioner computes Cohen's Kappa = 0.72 for a 3-class text classifier. What does this value indicate according to the Landis and Koch scale?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — a Kappa of 0.72 means you're seeing substantial agreement, or 72% more agreement than expected by chance. Per Landis and Koch, 0.61-0.80 is the substantial range.
Full Explanation
Cohen's Kappa measures agreement beyond chance. A Kappa of 0.72 means 72% more agreement than expected by chance, and it falls in the substantial agreement range (0.61-0.80) per Landis and Koch. The claim that the model correctly classifies 72% of documents is wrong because Kappa is not accuracy. The claim that this is moderate agreement is wrong because 0.61-0.80 is substantial, not moderate. The claim that the macro-averaged F1 score is 0.72 is wrong because Kappa and macro F1 are different metrics.