What is the relationship between temperature and top_k in language model sampling?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Temperature reshapes the probability distribution (squishing or spreading it). top_k cuts it off — only the K most likely tokens are even considered. Both reduce chaos, in different ways.
Full explanation below image
Full Explanation
Temperature and top_k both control sampling randomness but work differently. Temperature scales the logit distribution before applying softmax — low temperature makes the distribution sharper (model more confident in top tokens), high temperature flattens it (more uniform). top_k limits the candidate set to the K highest-probability tokens before sampling, regardless of their actual probability values. Anthropic's Claude API uses temperature and top_p (not top_k by default), but understanding both helps when working with different models. Option A is wrong — they are distinct mechanisms. Option C confuses with unrelated parameters (length=max_tokens). Option D is wrong — top_k is used in many model implementations.