The 'top_p' parameter in the Claude API is used for nucleus sampling. What does setting top_p=0.9 mean?
Select an answer to reveal the explanation.
Short Explanation and Infographic
top_p is about building a bucket of the most likely next words until they collectively represent 90% of the probability — then picking randomly from that bucket.
Full explanation below image
Full Explanation
Nucleus sampling (top_p) defines a probability threshold. The model considers the smallest set of tokens whose cumulative probability sums to at least top_p, then samples from that set. top_p=0.9 means Claude only considers tokens in the 'top 90% probability mass,' excluding very low-probability tokens. This can produce more coherent outputs than purely random sampling. top_p=1.0 means all tokens are considered. Option A confuses top_p with context window usage. Option C confuses top_p with a minimum per-token probability threshold. Option D is nonsensical — there's no factual/speculation split mechanism.