The 'top_p' parameter in the Claude API is used for nucleus sampling. What does setting top_p=0.9 mean?
Select an answer to reveal the explanation.
Short Explanation
When you set top_p=0.9, you tell Claude to build a bucket of the most likely next tokens until they collectively cover 90% of the probability, then pick from that bucket. That keeps the output coherent without forcing every token to be 90% likely on its own.
Full Explanation
Nucleus sampling (top_p) defines a probability threshold. The model considers the smallest set of tokens whose cumulative probability sums to at least top_p, then samples from that set. top_p=0.9 means Claude only considers tokens in the 'top 90% probability mass,' excluding very low-probability tokens. This can produce more coherent outputs than purely random sampling. top_p=1.0 means all tokens are considered. The context-window interpretation is wrong because top_p controls token selection during generation, not how much of the maximum context window is used. The individual-probability-threshold interpretation is wrong because top_p does not require each token to have 90% or higher probability by itself; it works with cumulative probability across a set of tokens. The factual-content-split interpretation is nonsensical because there is no mechanism that makes a response contain exactly 90% factual content and 10% speculation.