Why might two identical API requests to Claude with temperature > 0 produce different responses?
Select an answer to reveal the explanation.
Short Explanation
You get different answers because token sampling introduces controlled randomness. At each step, Claude samples from a probability distribution instead of always picking the top token, so the same prompt can produce different but valid outputs.
Full Explanation
Language models generate text through stochastic sampling. At each token position, the model computes probabilities for all possible next tokens and samples from that distribution. With temperature greater than zero, multiple tokens have non-zero probability of being selected. This means identical inputs produce slightly different but statistically similar outputs each time. At temperature zero, the model always selects the highest-probability token, producing near-deterministic outputs. This stochastic nature is a fundamental property of autoregressive language models, not a bug. The idea that Claude randomly modifies its training between calls is wrong because training doesn't change between calls. The idea that Anthropic's servers process requests on different hardware each time could cause minor floating-point differences but isn't the primary cause. The idea that Claude checks the current time and generates different responses based on the day of week is absurd.