Why might two identical API requests to Claude with temperature > 0 produce different responses?
Select an answer to reveal the explanation.
Short Explanation and Infographic
LLM outputs are probabilistic — at each token, Claude samples from a probability distribution rather than always picking the single best token. That's why the same prompt can produce different but equally valid responses.
Full explanation below image
Full Explanation
Language models generate text through stochastic (random) sampling. At each token position, the model computes probabilities for all possible next tokens and samples from that distribution. With temperature > 0, multiple tokens have non-zero probability of being selected. This means identical inputs produce slightly different (but statistically similar) outputs each time. At temperature=0, the model always selects the highest-probability token, producing near-deterministic outputs. This stochastic nature is a fundamental property of autoregressive language models, not a bug. Option A is wrong — training doesn't change between calls. Option C (hardware variation) could cause minor floating-point differences but isn't the primary cause. Option D is absurd.