Ask a chatbot the same question twice and you can get two different answers, because it is built that way. A language model does not look up a reply. It writes one a word at a time, and each word is picked by weighted chance from the words it considers likely. Two runs can take different turns early and end up in different places.
That much is well known. Two things are newer. The setting that used to turn the randomness down is being removed from the newest models by all three of the big vendors. And research has found that the way an answer changes between asks can tell you something useful about whether to trust it.
Built from Anthropic’s, OpenAI’s and Google’s developer documentation, checked on 25 October 2026, and two published studies. Nothing here was tested hands-on. The documentation describes the models underneath the apps; the consumer apps do not publish their settings.
The short version
- Variation is the default, not a fault. Each word is drawn from a range of likely options, so the same question can produce different wording, and sometimes different facts.
- The dial is going. Anthropic, OpenAI and Google have each removed or disabled the “temperature” setting on their newest models. Anthropic’s documentation says plainly that nothing replaces it.
- Even the old “temperature 0” never guaranteed identical answers. In one published experiment, 1,000 runs at temperature 0 produced 80 different completions.
- Disagreement is a warning sign. If the facts change between asks, treat every version as unverified. If they stay the same, that is not proof either.
Where the variation comes from
A model’s output at each step is a list of possible next words, each with a probability. Getting text out involves “sampling”, which, as one research team at Thinking Machines Lab puts it, “converts the language model’s output into a probability distribution and probabilistically selects a token.” Most of the time the likeliest word wins. Sometimes a less likely one does, and everything after it is written to follow on from that choice.
Temperature is the setting that reshapes those odds. Turn it down and the likeliest words win more often; at zero, the model always takes the single likeliest word. Turn it up and the long shots come in more often. For years, developers who wanted steadier output lowered it.
The dial is going away
All three vendors now tell developers to stop sending it to their newest models. Checked on 25 October 2026:
| Vendor | What the documentation says |
|---|---|
| Anthropic | From API version 2026-10-21, sending temperature, top_p or top_k returns an error, “even at the parameter’s default value”. Claude 4.7 and later models already accepted only the defaults, including a temperature of 1.0 |
| OpenAI | GPT-6 Astra “does not support custom temperature or top_p values” (changelog). For other GPT-6 models, remove temperature whenever the reasoning effort is set to anything but “none” (OpenAI) |
| “Gemini 4 and Gemini 3.6+ models ignore custom values for temperature, top_p, and top_k” and return deprecation warnings (Google) |
Anthropic’s migration note is the most direct about the consequence. On older models that still accepted it, “a request that lowered temperature for more consistent output loses that effect. No other setting replaces it.”
None of the three pages says why. OpenAI ties the rule to its reasoning setting, and Google describes its newest model, Gemini 4 Argon, as its “flagship reasoning model”, but that is as far as the documentation goes, so we will not guess at the rest.
Temperature 0 never meant identical anyway
Even when the dial existed, turning it to zero did not guarantee the same answer twice. Anthropic’s migration note describes lowering temperature as a route to “more consistent” output, not identical output.
The clearest public demonstration comes from Horace He and colleagues at Thinking Machines Lab (September 2025). They asked an open-weights model, Qwen3-235B, to “Tell me about Richard Feynman” 1,000 times at temperature 0. They got 80 different completions. All of them were identical for the first 102 tokens and split at the 103rd: 992 went on to say Feynman was born in “Queens, New York”, and 8 said “New York City”. Both are true, which is typical. Most variation is in the wording.
Their explanation is the surprising part. The servers that run these models process many people’s requests together in batches, and the arithmetic inside can come out very slightly differently depending on how big the batch is. In their words, “our request’s output does depend on the parallel user requests”, not because anything leaks between users, but because the calculation is not “batch invariant”. So the answer you get can depend on how busy the service is at that moment.
Two caveats. That experiment used an open model on the team’s own setup, not a commercial chatbot, and the post goes on to show that with rewritten calculations all 1,000 runs came out identical, at some cost in speed. What it establishes is that “same input, same output” is not something to assume.
In a chat app, more than the dice can change
The documentation above is about developers’ API requests with nothing else attached. In ChatGPT, Claude or Gemini, other things can differ between two asks: what is already in the conversation (a long thread shapes the reply, as our context windows explainer covers), anything the app remembers about you, and, if it searched the web, which pages it happened to find. A fresh chat removes the first of those, but not necessarily the others.
What the difference tells you
This is where variation stops being a nuisance and becomes useful.
In a study published in Nature in June 2024, Sebastian Farquhar and colleagues at Oxford looked at a kind of error they call confabulation: answers that are “arbitrary and incorrect”. Their method samples several answers to the same question, groups the ones that mean the same thing however they are worded, and measures how much the meanings disagree. The more they disagree, the more likely the answer is a confabulation. One example from the paper: asked “What is the target of Sotorasib?”, a model sometimes answered KRASG12 ‘C’, which is correct, and sometimes ‘D’, which is not, “despite identical instructions”.
The authors are just as clear about the limit. The method “does not guarantee factuality because it does not help when LLM outputs are systematically bad”. A model that learned a common misconception will repeat it every time, calmly and consistently.
That squares two pieces of advice that can sound contradictory. Our fact-checking routine says a regenerated answer is “a new sample, not a second opinion”, and our piece on why AI makes things up suggests asking again in a fresh chat and comparing. Both hold, because the comparison only works in one direction: disagreement is evidence of a problem, and agreement is not evidence of an answer.
A three-ask check you can run in any chat app
This is our adaptation of that idea for people without a research lab. Three asks is our suggestion, not the paper’s: enough to see a split, cheap enough that you will actually do it.
- Ask the same question three times, word for word, ideally in fresh chats.
- Compare the substance, not the wording. Names, numbers, dates, yes-or-no conclusions, the cited source. Different phrasing of the same fact counts as agreement.
- Read the result:
| What you see | What it suggests | What to do |
|---|---|---|
| Same facts, different wording | Normal variation | Check it as you would any claim that matters |
| A number, name or date changes | Likely confabulation | Treat every version as unverified; go to a source |
| Different conclusions (yes in one, no in another) | The model has no stable answer | Do not use any of them on their own |
| Different sources cited | At least one citation is probably wrong | Open each one and see if it says what is claimed |
Two adjustments. For creative work, the variation is the point: ask for three versions in one request and pick. And for a fact that matters, the three-ask check is a filter, not a verdict. It catches the answers that wobble, not the ones that are consistently wrong.
When you want the same answer every time
With the dial gone, consistency now comes from the request rather than a setting. These are our suggestions, not vendor guidance:
- Fix the shape. Give the format and one example of a good answer. The fewer reasonable ways there are to answer, the less room there is to wander. Our practical guide to prompting covers the parts that matter.
- Give it the material. A question answered from a document you pasted has fewer plausible answers than one answered from memory.
- Keep the good answer. Once you have checked an answer, save it. Asking again to “confirm” it just draws another sample.
- For numbers, make it calculate. A total read off a pasted table is one more sample; a total worked out by code or a formula can be checked. Our guide to using AI on a spreadsheet and still trusting the numbers shows how.
The thing worth remembering
An AI answer is one draw from a range of possible answers, and the newest models no longer let anyone narrow that range with a setting. Use that: when it matters, draw three times. If the facts move, none of them is an answer yet.
Header photo: PierreSelim / Wikimedia Commons, CC BY 3.0.
How this was made. AI Decode Wire uses AI tools for research and drafting. Factual claims link to their sources, and errors are corrected in the open. AI-assisted content policy · Report an error