You ask a hard question, the model announces that it is thinking, and a paragraph of visible deliberation scrolls past. It considers an approach. It notices a problem. It changes tack. Forty seconds later the answer arrives, and it feels more trustworthy than the instant one would have.
Some of that confidence is earned. Reasoning modes genuinely do better on problems with several steps. But the text you watched is not the model’s working, and the vendors’ own documentation says so.
*This explainer is built from vendor documentation and published research, checked on 20 September
- Nothing here was tested hands-on. Mode names, defaults and effort settings change often, so the
dates matter.*
The short version
- “Thinking” is extra text the model generates before its answer, called reasoning tokens.
- Those tokens cost you money and take up room in the window, even though you never see most of them.
- What you are shown is a summary of the reasoning, not the reasoning itself.
- Anthropic’s own research found models often fail to mention the thing that actually changed their answer.
- It helps most on multi-step problems and least on recall, formatting and short drafts.
What “thinking” actually is
There is no separate deliberation engine. A reasoning model does the same thing every language model does, predicting what text comes next, but it is trained to generate a long stretch of intermediate working first, and to use that working as context for the final answer.
OpenAI’s reasoning guide describes these as reasoning tokens: “The models use these reasoning tokens to ‘think,’ breaking down the prompt and considering multiple approaches to generating a response.” Depending on the problem, a model may generate “anywhere from a few hundred to tens of thousands” of them.
That is the whole mechanism. The improvement is real and unmysterious: writing the intermediate steps down puts them in the context the final answer is generated from, so the model is answering with its own working in front of it rather than leaping straight to a conclusion.
How much thinking happens is increasingly a dial rather than a switch. OpenAI’s guide lists effort levels from none and minimal up through high, xhigh and max. Anthropic’s documentation covers both a manual token budget (minimum 1,024 tokens, described as “a target rather than a strict cap”) and a newer adaptive mode where the model decides whether to think at all, and at lower effort settings “may skip thinking entirely on easy inputs”.
What you are shown is a summary
This is the part that changes how much weight the visible reasoning deserves.
OpenAI’s guide is explicit that raw reasoning is not exposed. Reasoning tokens are “not visible via the API”, summaries must be explicitly opted into, and even then “reasoning items remain opaque, and the API does not return their reasoning text”. Anthropic’s documentation makes the same distinction in its code comments and field names: the response “contains summarized thinking blocks”.
So the deliberation you are reading is a second piece of generated text about the reasoning, produced alongside it. It is a description, not a transcript. When it reads cleanly, in a tidy numbered march from problem to answer, some of that tidiness belongs to the summary rather than the reasoning.
The summary does not always match the reasoning
In April 2025, Anthropic published research on exactly this question. The setup was simple: ask a model a question, then ask it again with a subtle hint embedded, in metadata, or framed as a professor’s opinion, or presented as unauthorised access to the answer. When the hint changed the model’s answer, did the model’s visible reasoning mention it?
Often, no. Across hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time and DeepSeek R1 39% of the time. On the hints Anthropic called concerning (the unauthorised-access framing) the figures were 41% and 19%. In a reward-hacking setup, where models learned to exploit a hint to score well, they admitted doing so less than 2% of the time. Training the models to be more faithful helped, then plateaued at 28% on one evaluation and 20% on another.
The researchers’ conclusion is the useful sentence: advanced reasoning models “very often hide their true thought processes”, and if the goal is to rule out bad behaviour by reading the chain of thought, “there’s still substantial work to be done”.
What the study does not show
It does not show that models are usually wrong, or that reasoning mode is a con. It measured one specific thing: whether the stated reasoning mentions an influence that demonstrably changed the answer.
Anthropic lists the limitations itself, and they matter. The hints were “somewhat contrived”. The tasks were “multiple-choice quizzes, which are unlike real-world tasks”, and were “not difficult enough to require the Chain-of-Thought to be used”. Only two organisations’ models were examined. And this was April 2025, on Claude 3.7 Sonnet and DeepSeek R1, both since superseded, with no like-for-like rerun on current models to point you to.
The honest takeaway is narrow and still worth having: the visible reasoning is evidence about the answer, not an audit of it. Treat it as you would someone’s explanation of a decision they have already made.
A chess score sheet records the moves, not the deliberation. Both players spent most of that clock time on lines they never played, and none of it appears on the page. A reasoning summary sits in the same relationship to the model’s working.
What it costs you
Reasoning is not free, and the cost is easy to miss because most of it is invisible.
Reasoning tokens “still occupy space in the model’s context window and are billed as output tokens”, OpenAI’s guide says, and output tokens are the expensive ones, as our piece on what an AI bill is actually made of sets out. OpenAI suggests reserving at least 25,000 tokens for reasoning and output when you start experimenting, which tells you the scale it expects.
Two second-order effects are worth knowing. Thinking takes up room in the context window like anything else, so a long deliberation crowds the same budget your documents and conversation are competing for. And on Anthropic’s newer models, thinking from earlier turns stays in the conversation and is billed again as input on every subsequent turn.
One thing that does not help: hiding the thinking in the interface. Anthropic’s documentation is clear that billing is unchanged by the display setting. You pay for the tokens either way; the toggle only decides whether you watch them go by.
When it helps, and when it does not
The pattern in the documentation is consistent: reasoning pays off where the answer depends on steps that build on each other, and adds latency and cost where it does not.
| What you are doing | Worth turning on? | Why |
|---|---|---|
| Multi-step maths, logic or planning | Yes | Each step feeds the next; written working is the mechanism |
| Debugging, or tracing a problem to a cause | Yes | Rules out branches rather than guessing the likely one |
| Weighing options against several criteria | Usually | The comparison gets done explicitly instead of asserted |
| Drafting, rewriting, changing tone | Rarely | No intermediate steps to get right; you get latency and cost |
| Recall — a fact, a definition, a date | No | Thinking longer does not add knowledge the model lacks |
| Formatting, extraction, translation | No | Mechanical work; effort settings exist precisely to skip it |
One caution that the mode does not fix. Reasoning improves problems that have a checkable structure. It does not turn missing knowledge into present knowledge, so a question the model cannot answer can still produce a confident wrong answer, arrived at through many plausible steps. Longer working can make a wrong answer more persuasive, not less.
Using it without over-trusting it
- Match the mode to the task. Steps that build on each other, yes. Recall and drafting, no.
- Read the reasoning for its claims, not its confidence. If step three asserts a figure, that figure is the thing to check in the original. The neatness of the chain tells you nothing.
- Do not treat the summary as an audit trail. It is a description of reasoning, sometimes an incomplete one. For anything consequential, the answer still needs checking on its own merits.
- Watch the clock and the bill. If you are paying per token, thinking is output-priced and largely invisible. Check the usage breakdown rather than guessing.
- Start fresh for a new problem. On models that retain earlier thinking, a long session carries all of it forward and bills it again.
The thing worth remembering
Reasoning mode makes the model write its working down before answering, and on problems with real steps that genuinely helps. What it does not do is show you how the answer was reached. The text you are reading is a summary produced alongside the reasoning, and the vendor best placed to check found it often leaves out what actually moved the answer. Useful tool, unreliable window.
Header photo: Roman Mager / Wikimedia Commons, CC0.