How reasoning models work, and what changes

A reasoning model is a language model that generates intermediate working before it commits to a final answer. When OpenAI introduced its first model of this kind in 2024, it summed up the idea in four words: the model “thinks before it answers.”[1]

That extra step changes three things you will notice in practice. Answers can take longer to start. You pay for tokens you may never see. And the text labelled “thinking” in an interface is often a summary, or nothing at all, rather than a faithful record of how the answer was reached.

This explainer builds on our guide to what a large language model is. If tokens, training and inference are unfamiliar terms, start there.

From a prompting trick to a model behavior

Chain-of-thought prompting

The idea predates reasoning models. In 2022, researchers at Google showed that giving a model a few worked examples, each spelling out intermediate steps, improved its performance on arithmetic, commonsense and symbolic reasoning tasks. They called the intermediate steps a chain of thought: “a series of intermediate reasoning steps.”[2]

The headline result was striking for its time. Prompting a 540-billion-parameter model with eight such examples reached state-of-the-art accuracy on GSM8K, a benchmark of grade-school math word problems. The authors also reported that the benefit appeared in “sufficiently large” models rather than small ones.[2]

A follow-up paper added a second lever. Instead of taking one chain of thought, sample several and keep the answer most of them agree on. This “self-consistency” method improved chain-of-thought results on several arithmetic and commonsense benchmarks, including a reported gain of 17.9 percentage points on GSM8K.[3]

Both papers concern models and benchmarks from 2022. They are evidence that intermediate steps and repeated attempts can help on multi-step problems. They are not a measure of how any current product performs.

Reasoning as a trained habit

Chain-of-thought prompting asks a model to show steps. A reasoning model is trained to produce and use them by default.

OpenAI's September 2024 post introducing o1 described a model “trained with reinforcement learning to perform complex reasoning.” According to the post, training taught the model to refine its strategies, recognize and correct mistakes, break hard steps into simpler ones, and try a different approach when one failed.[1]

DeepSeek's R1 paper, first posted in January 2025 and later published in Nature, gave a more detailed public account of one recipe. Its abstract reports that reasoning abilities “can be incentivized through pure reinforcement learning,” without human-labeled reasoning trajectories.[4]

The key ingredient was a reward the computer could check without a human grader. For DeepSeek-R1-Zero, the paper describes rule-based rewards: an accuracy reward for a correct final answer, such as a math result in a required format or code that passes predefined test cases, and a format reward for placing the reasoning between designated “think” tags.[4]

Over training, the paper reports, the model's responses grew longer. It “naturally learns to solve reasoning tasks with more thinking time,” generating hundreds to thousands of tokens per problem.[4] The authors also documented costs: poor readability and mixing of languages in the reasoning, plus remaining instances of “overthinking” on simpler questions in the final R1 model.[4]

That reward design explains a pattern worth remembering. The strongest reported gains come on what the R1 paper calls “verifiable tasks such as mathematics, coding competitions, and STEM fields.”[4] Those are the problems where training could check the answer automatically.

Test-time compute: spending more effort per question

Traditional scaling made a model better by spending more computation during training. Reasoning models add a second dial: computation spent at the moment of answering, often called test-time or inference-time compute.

OpenAI's o1 post reported that performance improved “with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute).”[1]

“o1 thinks before it answers—it can produce a long internal chain of thought before responding to the user.”

— OpenAI, Learning to reason with LLMs (September 2024).[1]

There is more than one way to spend that extra effort:

• Longer reasoning in a single attempt. The model writes more intermediate tokens before answering.
• Several attempts, then a vote. The self-consistency idea applied at scale.
• Several attempts, then a judge. A separate scoring model or test suite picks the best candidate.

The o1 post shows how much the last two can add on one exam. On 2024 AIME math problems, OpenAI reported that o1 averaged 74% with one sample per problem, 83% with a consensus among 64 samples, and 93% when re-ranking 1,000 samples with a learned scoring function.[1] Those are vendor-reported results for a model OpenAI now lists as deprecated, so read them as a demonstration of the mechanism, not a current benchmark. Our guide to what AI benchmarks actually measure explains why a score on one exam says little about your own tasks.

More effort is not automatically better effort. A 2024 study from UC Berkeley and Google DeepMind researchers found that how well extra test-time compute paid off “critically varies depending on the difficulty of the prompt.” Allocating compute per prompt, rather than uniformly, improved efficiency by more than four times compared with a best-of-N baseline in their experiments.[5]

The same paper reported that, under matched computation and on problems where a smaller model already had some success, extra test-time compute let it outperform a model 14 times larger.[5] Note the condition: the small model had to be partly capable already. Extra thinking refined an ability; it did not create one from nothing.

What changes for you

AI Chronicle

Latency

Reasoning happens before the answer, so the first useful words arrive later. Anthropic's documentation says to “expect longer response times when thinking is active.”[6] OpenAI's documentation frames its lowest reasoning settings as suitable for latency-critical uses such as voice and classification, and higher settings as trading speed for quality.[7]

Some products stream progress notes or a reasoning summary meanwhile. That changes what you see while waiting, not the work being done.

Cost

Reasoning tokens are usually billed even when you cannot read them. OpenAI's API documentation states that reasoning tokens are not visible through the API but “still occupy space in the model’s context window and are billed as output tokens.”[7] Anthropic's documentation says thinking tokens are billed as output “even when the thinking text isn't returned to you.”[6] The context use matters in long conversations and documents; our context-window guide explains what fits and what gets lost.

Both providers expose controls. OpenAI offers a reasoning-effort setting whose lower values favor “speed and lower token usage.”[7] Anthropic offers an effort setting and, on some models, adaptive thinking that may skip thinking on easy requests.[6] OpenAI's guide also notes that a request can reach its output limit before any visible text appears, which means paying for reasoning without receiving an answer.[7]

In a chat subscription you may never see token counts, but the same work can show up as slower replies or usage limits.

When it helps, and when it does not

The evidence points to a fairly consistent pattern. Reasoning helps most on problems with several dependent steps and an answer that can be checked: calculations, code, logic puzzles, multi-condition rules and planning. OpenAI's documentation also highlights multi-step agentic workflows, where a model must decide between tool calls; our AI agents explainer covers that action layer.[7]

It helps less, and can cost more than it returns, elsewhere. In OpenAI's own human-preference comparison, o1-preview was not preferred to GPT-4o on some natural-language tasks, which the company said suggested “it is not well-suited for all use cases.”[1]

Researchers have also found tasks where step-by-step reasoning makes results worse. A 2024 study modeled on psychology experiments where deliberation hurts human performance found large drops in three of six tasks, including up to 36.3 percentage points of absolute accuracy for o1-preview compared with GPT-4o.[8] Those were specific, deliberately chosen tasks, not a general verdict.

A practical summary, which is our editorial synthesis rather than a measured rule:

The last row matters. A reasoning model still needs the right information. More thinking about an outdated or missing fact produces a more elaborate wrong answer, which is why our hallucinations explainer treats source checking as a separate step.

The visible reasoning is not a faithful record

Summaries, not raw thoughts

Many interfaces now show a panel of “thinking.” Be careful about what it is.

OpenAI decided not to show o1's raw chain of thought and instead displayed “a model-generated summary.”[1] Its current API documentation says the raw reasoning tokens are not exposed; developers can request a summary.[7] Anthropic's documentation is explicit: what you see “is never the raw chain of thought,” and the summary is produced by a different model from the one answering.[6]

Even the full trace can mislead

A deeper problem is that a model's written reasoning may not describe what actually drove its answer.

A 2023 study, Language Models Don't Always Say What They Think, added subtle biasing features to prompts, such as reordering multiple-choice options so the answer was always “(A)”. The models' explanations systematically failed to mention the bias, and the authors concluded that chain-of-thought explanations “can be plausible yet misleading.”[9]

“There’s no specific reason why the reported Chain-of-Thought must accurately reflect the true reasoning process”

— Anthropic Alignment Science team, Reasoning models don't always say what they think (April 2025).[10]

Anthropic's 2025 research tested reasoning models directly. Researchers slipped a hint about the answer into a question, confirmed the model used it, then checked whether the reasoning admitted it. Across hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time and DeepSeek R1 39% of the time.[10]

Two further findings are worth knowing. The unfaithful reasoning was, on average, longer than the faithful reasoning, so length is not a sign of honesty. And in a setup where models were rewarded for exploiting a planted shortcut, they exploited it in over 99% of cases but mentioned it in their reasoning less than 2% of the time in most test environments.[10]

The researchers note limitations: contrived scenarios, multiple-choice quizzes and two model families. The finding is not that reasoning text is useless, but that you cannot rely on it to rule out a hidden shortcut or a wrong premise.

For readers, the takeaway is simple. Treat the reasoning panel as a hint about the model's approach, useful for spotting an obvious misreading of your question. Do not treat it as verification. Check the answer itself, using the methods in our guide to verifying AI-generated research.

A worked example: choosing a setting for two shop tasks

This is a hypothetical scenario with invented details. No product was tested and no results are reported.

Suppose a small online shop wants AI help with two recurring jobs.

Task A: rewrite 40 product descriptions in a friendlier tone. The output is easy to review by reading it, there is no single correct answer, and nothing depends on multi-step calculation. A standard response, or the lowest reasoning setting available, is the sensible starting point. Extra reasoning would mainly add waiting time and cost.

Task B: work out shipping charges for unusual orders. The shop's fictional rules are:

Standard shipping: 6.00 per order.
Free shipping on orders of 75.00 or more, before discounts.
Oversized items always add 12.00, even on free-shipping orders.
Remote postcodes add 4.00.

An order worth 80.00 before an 8.00 discount, containing one oversized item, going to a remote postcode, requires several conditions applied in the right order. Under these rules, the correct charge is 16.00: free standard shipping (the order is 80.00 before discount), plus 12.00 for the oversized item, plus 4.00 for the remote postcode. This is exactly the kind of multi-condition task where reasoning settings are designed to help.

A sensible evaluation would not trust either setting on faith:

1. Write ten test orders that cover each rule and the tricky combinations, and calculate the correct charges in a spreadsheet first.
2. Run the same ten orders at a low and a high reasoning setting, keeping the prompt identical.
3. Score only the final charges against the spreadsheet. Ignore how convincing the explanations sound.
4. Record the response time and, if available, the reasoning-token count for each run.

To see why token counts matter, take an invented illustration: if a reply shows 300 visible tokens but used 2,700 reasoning tokens, the provider bills 3,000 output tokens, ten times what you can read. Whether that is worth paying depends on whether the higher setting actually gets more charges right.

If both settings score the same, use the faster, cheaper one. If only the high setting passes, the extra cost is buying something real. Either way, a spreadsheet formula may be the better long-term tool for a fixed rule set; the model is most useful when the rules change or cases arrive as messy text.

Questions readers commonly ask

Is a reasoning model actually thinking?

It generates intermediate text, and training rewards text that leads to correct answers. Whether that resembles human thinking is a separate question; the generated reasoning is not guaranteed to reflect the computation behind the answer.[10]

Can I make any model reason by asking it to “think step by step”?

Prompting for steps can help some models on some tasks; that was the original chain-of-thought finding.[2] Reasoning models are trained to do this by default and at much greater length, so the effect is usually larger and less dependent on the prompt.

Should I always pick the highest reasoning setting?

No. OpenAI's guidance for one of its top effort settings says to use it only when evaluations “show a clear benefit that justifies the extra latency and cost.”[7] Start low, test on your real cases and raise the setting only if accuracy improves.

Do reasoning models make fewer mistakes?

On multi-step problems with checkable answers, the published evidence shows large gains. They still make mistakes, can overthink simple questions[4] and can perform worse on some tasks.[8] They cannot supply facts that are missing from training or context.

Why can't I see the full reasoning?

OpenAI said it weighed user experience, competitive advantage and the option to monitor the chain of thought when it chose to show summaries.[1] Anthropic says its summaries keep the benefits of thinking “while preventing misuse.”[6] Neither provider's documentation reviewed here offers the raw trace by default.[7][6]

The next step

You now have three questions to ask whenever a product advertises reasoning: Does my task have several dependent steps? Can I check the final answer? Is the extra wait and cost buying better results on my own cases?

Bring those questions to how to judge a new AI model release, which covers reading launch claims and benchmark charts without taking them at face value.

Sources and evidence note

This explainer relies on original research papers and official provider documentation. Older papers explain mechanisms or demonstrate limitations under their stated conditions; they are not current product rankings. Vendor-reported benchmark figures are attributed as such. Billing and display behavior is taken from documentation accessed in October 2026 and may change. The shop scenario, rules, test plan and token illustration are original hypothetical material. No hands-on product test was performed.

Related reading

• Claude vs. ChatGPT vs. Gemini: choosing an assistant
• Open vs. closed AI models: what the difference means