1. What Is a Reasoning Model?

A standard language model generates tokens one at a time, left to right, without the ability to go back and revise. Each token is produced once and becomes part of the context for the next token. This works well for text generation, summarisation, and extraction. It works poorly for problems that require planning, backtracking, or working through multiple steps before committing to an answer.

A reasoning model is a language model trained to produce an extended chain of thought before generating its final answer. Instead of jumping directly from the question to the answer, the model writes out its reasoning process. It breaks the problem into steps, checks each step, considers alternatives, and sometimes corrects itself before arriving at a conclusion. The chain of thought is generated as intermediate tokens that are produced before the final answer.

The core idea behind reasoning models is that giving the model more tokens to think with, meaning more intermediate computation, improves its performance on hard problems. This is called test-time compute or inference-time compute. A standard model spends a fixed amount of compute per token regardless of question difficulty. A reasoning model scales its compute with the difficulty of the problem, spending more tokens on harder questions and fewer on easier ones.

2. Why Reasoning Models Exist

Standard language models have a well-documented weakness. They struggle with tasks that require multi-step reasoning, where each step depends on the result of the previous one. Math problems, logic puzzles, coding challenges, and scientific questions all require this kind of sequential thinking. A standard model that generates the answer directly often gets it wrong because it cannot look ahead and check its work.

Chain-of-thought prompting, covered in the prompt engineering guide, was an early attempt to address this. By asking the model to show its work, you could improve its performance on reasoning tasks. The model writes out intermediate steps, and those steps become part of the context that guides the final answer. This works because each token the model generates becomes part of the context for subsequent tokens, so writing out reasoning steps gives the model a richer context to draw from when it produces the answer.

Chain-of-thought prompting had limits. The model was still a standard model trained on next-token prediction. It had not been specifically trained to reason effectively, so its chains of thought were sometimes shallow, circular, or wrong. The model would write out reasoning that looked correct but contained logical errors, and then arrive at a wrong answer with apparent confidence.

Reasoning models close this gap through training. Rather than relying on the prompt to elicit reasoning, the model is trained on examples that include detailed reasoning traces. During training, the model learns to produce effective reasoning chains, not just plausible-looking ones. It learns to break problems down, check intermediate results, and backtrack when a path looks wrong. This is achieved through reinforcement learning, where the model is rewarded for arriving at correct answers through valid reasoning steps.

3. How Reasoning Models Work

The mechanics of a reasoning model build on the same Transformer architecture as standard LLMs. The difference is in training and inference.

4. Reasoning Models vs Standard Models

Reasoning models and standard models serve different purposes. Understanding when to use each is a practical decision that depends on the task, the latency budget, and the cost tolerance.

The choice between a reasoning model and a standard model is rarely either-or. Many systems use both. A standard model handles the initial query, determines what kind of question it is, and routes complex reasoning tasks to a reasoning model. This routing approach gives you the speed of a standard model for easy questions and the accuracy of a reasoning model for hard ones.

5. What Reasoning Models Are Good At

6. What Reasoning Models Struggle With

Reasoning models are not universally better. There are tasks where the extended reasoning provides no benefit and only adds cost and latency.

7. Cost and Latency Trade-offs

The primary cost of reasoning models is the additional tokens they generate. A reasoning trace can be 10 to 100 times longer than the final answer. If the answer is 50 tokens and the reasoning trace is 5,000 tokens, the model is generating and you are paying for 100 times more output than you see.

On latency, the model generates tokens sequentially. A 5,000-token reasoning trace takes 50 to 100 seconds to generate at typical speeds, depending on the model and infrastructure. For interactive applications, this delay is significant. A user who asks a question and waits two minutes for an answer may lose patience, even if the answer is better than what a standard model would produce in two seconds.

API providers typically price reasoning tokens separately from output tokens, and some allow you to set a reasoning budget, a cap on how many tokens the model can spend thinking. This lets you control cost, but setting the budget too low can prevent the model from finishing its reasoning, leading to incomplete or wrong answers.

8. When to Use a Reasoning Model

The decision to use a reasoning model depends on the task, the accuracy requirements, and the tolerance for cost and latency.

9. Reasoning Models and Hallucination

The previous guide covered hallucination, where models produce confident but factually wrong output. Reasoning models offer a partial improvement here, but the relationship is nuanced.

On reasoning errors, where the model makes a logical mistake or calculation error, the extended chain of thought helps. The model can catch errors in its own reasoning, check intermediate results, and correct course. This reduces errors on math, logic, and coding tasks.

On factual hallucination, where the model states something untrue, reasoning provides less help. If the model does not know a fact, thinking longer does not make the fact appear. The model may reason about the fact, drawing inferences based on adjacent knowledge, but those inferences can be wrong. Extended reasoning can even amplify factual hallucination, because the model generates more text and has more opportunities to introduce unsupported claims.

The practical takeaway is that reasoning models are not a substitute for grounding. For factual accuracy, RAG and knowledge graphs remain the primary tools. Reasoning models complement these by improving the model's ability to reason over the retrieved information, but they cannot conjure facts that are not in their training data or in the provided context.

10. The Future of Reasoning Models

Reasoning models represent a direction in AI development where performance improves by spending more compute at inference time rather than only at training time. This is a meaningful change because it decouples capability from model size. A smaller model that reasons longer can outperform a larger model that answers immediately.

Current reasoning models still have limitations. Their reasoning traces can be verbose and hard to parse. They sometimes reason in circles or fail to converge on an answer within their token budget. The quality of reasoning depends heavily on training data, and models can learn reasoning patterns that look correct but contain subtle logical errors.

Research is active on several fronts. Techniques for more efficient reasoning, where the model produces shorter and more focused chains of thought, could reduce cost and latency without sacrificing accuracy. Approaches that combine reasoning with tool use, where the model can call external tools like calculators, code interpreters, or databases during its reasoning process, extend what the model can accomplish. And the integration of reasoning models with agentic systems, where the model plans and executes multi-step workflows, is an area of rapid development.

Frequently Asked Questions