Gen AI

Reasoning Models

Instead of trying to instantly guess the right answer to a complex problem, reasoning models are trained to output a long, hidden chain of intermediate thoughts before providing their final answer.

Why Does This Exist?

Standard LLMs suffer from a fatal flaw: they spend the exact same amount of compute generating the word "cat" as they do generating the solution to a complex calculus problem.

Because standard models are purely autoregressive, they must output their answer immediately. If you ask a standard model a hard riddle, it has to essentially "guess" the final conclusion in a single forward pass. Humans don't work this way. When faced with a hard problem, humans use "System 2" thinking—we stop, we outline the problem, we try a few paths, we correct our mistakes, and then we state our answer.

Reasoning models bring this capability to AI. They are trained to generate a long chain of intermediate "thought" tokens before they are allowed to output the final answer.

Think of It Like This

The math test scratchpad

A standard LLM is a student taking a math test where they are only allowed to write the final answer in the box. If the problem is $12 \times 34$ , they have to do the entire calculation in their head instantly and write "408".

A reasoning model is a student who is given a massive piece of scratch paper. They can write out $12 \times 30 = 360$ , then $12 \times 4 = 48$ , then $360 + 48 = 408$ , and finally copy "408" into the answer box. By externalizing the intermediate steps, they can solve infinitely harder problems.

How It Actually Works

Externalizing Compute

In a transformer architecture, the only way to "think longer" about a problem is to generate more tokens. Every time the model generates a token, it runs another full forward pass through its neural network, effectively unlocking more compute time.

Reasoning models leverage this. When prompted, they don't immediately begin outputting the answer. Instead, they output specialized tokens (often internally tagged like <thought>) where they begin breaking the problem down.

The Hidden Chain

This chain of thought is often hidden from the end user. The model might generate 2,000 tokens of internal monologue: exploring hypotheses, running Python code in a sandbox, realizing a mistake, back-tracking, and summarizing findings. Because these tokens are appended to the context window, the model's self-attention mechanism can refer back to its own earlier logic.

Once the model is confident, it outputs an ending token (like </thought>) and begins generating the final, user-facing answer based on the rigorous logic it just laid out.

Why is it hidden?

Companies often hide the raw reasoning tokens from users for two reasons:

Safety and Alignment: The raw reasoning process is often completely unfiltered and can bypass safety training. The model might "think" about how to build a bomb, realize it's dangerous, and output a safe refusal. Exposing the thought process exposes the dangerous information.
Competitive Advantage: The specific patterns the model uses to reason are the product of incredibly expensive chain-of-thought training. Exposing them allows competitors to train their own models on the data.

Show Me the Code

You can simulate the effect of a reasoning model on a standard model by enforcing a strict prompt template that forces the model to use a scratchpad.

1# A standard zero-shot prompt forces the model to guess instantly2standard_prompt = """3Q: If John is twice as old as Mary, and Mary was 5 years old 3 years ago, how old is John in 10 years?4A: 5"""6
7# A simulated reasoning prompt forces the model to externalize compute8reasoning_prompt = """9Q: If John is twice as old as Mary, and Mary was 5 years old 3 years ago, how old is John in 10 years?10
11Before answering, you MUST use the <scratchpad> tags to break down the problem step-by-step.12
13<scratchpad>14# The model is now forced to generate tokens here, 15# effectively unlocking "test-time compute".161. Mary was 5 years old 3 years ago. 172. So Mary is currently 5 + 3 = 8 years old.183. John is twice as old as Mary.194. So John is currently 8 * 2 = 16 years old.205. In 10 years, John will be 16 + 10 = 26.21</scratchpad>22
23Final Answer: 26.24"""

True reasoning models are actually fine-tuned on data shaped exactly like the reasoning_prompt example, fundamentally rewiring them to prefer this step-by-step approach over instant guessing.

Watch Out For

Overthinking simple tasks

Because reasoning models are heavily trained to externalize their thoughts, they often apply this rigorous, compute-heavy process to incredibly simple questions. If you ask a reasoning model, "What is the capital of France?", it might spend 500 tokens "thinking" about European geography before answering "Paris." This wastes massive amounts of time and API credits. You should only route complex, multi-step logic problems to reasoning models.

The Quick Version

Standard LLMs are forced to instantly guess the answer to complex problems in a single forward pass.
Reasoning models are trained to output a long sequence of intermediate "thought" tokens before answering.
Generating these extra tokens unlocks more compute time (test-time compute) for the problem.
This allows the model to outline steps, catch its own mistakes, and solve problems standard models fail at.
The raw reasoning chain is often hidden from the user for safety and commercial reasons.