What Are Reasoning Models? How AI Learned to Think Step by Step

In September 2024, OpenAI released a model that did something unusual for the time: it paused before answering. On a hard high-school math contest called AIME 2024, the previous flagship model scored around 12%. The new one, which took time to work through the problem before committing to an answer, scored around 74%. The underlying knowledge in the model hadn’t changed dramatically. What changed was when the computation happened — at answer time, not just during training. That shift created a genuinely new category of AI system: the reasoning model.

What a Reasoning Model Actually Is

A standard large language model, the kind covered in how LLMs work, predicts its response in essentially one pass, generating the most likely next word over and over until the answer is complete. A reasoning model does something different. Before committing to a final answer, it first generates an extended internal chain of reasoning — exploring the problem, checking its own logic, and sometimes backtracking when a line of thought doesn’t hold up — and only then produces the response you actually see. The comparison researchers reach for most often comes from psychology, borrowed from Daniel Kahneman’s framework in Thinking, Fast and Slow: standard models operate like System 1, fast and intuitive, producing an answer in one instinctive pass. Reasoning models add something closer to System 2, slow and deliberate, willing to spend extra effort working through a problem before responding.

Chain-of-Thought: The Idea That Started This

The underlying mechanism, chain-of-thought reasoning, isn’t new. Researchers discovered back in 2022 that simply adding “think step by step” to a prompt made existing language models noticeably better at multi-step problems, because it forced the model to write out intermediate steps instead of jumping straight to a final answer. For a couple of years, this stayed a prompting technique — something a user had to explicitly ask for. The real shift happened when researchers stopped treating this as a prompting trick and started training it directly into the model through reinforcement learning. Instead of a user asking the model to think step by step, the model learns, during training, that generating and checking its own intermediate reasoning produces better final answers — and it starts doing this automatically, without being asked. This is the difference between a model that can reason if you prompt it correctly and a model that reasons by default.

The Third Scaling Axis: Why This Wasn’t Obvious Sooner

For most of the past decade, improving AI models meant one of two things: bigger models, or more training data. Around 2024, both of those started hitting diminishing returns — high-quality training data in particular became genuinely scarce. A research paper on scaling test-time compute demonstrated a third lever nobody had fully exploited: performance keeps improving, following a predictable pattern, as you increase the compute spent during inference — the thinking a model does at answer time — independent of making the underlying model itself any bigger. This matters because it changes where the improvement comes from. A reasoning model working through a hard problem might generate thousands of tokens of internal working before producing a final answer that’s just a single number or a short paragraph — most of that computation never gets shown to the user, but it’s what made the answer more likely to be correct. This connects directly to how tokens and context windows work, since a reasoning model’s extended thinking process consumes real token budget even when the visible output is short.

How the Training Actually Works

Reasoning models are typically trained with a combination of techniques covered separately in how AI models are trained. Reinforcement learning plays the central role here specifically: rather than being told exactly what the “correct” reasoning path looks like, the model is rewarded when its reasoning leads to a verified correct final answer, and penalized when it doesn’t. Over many training iterations, this teaches the model to internalize which kinds of intermediate steps — double-checking a calculation, considering an alternative approach when the first one stalls — actually improve its odds of getting the answer right. This training approach turned out to be reproducible outside of any single company. OpenAI’s o1 was the first widely used commercial reasoning model, but within a few months, an open-weight model demonstrated that the same reinforcement-learning-driven reasoning approach could be built and released publicly, showing the technique wasn’t dependent on any one lab’s proprietary methods. That moment mattered for the field because it meant reasoning capability, not just raw model size, had become something researchers outside a handful of frontier labs could work with directly.

RL is increasingly the layer that turns a raw language model into one that actually follows instructions — see how reinforcement learning taught AI to reason for the loop of act, get feedback, and adjust behind it.

Beyond a Straight Line: Search and Self-Checking

The simplest version of this — generating one long chain of reasoning and stopping — isn’t the whole picture in more advanced systems. Some reasoning models explore multiple possible reasoning paths simultaneously rather than committing to a single line of thought, evaluating which branches look most promising and allocating more effort to those, conceptually similar to how a chess engine considers multiple possible moves before selecting one. Verification is often built in as a separate step too: the model generates a candidate answer, checks it against the constraints of the problem, and revises if something doesn’t hold up, rather than treating its first attempt as final. This is part of why reasoning models tend to perform meaningfully better on tasks with a verifiable correct answer — math, coding, logic puzzles — than on more open-ended writing tasks, where there’s no clean way for the model to check its own reasoning against a ground truth.

The Trade-Off: Reasoning Isn’t Free

None of this comes without cost. A reasoning model’s extra “thinking” time translates directly into slower responses and higher compute cost per query, since all that internal reasoning consumes real processing regardless of how short the final visible answer is. For a simple factual question or a straightforward rewrite, this overhead buys nothing — a standard model gets the same answer instantly, without the extra latency or cost. Reasoning models earn their advantage specifically on genuinely hard, multi-step problems where the extra deliberation actually changes the outcome. This has a practical implication worth understanding if you’re using AI tools day to day: choosing a reasoning-heavy model for every task isn’t automatically the better choice. It’s the right choice for a task that benefits from careful, checked, multi-step thinking, and unnecessary overhead for one that doesn’t.

Where Reasoning Fits Into the Bigger AI Picture

Reasoning ability is also one of the components that made more capable AI agents possible, since an agent covered in agentic AI often needs to plan a multi-step sequence of actions, check whether each step actually worked, and adjust course when something fails — all of which draws on the same underlying reasoning mechanism rather than a single-pass response. It’s also connected to a separate but related problem: reasoning doesn’t eliminate the risk of AI hallucinations, since a model can construct a long, confident-sounding chain of reasoning that still arrives at a wrong conclusion. Extended thinking makes correct answers more likely on verifiable problems, but it isn’t a guarantee against a model reasoning its way to a plausible-sounding but incorrect answer, especially on questions with no clean way to check the result. Efficiency has also become part of this story, connecting to how mixture-of-experts models control compute cost — since reasoning models are inherently more expensive to run per query, combining reasoning ability with more efficient underlying architectures is one of the active directions the field is pushing toward, rather than simply making reasoning models bigger across the board.

The Practical Takeaway

The core idea behind reasoning models is simple even though the underlying mechanics are involved: instead of getting better only by growing the model or feeding it more data, AI systems can also get better by learning when and how to think longer before answering, and being rewarded specifically for reasoning that leads to correct outcomes. That’s a genuinely different lever than the ones AI development relied on for most of the last decade, and it’s part of why the field’s most capable systems in 2026 look meaningfully different from the ones that came just two years earlier.

---

About the Author
Shurah writes and maintains DataPips independently, drawing on hands-on experience in trading and entrepreneurship. Articles are shaped by personal research, real trading lessons, and the process of building this publication from scratch — not by a formal financial credential.