There's a lot of excitement surrounding the power of AI agents and their ability to autonomously complete complex tasks (as I touched on in a recent post.) But behind this trend toward agentic AI, there has also been a key underlying breakthrough in Large Language Models (LLMs) - the emergence of state-of-the-art reasoning capabilities.
In this article, I want to talk about this pivotal shift from models that operate through pattern recognition to those that approach more sophisticated human-like reasoning, displaying capabilities such as 'thinking' before responding.
Companies such as OpenAI, Alibaba Cloud, Google, and, more recently DeepSeek, are among those at the forefront of developing reasoning models. IBM is now incorporating reasoning capabilities into its Granite series of LLMs designed for enterprise use.
In fact, OpenAI lists reasoning as the second step on what it suggests are the five stages that AI needs to progress through to achieve Artificial General Intelligence (AGI) - the point at which AI will outperform humans on most tasks.
So, what are reasoning models?
Until recently, most popular LLMs - like OpenAI's GPT 4, Google Gemini and Meta's Llama - operated by using pattern recognition. When asked a question, they refer to all the vast number of sources they have been trained on to find similar patterns of words. They rely on this to quickly develop an answer by predicting each next word to construct a coherent reply.
These AI models are powerful, and they excel in use cases such as writing human-like text, extracting insights from longer text, and summarizing. They are also good for translating text and answering general knowledge questions. However, they are far away from anything that we would recognize as thinking.
Reasoning models like OpenAI's GPT O3-mini and O1 and DeepSeek R1 work differently. Instead of responding immediately by pattern matching and prediction to create a statistically more relevant answer, they seem to take a step back to consider and decide on the best way to respond. They mirror the human capability to break down complex challenges and tackle them step-by-step. IBM Fellow, Kush Varshney goes as far as saying, "We are now starting to put wisdom into these models, and that's a huge step."
Here's a simple comparison of how traditional chat models differ from reasoning models:
LangChain (2025, January 22). Understanding and effectively using AI reasoning models [Video] YouTube
How does reasoning work?
It's helpful to refer to Daniel Kahneman's concept of the two types of human thinking - fast and slow. People can think fast - when responding quickly and intuitively to simple, straightforward questions (what is your address or your age?, name the capital of Italy, etc.) And, we can also think slowly, for example, when working out a difficult math problem by breaking it up according to the formula and tackling each part before arriving at a final answer.
Thinking fast - also known as 'System 1' thinking - is akin to when a chatbot AI model responds immediately to questions based on pattern recognition. Slow thinking - or 'System 2' thinking - involves the capability to consider and think things through which is mimicked by reasoning models.
Reasoning models rely on various techniques when thinking slow. One of this is Chain-of-Thought (CoT) reasoning, whereby the AI breaks down complex problems into manageable components tackling each one in a logical 'chain-of-thought' as I have already described.
Another is the self-reflection technique which involves the AI stopping to evaluate and consider the outputs or answers it has produced. By questioning and challenging what it has produced already, it can make refinements, leading to more accurate results.
Reinforcement Learning (RL) or Reward Functions involves training the underlying model to learn effective reasoning strategies through reward signals. The model is allowed to use an iterative process to tackle tasks in different ways; each time it is able to respond correctly or accurately, it is rewarded, enhancing its reasoning capabilities.
Impact on problem-solving
The introduction of reasoning is expected to significantly enhance AI's problem-solving abilities across various domains and use cases.
For example, in mathematics and coding, reasoning models have the potential to deliver better performance in solving mathematical problems and generating sophisticated code. In scientific research, there's the prospect of accelerating our capabilities to tackle important global challenges. Could AI reasoning help us develop life-saving drugs or more sustainable building materials?
In the legal and medical sectors, reasoning can deliver in areas such as comparing legal contracts, medical data analysis, and running complex workflows.
Choosing reasoning versus pattern recognition
While I've praised reasoning and talked about its positives, it is also important to appreciate that when reasoning models think slow, this comes at a cost. They're also doing more work. In other words, they are using more compute power and energy. So, it's not efficient to use reasoning for all tasks/use cases, nor is it environmentally friendly.
In use cases, like answering general knowledge questions such as "who was the third president of the United States" or a translation such as "tell me how to say 'I love you' in Spanish", the fast-thinking pattern recognition approach will be more efficient. Reasoning would be overkill. However, for more complex problems that require working through numerous steps, pattern recognition is less likely to provide a correct answer or solution. Reasoning would be the more effective.
So you need to select the correct approach, depending on the requirement. Some LLMs like the IBM Granite series models, are making this easier by allowing users to toggle reasoning capabilities on and off, depending on the task they need to address. This allows you to use the most effective and efficient tool - pattern recognition or reasoning - to tackle the job.
How reasoning models underpin AI agents
Let's now go back to AI agents. What defines an AI agent is that when you give it a complex task, not only can it come up with a plan for how to address it, but it can also execute that plan without constant human supervision or interaction.
For example, if you asked an AI agent to find a free week that fits in with the schedules for you and your three friends for a holiday in London - and to identify the flights and hotels at a specific price point and let everyone involved know about this by email, it could do this.
For this to happen, the agent first and foremost needs to have reasoning capabilities to break the task down into smaller steps (i.e. check people's calendars to find the dates, surf the web for flight and hotel details and then send an email to each of the friends).
However, as I wrote in a previous post, in addition to reasoning, an AI agent needs three other important resources.
It needs access to tools (such as email calendars, web search capabilities and to email/comms tools); the ability to gather feedback (e.g. through APIs or from humans) to check if the plan is working (for example, by reflecting on the outputs at each stage using self-reflection) or needs to be adjusted; and memory, to allow the agent to keep the plan 'in mind' as it goes through each step.
But at the end of the day, the emergence of reasoning is probably the biggest factor in the development of AI agents.
Reasoning will shape the advancement of agentic AI
Reasoning is key to the way AI agents tackle complex tasks autonomously. By simulating human-like reasoning, reasoning capabilities can help AI agents to break down problems, self-reflect to evaluate their own outputs and to improve over time. However, reasoning also comes with a trade-off in efficiency, so it is essential to choose the right type of model - reasoning or pattern recognition - for the task at hand. As AI continues to evolve, reasoning capabilities are likely to play a leading role in shaping more sophisticated, effective AI agents.
This blog was first published on the IBM Community.