Artificial Intelligence
August 18, 2026
STAMFORD, Conn., Aug. 18, 2026 — AI inference costs per agentic workflow will increase more than fivefold through 2028 Inc., a business and technology insights company
As AI products evolve from assistive features to multistep execution, product leaders face a new margin challenge with falling model prices subsidizing more complex workflows and escalating total AI costs. As a result, inference cost management has become a top priority for product leaders.
“Product leaders cannot rely on more efficient token economics to rationalize AI costs,” said Will Sommer, Sr. Director Analyst. “Each successive generation of AI capability will necessitate more, and often more expensive, tokens. There is no reliable, economical one-size-fits-all model on the horizon. Producing competitive AI products will require developing and maintaining complex multimodel ecosystems.”
Gartner has identified three fundamental trends driving token economics:
- Foundational model cost economics are rapidly improving.
- Improved AI efficiency is unlocking the deployment of more powerful, and more expensive, models to enable higher-value, more sophisticated AI applications.
- More sophisticated AI workflows use far more tokens than simple chatbot interactions, driving higher overall inference costs.
These dynamics mean that tokens are becoming more cost-efficient, but not as quickly as AI capabilities and the costs associated with those capabilities are increasing (see Figure 1). The rate of innovation is outpacing the cost curve.
Figure 1: AI Complexity Outpaces Falling Token Costs
This is the Inference Paradox, defined as better unit economics escalating the overall cost of AI without providing a clear pathway to commensurate and predictable value.
“The harsh economics of the Inference Paradox are exemplified by the differences between a simple chatbot and an AI agent,” said Sommer. “Where a simple chatbot must read and interpret a query and quickly respond with a probabilistically reasonable answer, an AI agent must constantly reason, negotiate, and question itself.”
All of these responsibilities add up. Compared to a basic chatbot interaction, routing a task to an agentic reasoning model increases provider inference costs by at least five times, and often much more as task complexity grows.
Ensuring ROI from advanced AI, like reasoning agents, demands exponentially higher returns relative to basic models, or highly optimized inference-tiering, routing and orchestration to calibrate complex tasks relative to more cost-efficient intelligence. Both of these outcomes are eminently possible but will require significant effort across complex workflows.
“Defaulting to generic autonomous intelligence will result in unbounded costs orders of magnitude higher than those of optimized product ecosystems,” said Sommer.
Gartner (NYSE: IT) delivers actionable, objective business and technology insights that drive smarter decisions and stronger performance on an organization’s mission-critical priorities. To learn more, visit gartner.com.
The Big Data Inside Amazon’s New Fire Phone
The new Fire phone that Amazon launched this week looks like your ordinary black smartphone,…
Three Reasons to be Scared of the Internet of Things
We know the Internet of Things forecasts: 50 billion connected devices by 2020. Apparently, there’s…
Wanted: Intelligent Middleware That Simplifies Big Data Analytics
We’ve seen tremendous technological innovation in the data analytics space over the past 10 years….
GPUs Tackle Massive Data of the Hive Mind
LIVE from GTC12 — The flock of birds that weaves seamlessly through the sky, propelled…
Why Hadoop on IBM Power
In the quest to achieve data-driven insight, Hadoop running on Intel X86-based processors has emerged…
Datanami’s Leverage Big Data Summit Wraps Up
Dialog and networking were on the Datanami agenda this week as we kicked off our…
AI Agents Are Creating a New Data Problem. Ciklum and ClickHouse Have a Plan
Technology services company Ciklum has formed a strategic partnership with real-time analytics database provider ClickHouse,…
AI Is Forcing Analytics Teams Into a New Role
For years, analytics teams have been responsible for helping organizations understand their data. They built…
DeepSeek Open-Sources the Missing Layer Between AI Models and Agents
AI models are getting better at a rapid pace. They are now able to reason,…
AI Turns Genomic Data Into 16 New Bacteria-Killing Viruses
Scientists are using GenAI and massive stores of genomic data to design new biological systems…
NVIDIA’s $500 Billion AI Bet: Jensen Huang Brings Wall Street Into the Race
Jensen Huang believes NVIDIA’s chips are becoming much more than expensive pieces of hardware. As…
Your Data is Not Ready: Solving the First Mile Gap for Enterprise AI
There’s trouble brewing in the enterprise AI world and IT leaders are faced with a…
