AI tokens are cheaper than ever. So why are companies suddenly panicking about how much they’re spending on AI? Over the past year, companies pushed employees to use AI as much as possible. Engineers got access to tools like Claude Code, Cursor, Copilot, and a growing list of AI agents. Some companies even turned usage into a goal. The result was a wave of “tokenmaxxing,” where more AI use was treated like more progress. But that did not always happen. Uber blew through its AI budget in 4 months. Meta workers reportedly used tens of trillions of tokens in a month. Amazon had an internal AI leaderboard that employees started gaming. And across the industry, companies are realizing that more tokens does not always mean better work. The weird part is that token prices have dropped a lot since 2023. But AI tools now think longer, loop through more steps, reread more context, and handle more complex tasks. That means usage can explode even when each token gets cheaper. This is the strange economics of AI right now. Cheaper tokens created more demand. More demand created higher bills. And now the entire AI market is shifting from simple subscriptions to usage-based pricing.
Timestamps: 0:00 - AI Bubble 0:52 - Tokenmaxxing 6:23 - Over Reasoning 12:01 - The Token Market
AI Generated·May contain errors
You've probably seen the headlines. Uber burnt through its AI budget in 4 months.
Another company spent 500 million in just 1. Token costs have even gone far beyond the costs
of the employees. Yet all these companies try to spend as much as possible on tokens.
Companies have wildly overspent on tokens and are now cutting back. But all of this is especially
because tokens aren't actually getting more expensive. They're getting cheaper. The price
of a single token has fallen by about 90% since 2023. But AI spend keeps going up.
So what's going on? Why did cheaper tokens make AI more expensive than ever?
Something's happened that may have changed the AI industry for good.
It feels like just a few months ago, companies were pushing every employee to spend as many
tokens as possible. Many were calling it token maxing, literally meaning maximizing token usage.
You can probably see where this is going. At Uber, engineers were given access to cloud code
and cursor, and by March 2026, 84% of Uber's developers were classified as agentic coding
users. Then Uber placed a cap on employee AI spend after going through their entire AI budget
in just 4 months. I'm back to the drawing board because the budget I thought I would need
is blown away already. Even worse, Uber's COO and president said,
if you're not actually able to draw a direct line to how many useful features
and functionality you're shipping to your users, that trade becomes harder to justify.
That link is not there yet. But this list goes on and on. In 30 days,
meta workers reportedly ran through 60 trillion cloud tokens. Microsoft encouraged employees to
use different AI tools, but is now pushing everyone back to GitHub Copilot. Even though
this is framed as consolidation, most see it as cost cutting. Amazon set targets for more than
80% of developers to use AI each week. They even had an AI leaderboard, CairoRank,
which tracked AI token use among employees. Employees then began inflating their scores
through token maxing, basically giving AI useless tasks to increase token use.
CairoRank was then taken offline. Amazon's senior vice president said,
please don't use AI just for the sake of using AI. An even stranger case is Nvidia.
So are you spending, in Nvidia, a billion, two billion dollars on tokens for your engineering
team right now? We're trying to. Let me give you a thought experiment. Let's say you have a
software engineer or AI researcher and you pay them $500,000 a year. We do that all the time.
Okay. This is happening all of the time. That $500,000 engineer, at the end of the year,
I'm going to ask him, how much did you spend in tokens? And that person said,
$5,000. I will go ape something else. If that $500,000 engineer did not consume
at least $250,000 worth of tokens, I am going to be deeply alarmed.
About a month later, an Nvidia executive said, for my team, the cost of the compute is far beyond
the costs of the employees. Another anonymous AI client burned through 500 million on AI in just
one month because they didn't set limits on employee licenses. Shopify, Spotify, ServiceNow,
and Roku all mentioned in their earnings calls that AI has become a major pressure point on
operating expenses. Not only is the cost spiking, but so are the problems. One being code churn,
aka lines of code deleted versus added. Alex Cerci, the CEO of WayDev, said that managers
are seeing code acceptance rates of 80-90%, aka the AI code developers approve and keep,
but don't consider the amount of revision engineers have to do in the following weeks.
This drives the real world acceptance rates down between 10 and 30% of generated code.
KidClear found in a report that AI tools did make developers more productive,
but the amount of code churn they had to do was 2.2x greater than their productivity increase.
Pharaoh's AI found that based on 2 years of customer data, code churn had increased by 861%.
Another study of 2,444 companies found that for every dollar a company spends on AI tokens,
$0.44 is used to fix bugs generated by AI, $0.27 is to rewrite AI-produced code,
and $0.11 is consumed in review and merge delays. In other words, behind every dollar of AI
procurement cost lies nearly 80% in hidden losses. People are calling this the tokenpocalypse.
Companies tied token consumption to productivity and output, but all that happened was way higher
costs. They all fell into the same problem, Goodhart's law. When a measure becomes a target,
it ceases to be a good measure. Silicon Valley collectively just went through a very strange
experiment. Goldman Sachs reported that companies are overrunning their initial budgets for
inference by orders of magnitude. And what makes all this even stranger is that token cost has gone
down 90% since 2023. By other metrics, they've dropped by 280x, but simultaneously AI spend is
exploding by an estimated 320%. Worldwide ID spending is expected to hit $6.31 trillion in
2026, up 13.5% year over year, more than the entire annual GDP of the UK. US enterprises have
become more cost conscious in their AI spending, and this has caused another surprising phenomenon,
but more on that later. More importantly, why isn't more token usage converting into better
results? Why does it feel like AI is getting more expensive? Before we can answer this,
what does token actually mean? LLMs like ChatGPT don't understand language. They can't read what
you say, see an image you send, or hear the audio you speak. The word darkness, for example,
would be split into two tokens, dark and NES, with each token bearing a numerical representation,
such as 217 and 655. The model processes these input tokens, generates its response as tokens,
and then translates it to the user's expected format. According to NVIDIA, the goal is to achieve
the fastest processing time and lowest cost per token to optimize AI infrastructure and maximize
revenue generation. Keep that in mind, because something that's happening now makes that claim
rather ironic. And as a disclaimer, I'm not an AI expert, not even close, but what I am extremely
interested in is the business side, and why many businesses have done a 180 on their token spend.
There are a few factors to this. The first is that tokens consumed don't scale linearly with
productive output. A study by Jellyfish found that larger token budgets did result in more
pull requests, aka proposed changes to a shared codebase, but 10 times the tokens produced only
about 2 times the output. Tokens, in this sense, behave less like a linear input and more like
rocket fuel. Going faster is possible, but it requires exponentially more resources to do so.
Another problem, one that's perhaps most surprising, comes from the more advanced
reasoning models. Remember what NVIDIA said about fastest processing time? Well,
that's where we run into a problem. Cloud Opus and GPT-03, the ones that can tackle harder problems,
spend minutes or even hours thinking to solve a problem. Reasoning AI models, the latest
advancement in LLMs, can tackle more complex queries by treating tokens differently than
before. Here, in addition to input and output tokens, the model generates a host of reasoning
tokens over minutes or hours as it thinks about how to solve a given problem. The power of these
things is pretty unbelievable. So powerful that the US government has issued an export control
on Cloud Fable due to national security risks. But the more powerful models have caused an
interesting problem to occur. If you ask your friend what's 1 plus 1, they'll probably say 2
straight away. But imagine if they stopped and really deeply thought about it for 30 seconds
and then said 2. You'd think you already know the answer. Why'd you think so hard about it?
That's basically what's happening. AI right now has a bit of a reasoning problem,
or rather an inefficiency problem. If you ask some, not all, but some of the state-of-the-art
reasoning LLMs this question, it will stop, break down the question, deliberate over it,
and may even take 10 or 20 seconds. They can break tasks into smaller subtasks of a larger problem.
Greater potential output, but more tokens consumed. What's weird is that the lighter
AI models can do this immediately. Not that they inherently do math, but based on training
on probability, the right response to your query is 2. Every unnecessary reasoning cycle
increases latency, compounds infrastructure costs, and consumes energy. Recent analyses suggest that
unnecessary prompt verbosity alone costs tens of millions of dollars in excess computation annually.
So even though all those engineers get encouraged to use more AI, if the way they're used isn't
extremely specific and intentional to the task, it can be totally off. Complex tasks to a low-level
model, more problems, inadequate results. Easy tasks to a high-level model, longer task duration,
overconsumption of tokens, inefficient results. While these are powerful tools, they're often
deployed indiscriminately across a wide variety of tasks, including countless queries that likely
require no reasoning at all, and this inefficiency has real consequences. All of this compounds even
more when we look at agentic AI. This is when you give an AI model a goal and a collection of tools.
Writing code, searching the web, writing files, calling APIs, and it runs autonomously. But while
chatting is a single call and response, an agent goes in a loop until it deems the task is done.
Look at the result, think again, do the next thing, repeat. Lots of small steps, lots of tokens used.
But this gets more expensive, because every time it loops back to think, it rereads all the context
so far. This is the agentic loop multiplier, and it's what's driving up a lot of cost. Goldman Sachs
forecasts agentic AI could drive a 24-fold increase in token consumption by 2030, reaching
120 quadrillion tokens per month. With this in mind, you can see how easy it would be for employees
to waste tokens on nothing, especially when they're given a blanket rule to spend more tokens.
Again, good hearts law. More tokens doesn't always mean better results. But something doesn't quite
add up. We still come back to the question, why did cheaper tokens cause a spike in cost? Well,
this isn't actually anything new. Not only that, have you noticed how almost all AI companies have
recently changed their business model? We might be moving into a totally new stage in the AI
lifecycle. If you haven't subscribed yet, consider subscribing. We're on the road to 1 million and
are over 90% there. Thank you. Ironically, cheaper tokens are what drove more cost. As cost of
production decreases, demand and consumption often increase. The Jevons paradox. The more efficient
you make a resource, the greater the demand for it. This gives us crucial context for a major
change in the world of AI. Have you noticed that many of these platforms have changed from
subscriptions to token usage pricing? There are still subscriptions of course, but these just
give you an allocation of tokens which you can use daily. The high spending engineers and companies
will likely be setting token budgets rather than a fixed subscription rate. We're experiencing what
is a major market transition we've seen in all kinds of industries. Streaming services appear,
charging a low subscription, gaining market adoption, then raising prices and introducing
new revenue streams. Delivery apps did the same thing, losing money to get market penetration,
then changing to focus on profit. AI is doing the same thing, except changing the model to change
Since spring 2025, every major AI agent company, Cursor, Versos V0, Replit, Lovable,
almost simultaneously moved to token-based billing, shrank or eliminated free tiers,
and pushed overage charges onto users. It would be foolish to still charge a subscription base,
because AI is so powerful that it will automate a lot of tasks. And it's not just AI models either.
Just a few years ago, most software as a service companies used to use seat-based pricing. But
with AI agents, that's largely disappearing, hence why so many are overhauling their entire
business model. We're at a tipping point with AI right now. When reviewing spending data from
over 50,000 US companies, RAMP, a US corporate spending management platform, found a deep-seek
that Chinese AI is rising fast. Past regulars on this list have been Silicon Valley favorites,
like Figma and Fireworks AI. This is the first time a Chinese model company has topped it.
DeepSeek's V4 Pro model will cost $3.48 for 1 million tokens of output. For comparison,
OpenAI and Anthropic charge $30 and $25 respectively for the same amount of work.
Another Chinese AI, Kimi, costs $4 per million tokens. Note that this is just price, not quality
of model output. But not only are token prices from Chinese AI cheaper, but their models are
surpassing US ones in downloads. Does this mean total use? I don't think so. Anthropic and AI
are still leading corporate adoption at 34.4% and 32.3%. But DeepSeek is securing a place as
the cost-effective alternative. I do think we're moving from one stage of AI to another, even if
it takes a while. Companies are becoming more critical of their AI spend and how it translates
into output. In a lot of cases, it's just the same results, but with more cost. Once again,
humans are underrated. Efficient AI can learn from human cognition's adaptive resource allocation,
knowing when to engage deep processing, not just how to process deeply. Unlike AI,
human workers can actually be held accountable for their mistakes.
What I find particularly funny is actually Microsoft pivoting back to Copilot. They've
had nothing but problems with this thing and managed to annoy seemingly all 365 customers
and Windows users. Click here to learn the rest of that story, but until then,
I'm Hari and I'll see you guys on the next one.