Cerebras fell as much as 47% from its first-day close after SemiAnalysis reported that OpenAI's new Ultrafast tier was running on Nvidia chips rather than Cerebras silicon. Then Sam Altman called Cerebras a close partner and the stock jumped 9%.
The explanation turned out to be supply. OpenAI holds Cerebras's available capacity, and SemiAnalysis reported that Jane Street had taken the fastest slice of it for high-frequency trading, which is why GPT-6.1 Astra's Ultrafast tier reached for Nvidia instead.
Timestamps 0:00 The Firm That Bought Everything 1:56 A Bus And A Sports Car 3:34 The Post That Crashed It 7:29 One Tweet Changed It 8:59 The Megawatt Math 10:21 Why Milliseconds Cost Millions 13:13 Memory On The Chip 16:13 Watch How Fast 18:23 Not Just Traders 21:17 Every Fortune 500 Is Watching 22:35 The Jarvis Problem
AI Generated·May contain errors
Video Summary
AI Generated
This video analyzes the high-stakes market for fast AI inference, focusing on Cerebras and its specialized hardware. It explores the technical advantages of Cerebras over Nvidia, the financial motivations of high-frequency trading firms like Jane Street, and how ultra-fast inference is transforming enterprise R&D and the future of real-time AI interaction.
There's a trading firm on Wall Street currently paying hundreds of millions of dollars to get
access to this very special type of AI chip. The company that created this AI chip is called
Cerebris, and they just had one of the largest AI IPOs earlier this year, raising about $6 billion
in a single day. OpenAI committed to $20 billion worth of purchases over the next couple of years.
So the question then is, why is the stock down 50% since they IPO'd? Well, last week,
a rumor broke that OpenAI had stopped using Cerebris chips for their new ultra-premium
fast mode, instead opting for Nvidia chips, and the stock went down another 20%. It honestly looked
like things were completely over for the company. So I decided to dig into all of this myself,
and I realized that the market opportunity for faster, quicker AI models is absolutely huge.
In fact, Jane Street themselves are making $200 million of revenue per megawatt of Cerebris chip
compute that they use every single year, which is absolutely insane. But it's not just Wall Street.
There is a very good reason why trading firms as well as any other Fortune 500 company will want
to get their hands on chips that deliver faster inference. This isn't an episode about Cerebris.
I don't own any Cerebris myself, but it's an episode about why faster inference is a very
lucrative market for AI, and we'll use Cerebris as an example. All of that and more on today's
episode of In The Loop, brought to you by Qualcomm. Now, before we get into it, some quick context
on what Cerebris is. They're basically a company that creates AI chips the size of a dinner plate.
I'm not even kidding. I've held one of these things before, and it's just like, it is cute.
It looks ridiculous, and it's composed of millions and millions of these tiny transistors which allow
AI models to run and function. Now, the best comparative to this is an Nvidia GPU. We've all
heard of Nvidia. We've all kind of seen these big large data centers, but a single GPU is kind of of
a similar size, and you kind of stack them together. Now, the best way to compare a Cerebris AI chip to
a standard Nvidia GPU is an Nvidia GPU can kind of be thought of as a public transit bus. It is large.
It picks up a lot of people. It is cheap. It's reliable, but there are a lot of stops in between
point A and destination B. Its whole focus is around transporting as many people as they can
from point A to point B reliably and cheaply, and that's good. When you compare it to a Cerebris AI
chip, though, that's more of like a vintage, really fast sports car. It's expensive. It could probably
seat maximum one or two people, but it gets you from point A to point B really quickly, but it's
really quite costly. So you might then wonder, okay, which business or why would I use one over
the other? Well, Nvidia has typically been the reliable one. They make up around 80 to 85% of
the entire chip market. Nvidia and Jensen Huang has had a stronghold on this entire market for a
while, well before AI was even a thing, but the need for faster inference or rather AI models
responding to you quicker can apply to many very specific use cases. The most obvious one being
financial trading, for example. If you are Jane Street or if you are a competitor to Jane Street,
you want to get access to information that could inform particular trades that you might make
as quickly as you can. And so if you could build at the infrastructure level a model that basically
delivers you that answer a couple of milliseconds or even seconds faster than your competitor,
you can make a trade that puts you at an advantage versus another firm. Now, this is very
focused, but you could imagine applying this to anything and everything that AI touches.
Okay, so let's get into the drama of last week and what ended up becoming
an absolute chaotic nightmare for the Cerebro stock. There was this article posted by Semi
Analysis. Now, Semi Analysis is a very reputable research firm and they have this kind of paywalled
set of content. They dive really deep into semiconductors and chips analysis. One of their
subscriptions costs thousands and thousands of dollars every single year. Nvidia pays them a lot
of money to get access to that information and they use it to inform the decisions of the chips
that they make themselves. So, very reputable firm. Now, they posted what you're seeing in
front of your screen right now last week after opening AI's dev day. Now, they had a dev day
where they announced a series of different products, 20 plus products. We have an episode
on that. Definitely go check that out. But one of their announcements was about this new mode
called ultra fast or ultra mode. I can't remember the name. But the idea with this is you pay $500
a month, which is an extortionate amount of money, and you can get access to GPT's latest models
at a rapid speed, which means that as soon as you click enter, you get the result in front of you.
Now, that may not be interesting to you if you are just typing in casual prompts and trying to get
an answer like you would a Google search. But imagine if you're a software engineer. Imagine
if you're a financial trader. Imagine if you're using this to build a completely different business
that is meant to compete in the wide world out there. You could tap enter and end up with a
finished product in milliseconds that allows you to iterate and build quicker. So there's an
advantage to that. So people were really excited about this, and they assumed that OpenAI was using
cerebrus chips under the hood. There's a few reasons why they assumed that. About six months
ago, they announced their first version of the cerebrus powered model. I think it was GPT 5.6.
I can't remember, but it spat out 750 tokens per second, which for context is absolutely insane.
You could create a fully produced website in a fraction of a second just by tapping enter. It
would just appear on the screen instantly like magic in front of you. So on Dev Day last week,
they released a new version of this, which applied to GPT 6.1 Astra, which is their brand new model.
So everyone was like, okay, this is crazy. This is amazing. Cerebrus obviously powered this,
except that it was revealed by this post from Seminary Analysis that OpenAI was not using
cerebrus chips. They'd opted for NVIDIA chips. Now, you have to understand, if you are an AI
chip company, especially as small as Cerebrus compared to the big giant that is NVIDIA,
that is worth around $5.7 trillion, one of the largest, most valuable companies out there today,
you need to have a unique selling proposition. And Cerebrus' entire vision was,
we won't build better general purpose chips than NVIDIA, but we will build faster chips,
the fastest chip. It might be expensive, but it will be one that you would use for your high
value work. The fact that OpenAI is using NVIDIA was extremely concerning, which resulted in
a crazy stock launch. Let me show you this chart. Okay, fine. It's up 10% today, and I'll explain
about that later. But if you look at the last month or even the last six months, it is down
50%. It is so painful. And so we saw a huge dump last week, which I think took it to around $150,
which was crushing for a lot of people. And also it was honestly pretty bad timing because
their first major unlock for people who had invested in the company a lot earlier on
also unlocked. And it was revealed that not only had the CEO and CFO sold a bunch of stock,
but the COO right before this news was released had sold $80 million worth of the stock. So the
public reacted pretty understandably to all of this news, and they thought, okay, well,
Cerebrus is done. They don't have a future. Remember, at the end of their IPO day, the value
of the stock was $330, and it was then sitting close to $150 on, I think, Wednesday or Thursday
of last week. But then things changed with a single tweet from Sam Altman, CEO of OpenAI. He
goes, listen, there's some speculation about our partnership with Cerebrus. Cerebrus is a
close partner, and we have a deep engagement pushing on the frontiers of speed. And it all
started to make sense from there because when you actually read the article from Semi Analysis,
which the entire market reacted to by dumping Cerebrus stock, they actually don't call out
any other model bar GPT 6.1. The point being that most of the other fast modes that are running on
all of OpenAI's more recent and older models were still using Cerebrus chips. So there's obviously
a big amount of demand there. But the question then becomes, why not use it for GPT 6.1? Well,
the answer was revealed not by Sam Altman or OpenAI or any other AI lab, but by a trading firm
on Wall Street that goes by the name of Jane Street. You see, Jane Street is one of the most
successful quant trading firms out there. In fact, they're so successful that they don't even take
external capital. It's run by the partners and their own capital that they've made because
they've made an entirely inordinate amount of money. And they are active in many different
types of trading strategies, whether it's directly on stocks in the public market or whether it's
in market making itself, helping people buy and sell specific public equities. And it was revealed
that the reason why OpenAI could not use Cerebrus chips to serve their new ultra fast mode for their
latest, fanciest model is because Jane Street had spent hundreds of millions of dollars to secure
the compute before they could use it. Now, it's important to understand the kind of like topology
of how this works. Cerebrus has committed to $20 billion worth of sales to OpenAI. That's the
equivalent of around 750 megawatts. Now, what Cerebrus actually has live right now is 600
megawatts. And they're aiming to have all of that live by the end of this year, 2026. But they've
committed all of that compute to OpenAI. So for better or worse, OpenAI has the market dominance
on all of Cerebrus's available chip capacity. So then why is this random trading firm or not
random, but why is this very successful trading firm on Wall Street using up all the compute when
OpenAI could just be selling it to retail users? Well, the answer is pretty obvious. Jane Street
is paying an extremely high premium. Now, I don't exactly know how much they're paying them. But
if you do the math on OpenAI's $20 billion spend, you'll realize that they're paying around $26
million per megawatt. So 750 megawatts per megawatt, they're paying $26 million to Cerebrus.
So presumably, they're making some form of a profit, a high premium if they're selling it
to Jane Street. So let's just hypothesize here that it is $50 million being paid by Jane Street
to get access to a megawatt that OpenAI already has access to. So Jane Street buys directly from
OpenAI to get access to this super fast inference. So the question then becomes, why on earth is Jane
Street even paying this amount of money? Don't they already make enough money? Well, the answer
is very simple. In the same way that quantitative trading firms would pay so much money to get
access to under the sea internet cables to get information milliseconds earlier than their
opponents so that they can execute a trade seconds before they do so that they can make a bunch of
money on arbitrage trades or whatever that might be. That's exactly what Jane Street is trying to
with AI. It's that it's applied to anything and everything around their trading algorithm.
Typically, in the past, quant trading would look very much like, it's quite binary. It's like,
okay, I want to set this program to do A, B, or C. It's very deterministic. In AI, it's very
probabilistic, which means that you can apply it to anything and everything, which means that there's
a wider scope of competition. Jane Street wants to be ahead of the game here. And so they paid
for access to the fastest AI chips in the world so that they can make the quickest trades using AI
powered models. Now, presumably they're using GPT 6.1 Astra, but quicker than any of their
components. And they bought up all the available supply, which meant that open AI had the least
amount of supply to serve their public retail audience, which makes a lot of sense because
they're charging $500 a month. I can only imagine that for the opportunity cost of charging your
loyal customers that have been, they have a billion weekly active users at this point,
to introduce a new tier and say, listen, it's not just double the most expensive
subscription that we previously had. It is 150% more than the most expensive subscription that
we used to have. The trade-off must be huge. Jane Street must be paying them a lot of money.
And it was revealed that Jane Street is actually making around $200 million per megawatt of compute
annually using Cerebrus's chips. Let me reframe that again. Of the 750 megawatts that is available
to open AI, they are selling the bulk of that, let's say a couple hundred megawatts to Jane
Street. Jane Street is likely paying $50 million, maybe more per megawatt, which is already billions
and billions of dollars worth of cash and expenditure to get access to super fast inference.
But for each megawatt of compute that Jane Street pays for, they make $200 million of trading
revenue per year, which means that they're probably making billions and billions of
dollars just by getting access to Cerebrus compute. And that's the misinformation that's
been going on around it. I've seen some people say, listen, I think Jane Street is paying $200
million per megawatt. I don't think that's what's actually implied by this seminal analysis article
or what's being implied at all. It's that they're making $200 million worth of trading revenue
by using Cerebrus chips, which opens up my mind to so many different ways that this chip could
potentially be used for. It's not just about Cerebrus. It could be a completely different
company for all I care that is designing these specific chips. But the question on my mind is,
how does this chip work and why is it so useful to AI companies or companies in general?
Well, in order to understand why these particular chips are useful, you have to understand how the
AI models work themselves. Now, when you or I interact with an AI model, you type in a request
or prompt, you tap enter, and you get a response. And it's in words and sentences. Now, before an
AI model can generate a word in response to whatever your query is, it needs to run through
its entire set of model weights. Model weights are basically its blueprint, its DNA, the thing that
gives it a basic foundation of knowledge. It's kind of like our brain, basically, in order to
spit out a word. The equivalent is, imagine before you spoke every single word to your partner or
your friend, you needed to actively think through every single neuron that lives in your brain. We're
talking about billions and billions of neurons. It's just unfeasible. But that's how computers
or AI models effectively work. Now, typically, with an NVIDIA chip, the model weights don't
actually live on the chip. They don't live on the GPU. They live in a separate data server or
component that houses memory. That's where all the high bandwidth memory and memory chips that
are also skyrocketing in price live. So it's external. So the point I'm making is, if you send
your prompt to a GPU, it then needs to go to the memory file to retrieve and run through the entire
model weights, which is where it's stored, and then come back to the GPU, process, create the word,
give you a word, come back and run the same loop over and over again. It's that public bus analogy
that I gave you earlier. It's slow, multiple stops, but it is somewhat efficient when it comes to
generating or using compute. It's less expensive than Cerebrus. Now, when you look at a Cerebrus
chip, they combine the memory with the GPU. So it's just one thing that you're looking at. And so
the distance or the amount of time that is spent from sending a prompt to a model that runs on a
Cerebrus chip is much, much shorter. It's more expensive, but it's much shorter and it's much
quicker, which means that you spit out a response immediately and quickly. Now Cerebrus chips have
about 44 gigabytes worth of memory capacity on their chip. And if you think about it,
modern models today are worth terabytes worth of data, especially if you look at their model weight.
So you still need multiple of these Cerebrus chips to make it work. But if you're willing
to pay the upfront cost, you can serve so many people so quickly, which in certain cases of
And so that's Cerebrus' whole unique selling point. It's we sell faster inference. But then it
got me thinking, okay, whether this is Cerebrus or any other company, what's the actual market
opportunity for fast inference? Like, is it just trading algo firms like Jane Street or are there
other companies that would feasibly use a chip like this or pay premium to get access to a chip
so that they can beat their competitors and make a ton of money? Well, the answer is yes. And you're
going to want to hear about this, all of this and more on today's episode of In The Loop brought to
you by Qualcomm. Now, before I explain which industries are using super fast inference chips,
I think it's important to actually show you how quick these chips are. I've got this fun little
demo that I generated using Claude to figure out how this works or how this looks for you. So
what you're looking at on your screen is the standard response delivery that you would get
if you were to type an average query into chat GPT or any AI model, right? So it looks like this.
This is how quickly it delivers an answer. It's pretty quick, right? So for the average person,
you know, it's fine. You don't need access to a much faster inference than that. But then you
kind of shift to businesses. So let's take it up a rung. This is NVIDIA's ultra fast. And if you
blinked, you might have missed that demo. Let me run it again. Super quick, right? So it just kind
of like you blink and your answer's already there, right? It's less useful for if you're an average
person and you want to get a result quicker. Sure, fine. Who cares? I just wanted a Google query. I
just wanted to figure out what this recipe was. So I don't need it super fast. You can just give
it to me a little so I don't want to pay 500 bucks a month for this. That's completely fine.
But if you are Jane Street, information, milliseconds, seconds matter down to a T.
So this is what Cerebrus ultra fast 750 tokens a second looks like. Did you catch that? Let's
try again. And again. And again. Literally, I'm looking at this and I'm blinking and I'm still
missing it. It's crazy. It's as if it just materializes or just appears in front of your
screen. Now, it took point two seconds to appear right in front of us. And so the target that
Cerebrus is going for is 5000 tokens a second, which looks something like this. You actually
can't see it because it's I don't think this is accurate, but it says 0.0 seconds,
which probably for the average human being, you can't necessarily pick up. The idea is
they're not serving retail customers. The whole point of these chips isn't to serve retail
customers. It's to serve businesses where time matters, where it could cost you tens to hundreds
to billions of dollars. And it's not just trading firms that this applies to. There are many more.
Well, let's start with agents or coding in general. Now, it's no mystery at this point that
AI coding has become a massive industry since code code launched at the end of last year. It's
crazy to think that that was less than a year ago. It has seen an absolute meteoric rise,
such to the extent that software engineers of varying capabilities, L4 to L6 to L7 staff
engineers are all using AI coding models pretty aggressively, intuitively in their day to day
workflow. Now, you might wonder, OK, well, it's important that they should code. I get
that it's important that they should build products faster. But why do they need the
fastest chip? Well, imagine that you are Fortune 500 company and you are in a very competitive
field with other companies breathing down your neck. Product iteration, product development,
product testing. Let's say you have a million different ideas becomes extremely competitive.
The cost of spending two months and tens of millions of dollars testing out one idea only
to be wrong could cost you the race against your major competitor. So you need to figure out a few
things. How can I reduce that cost dramatically? And how can I speed up my testing or iteration
of different products such that I can quickly figure out what works, what doesn't work,
and build and double down on that thing quicker than my opponent or my competitor?
This applies to pretty much every single Fortune 500 company that is looking at this AI thing,
trying to figure out how on earth do I use this to my advantage? And the people that work quickest,
and I use that adjective very specifically, will end up winning or pulling ahead vastly further
than any of their competitors. So coding as a general practice applies really well to
Cerberus chips or faster inference chips in general. In fact, if you look to the right,
you'll see partnerships or customers that Cerberus already counts on their backlog,
which is, by the way, worth around $25 billion. I looked at their Q2 earnings report from a few
months ago, and I realized that whilst they're making around 500 and I think $14 million per
year right now, their backlog over the next couple of years is $25 billion. Now, of course,
a huge chunk of that comes from OpenAI, but they're starting to build a roster of independent
customers. Now, some of them are counted on the screen that I'm showing on the right over here,
which includes IBM, Mistral, Notion, Mayo Clinic, including Lovable and Cognition.
Now, Lovable and Cognition is really interesting because they are AI coding platforms. They have
their own AI agents. So it's interesting that they want to plug into the fastest infrastructure
to beat each other, which is pretty ironic. And then you have IBM, which is more traditional
enterprise exposure. And then you have Mistral, which is another AI lab themselves. And then
Notion, which is in work productivity or Mayo Clinic, which is focused on scientific research.
So you start to understand, oh, maybe it's not quantitative financial research firms or
financial trading firms that want to use cerebrus chips. It's anything and everything that touches
a computer. And so I guess what I'm trying to communicate here is we are entering a very weird
world where the olden days of fast information gets delivered and results in a multi-million
dollar output or result isn't just reserved for finance people or trading firms alike.
It is now reserved for anyone and everyone that can get access to a computer that needs to automate
specific types of work, that needs to iterate and build products, especially if you're a company
that's building software. And if you're a major multi-corporation where spending tens of millions
of dollars is counted as R&D to try and make a better product. Imagine if you spent 50 million
dollars and you ended up creating a product that earned you hundreds of millions of dollars to
billions of dollars over the next couple of years. That is a capex cost that is worth spending. So
every company in the Fortune 500 is stepping back. They're looking at this example of
Jane Street making 200 million dollars per megawatt per year. And they're thinking,
I wonder what this could apply to for ourselves. How can we beat our competitors? What are our
competitors doing with potential access to these chips? So that's why Cerebrus is out of supply
right now. And why I think OpenAI is charging an extortionate amount of money to sell to retail.
I don't think they need to be doing that. I think they're just doing it as a favor.
Now, if we step back for a bit, the final point I'm going to make here is if you're listening to
this and you're like, okay, cool. I get the whole kind of thesis, but I don't understand how this
applies to me personally. I don't have a great answer for you. But what I would encourage you
to think of is Tony Stark from Iron Man when he's interacting with Jarvis. He speaks to this AI. For
those of you who haven't watched the film, also go watch the film. He's speaking to this AI.
Jarvis is speaking back to him in a very human sounding voice. It has emotion. It has intent.
And whatever Tony Stark asks to be created magically appears to him in milliseconds
right in front of him. In holograms, in software packages, in software updates to his Iron Man
suit, it just appears to him magically. That's the world that we're going to enter. And it's going
to change the types of ways that we interact with the devices around us, whether it's mobile phones
or whether it's the laptop that's sitting in front of me. Instead of navigating to a website address
or downloading an app and scrolling through it, it will just materialize in front of you. But it's
not just specific apps. It'll be a specifically tailored app to you. And it may not exist for
anyone else in the world. Imagine this being applied to Hollywood or films or movies or videos
where the content is still the same, but it's tailored specifically around you, materializing
instantaneously, adapting to how your facial expressions react. It is a very weird kind of
Blade Runner-esque type future. I hope it's not as dark as that. But it's very interesting to
explore. And I don't quite know how that's going to map out. But yeah, that's the episode. I just
wanted to kind of talk about inference because this entire thing kind of unraveled with the
thing. And I realized, wait, there's actually a much more important story to talk about here. But
that is it. Thank you so much for listening. If you enjoyed this and you want to subscribe,
please, please subscribe. I think this is, you know, I'm doing my best to try and break down
some of the most complex topics and give you a bit of value beneath just the headlines. Hopefully,
you found this useful. If you didn't, also let me know in the comments. I want to hear from you
guys. But if you enjoyed it, and you think of a friend that might want to listen to this, please
copy the link and share it with them. It helps us out massively. Yeah, I guess that's it. And I'll
see you on tomorrow's episode. Thank you so much for listening on today's episode of In The Loop
brought to you by Qualcomm.