Sam Altman Pre-Training GPT-4.5 in 10 Minutes

Fountain Lights Career Hub
4 followers
Pre-Training GPT-4.5 in 10 Minutes
Conect With Economy Media Everywhere
#economymedia #openai #gpt #gpt4
AI Generated·May contain errors
Video Summary
AI GeneratedThis discussion features the team behind the development of GPT-4.5, detailing the immense technical and systemic challenges of training frontier AI models. The conversation covers the transition to massive compute clusters, the persistence of scaling laws, the gap in data efficiency compared to human learning, and the infrastructure hurdles encountered at extreme scales.
Okay, so usually when we do these it's to talk about a new product that we're about to launch,
but we're going to do something a little bit different today, which is to talk about the
research that went into a product. When we launched GPT-4.5 we thought people were going to like it,
we were very proud of the model, but people liked it much more than we thought. People
would say all kinds of things like, I never thought I was going to have this experience
talking to a model, it's so different than GPT-4, it's way better in these ways that are either
obvious or hard to explain or this or that, but there was like a lot of interest about what went
into making GPT-4.5. So today we have some of the key team that made GPT-4.5 and we're going
to talk about it. We're going to talk about sort of like what went into it, what we learned, and
just like what it takes to make a giant model like this. Yeah, so I think we started this project
basically two years ago or so, and we kind of knew that we had a big new cluster coming online,
and we kind of saw this on the horizon, and we started doing a bunch of work to
kind of convince ourselves of the features that we wanted to include in the run,
doing a lot of large de-risking runs, building out a very long plan for this,
and kind of across the full stack from systems, ML, everything. And yeah, it was a long story
of execution for de-risking it and kind of preparing for the run before the run itself,
which itself was a very large endeavor. Yeah, I think it's a process that starts at the inception
with the collaboration between the ML side and the systems side and goes all the way to the time
that we know what model precisely we want to train, and then it's starting the run process.
In itself, with the pace that we are working at and especially trying to make use of our
most recent compute that is made available to us, it becomes something that is difficult to
plan perfectly. So we almost always go into a launch with a lot of unresolved issues
and try to make forward progress throughout the run despite all the challenges.
Basically add more compute, resolve all the issues that we
probably might not have anticipated despite all the projections that we had,
both on the ML side and the systems side, and try to basically close the gap between what
we predicted should happen and what is happening. Why does going from 10,000 GPUs to 100,000 GPUs,
why does that make the problem much harder? A lot of issues. I do believe that issues that
you observe at the scale, if you have a very keen eye, you would observe them at a smaller scale.
It's not that they only manifest at a larger scale, but something that is a rare occurrence
becomes something that is catastrophic at the scale, especially if you haven't anticipated it
being problematic. What are some of the kinds of things that have become catastrophic?
I mean, among those things that I think is quite well known are issues with the infrastructure,
the failure rates that you observe, the variety of failures that you observe,
both in terms of the types of failures and also the count itself. So we get to observe something
that I'm sure the vendor hasn't observed because this is a large pool of samples,
and we get to observe the entirety of the statistical distribution of a large pool of
resources that we are executing on. The network fabric is always part of it, the individual
accelerators are part of it, but at the end of the day, this is the beauty of it, at the
same time that almost everything needs to work as expected for the result to hold, and the job
is to basically minimize that variance. Second part of the question, obviously it's really hard,
this is for all of you, really hard to do things at the edge of scale. So even going, as we go off
and do the next training run, even kind of crazier. But I've also noticed it gets much easier to go
do things that are now no longer frontier. So it took like hundreds of people, almost all of OpenAI's
effort to do GPT-4.5. If you guys could go pick whoever you wanted, what is the smallest team
from OpenAI that could go retrain GPT-4 from scratch today with everything we know and have
and all the systems work? I think to get to a GPT-4 level model, it's probably on the order of maybe
five to ten people. Yeah, we did it with that number of people with GPT-4. 4.5 was
different in the sense that a lot of work history, a lot more people come together, and it was a very
different effort than GPT-4. If you could have any one ML question answered before the next big run,
what would you most like to know? I think what algorithms we should employ for
limited data and certain domains is the main thing. It's a kind of big question, big answer.
And if you could have any change to current hardware, you could have like a new kind of
network invented or like a totally new chip architecture. What is the most, what is the
limiter on systems at this point? You don't get to say all I want.
This is a transport level network and transfer level change. It's just that
there are faults that could be worked around at a different level than the application level.
I would rather the transport, the network transport do its job and keep running and
give me the available bandwidth without me worrying about it. Is there anything promising on that front?
Yes, we can talk about it. Okay, well that's good at least. Two part one for you, Dan.
So on the data efficiency question, humans, for whatever other flaws we have about learning things,
we seem unbelievably data efficient. Yeah. How far away is our very best algorithm?
By language, astronomically far away. 100,000x? Something in that range.
It depends on whether you count every bit of pixel information on the optical nerve,
but we don't know algorithmically how to leverage that to be human level at text. So I
think algorithmically, we're quite far away in an apples to apples. And then part two is,
do you think with the direction of our current approach, we will get to human level
data efficiency or is that just not going to happen and it doesn't matter? Well, I think
for decades, deep learning has been about compute efficiency. And what's magical besides
the data and compute growth is that the algorithmic changes stack so well. You've got
different people in different parts of the world finding this little trick that makes it 10% better
and then 20% better and they just keep stacking. There just hasn't yet been that kind of
mobilization around data efficiency because it hasn't been worth it. Because when the data is
there and your compute limited, it's just not worth it. And so now we're entering a new stage
of AI research where we'll be stacking data efficiency, 10% here, 20% there. And I think
it would be a little foolish to make predictions about it hitting walls. I have no reason to
predict a wall, but the brain certainly operates on different algorithmic principles than
anything that's a small tweak around what we're doing. So we have to hedge a little bit there,
but I think there's a lot of reason for optimism. Will humanity ever do a 10 million GPU or greater
synchronous pre-training run? I don't know if it will exactly be a pre-training run,
but I think there will probably be some kind of training run. Oh, there will be 10 million GPU
training. Yeah, yeah. I don't know if it'll look, it'll probably look totally different than what
we're doing today, but there'll be something that is kind of in the spirit of unsupervised learning
that is like at that scale. I think so. To go off on a little bit of a tangent, do any of you
guys have like an intuition of, is it weird or is there anything to take away from the fact that
pre-training seems to be so general across everything? And when we teach a model reasoning,
we can get it so good only at one category of things? Yeah, I don't know if it's the most,
yeah, I think it's interesting. I think it's kind of not surprising to see this out of pre-training
when you're just looking at what you're training with. When we construct like a training dataset
for pre-training, it's inherently very broad where we're targeting breadth and diversity.
And I think it's kind of difficult to always get the same breadth when you talk about doing
RL and having like environments that you can kind of cleanly get good reward signal out of
and good environments out of. In some sense, this whole effort, which was hugely expensive in
terms of people and time and dollars and everything else, was an experiment to further validate that
the scaling laws keep going. Yeah. And why, and turns out they do, and they probably keep going
for a long time. I accept scaling laws like I accept quantum mechanics or something, but they
still don't, like I still don't know why, like why should that be a property of the universe?
So why are scaling laws a property of the universe? I can take a step. Well, the fact that more
compression will lead to more intelligence, that has this very strong philosophical grounding. So
the question is why does training bigger models for longer give you more compression? And there
are a lot of theories here. The one I like is that the relevant concepts are sort of
sparse in the data of the world, and particularly it's as a power law, so that the like the 100th
most important concept appears in one out of 100 of the documents or whatever. So there's long tails.
Does that mean that if we make a perfect data set and figure out very data efficient algorithms,
I mean can go home? It means that there's potentially exponential compute wins on the
table to be very sophisticated about your choice of data. But basically when you just scoop up data
passively, you're going to require 10xing your compute and your data to get the next constant
number of things in that tail. And there's just that tail keeps going. It's long. You can keep
mining it. Although as you alluded to, you can probably do a lot better.
