When Aravind Srinivas arrived at UC Berkeley in 2020 hoping to join Pieter Abbeel’s robotics lab, he encountered an immediate obstacle: The lab was already full.
Undeterred, Srinivas proposed that he and Abeel work together for six months so he could prove himself. What followed was such a prolific stretch of research that Abbeel found himself repeatedly asking about his new student’s well-being.
“Honestly, the only big advice I ever had to give Aravind was the question, ‘Did you sleep?’” Abbeel recalls. He says Srinivas replied, “I cannot afford to sleep until I’m officially in your lab.” After joining Abbeel’s lab and being persuaded to rest, Srinivas completed what Abbeel calls “a really amazing Ph.D.,” went on to work at OpenAI and later founded Perplexity, the AI company built around answering questions with source-grounded responses.
In this Berkeley Talks episode, Srinivas’ determination is one of several stories Abbeel, a professor of electrical engineering and computer sciences at Berkeley, shares about former students and postdoctoral researchers who went on to help found AI and robotics companies, contribute to influential research advances and, in one case, become a NASA astronaut.
That astronaut, Woody Hoburg, came to Berkeley from MIT in 2011. At the time, he told Abeel that he knew the likelihood of becoming an astronaut was “effectively zero,” but was determined to try: “If I can make it from 0.1 to 1 by giving it my all, that’s what I’m going to do.” Hoburg went on to earn a Ph.D. in electrical engineering and computer sciences at Berkeley, join NASA’s 2017 astronaut class and, in 2023, serve as the pilot of NASA’s SpaceX Crew-6 mission aboard the Crew Dragon spacecraft, spending 186 days in space.
For Abbeel, the common thread among his former students is not only brilliance but also generosity. “All of them (are) super nice,” he says. “And what I mean with that is it’s easy to be nice to your boss, you have to be. But they’re super nice to all their peers, everybody around them, new talent that wants to join. I’m really proud.”
Abbeel’s lecture took place on April 17, 2026, as part of Berkeley’s “Brilliance of Berkeley” speaker series.
Professor Peter Abbeel discusses the evolution of generative AI, from the technical shift toward reinforcement learning (RL) and post-training to the breakthroughs in diffusion models and robotics. He emphasizes the intersection of technical conviction, academic excellence, and the strategic pursuit of \"the right problem\" in both research and entrepreneurship.
Welcome back, everyone.
Please join me for this lecture is gonna be given
by Professor Peter Abbeel, who's gonna share anecdotes
about some of his most successful students.
A group of incredible Berkeley alums
that includes co-founders of over a dozen AI companies
and a NASA astronaut.
He is one of the world's leading researchers,
educators, and entrepreneurs in artificial intelligence.
And as a professor in artificial intelligence
and robotics here at Berkeley,
he researchers generative AI,
reinforcement learning, and robotics.
His pioneering AI contributions with his students
include diffusion models, large world models,
Unisim, TRPO, SAC, ring attention,
MAML, hindsight experience replay,
domain randomization, decision transformer,
LLM, a zero shot planners for robotics, and RFM1.
His awards and honors include
the Presidential Young Investigator Award,
an NSF Career Award, an ONR Young Investor Award,
a Guest Investigator Award,
the DARPA Young Investigator Award,
TR35, IEEE Fellow, and the most prestigious one of the lot,
the ACM Prize in Computing.
So please join me in welcoming Professor Peter Abbeo
to the brilliance of Berkeley.
Thanks for coming out, everyone.
Pleasure to be talking with you.
I've actually never had the pleasure to talk
after somebody who talks about love.
I have no expertise in love,
but now I feel like it's my opportunity
to share my non-expertise in love for just one minute.
If I reflect, just a reflection, no professorial expertise,
however, I reflect on my love life,
I'll highlight two things.
One is, even though it's kind of unavoidable
when you're broken up with that you're sad for a bit,
the time you spend being sad
and obsessing still over that person, you never get back.
So keep it short.
And the other one is,
I think most of you probably don't have kids.
So right now, love is this thing
you think about like your partner, that's love.
But once you have kids,
you experience a whole different level of love.
It's something completely different.
Keep that in mind.
There's something yet way beyond out there,
and I hope you all get to experience that also.
Okay, leaving that behind,
do have three kids, very lucky to have three kids.
I wanna talk about some success stories
of some of my students.
But since I and my students work in AI,
it's kind of an interleaved story
of what's happening in AI
and what some of my students have done
to make this happen in AI.
Let's see.
So I guess, Oliver, you did plenty long in intro,
so I'll keep this short.
I'm a professor at Berkeley,
I also spend time in industry and all kinds of things,
including help start Gradescope,
which some of you might use for some of your classes,
help start OpenAI, Covariant.
Recently, I'm at Amazon doing a lot of AI
and robotics work there.
Also did a lot of investing for a while,
still do some of it now, but a little less time for it.
I think for many of you, not necessarily all of you,
but many of you, probably the moment AI
became something that felt like you like it or not,
it's gonna be there for you, it's gonna be present,
is when ChatGPT came onto the scene.
Before that, AI place go, sure, cool,
but how does it affect your life?
Not really.
But AI starts having conversations,
starts taking exams really well,
like Wharton MBA exam, the bar exam,
which is something that lawyers study for
for a very, very long time,
and somehow scored in the top 10%.
And you can ask it things like,
hey, can you describe UC Berkeley in a poem,
where every line starts with the letter R?
And that doesn't exist on the internet,
that's not just like a Google search,
that's actually a creative effort that AI has to do
to do this for us, and sure enough,
resplendent jewel on the bay, Berkeley stands proud,
rigorous minds in pursuit of knowledge,
loudly avowed, and so forth.
Everything starts with letter R,
it rhymes, because it believes that that's the style
of poem we might be looking for,
and it actually is grounded in facts about Berkeley.
So this would take most people a good amount of time
to come up with, I'm not saying the best poets
in the world can't do this fast,
but many of us would take a long time,
now this tool is available to us,
we can prompt AI with the vision of what we want,
and it can fill in the details and generate it for us.
Can also do math for us, five X plus three equals 33,
X equals six, that's right,
just as good as the AI on this particular problem,
AI is still worse on many other things,
but here AI is pretty good,
and it even explains how it got there,
so when you're stuck on a homework problem
or something you're just studying on your own,
AI can not just give you an answer,
but also show you how it got there,
and that's nice, because it allows you to learn
beyond just seeing the answer.
With these capabilities,
a lot of money started pouring into AI,
in 2023, everybody was like, oh my god,
Microsoft 10 billion in OpenAI,
that's so much money to invest into a company,
and OpenAI worth 30 billion, that's kind of crazy,
but then OpenAI became worth 80 billion,
300 billion, 500 billion, 850 billion,
and this just keeps going up,
because more and more use cases surface,
and more and more use happens of AI
in all kinds of productive ways.
Other companies, Anthropic,
essentially a bunch of folks who were at OpenAI,
then left OpenAI to start their own company,
they started in 2020,
they raised at a 380 billion valuation,
just in February,
if you look in the betting markets valuation
of Anthropic right now, it crossed a trillion today,
so it's likely the next round will be at a trillion,
which is pretty wild,
between like a 3x in like two months,
when you're already worth that much,
it's pretty unheard of,
now of course, if you write software, you'll know why,
you use cloud code, you write software 10 times faster,
well, if every software engineer in the world
is 10 times more productive,
sure, companies are willing to pay a good amount of money
for that productivity tool,
to make their engineers more productive,
and the revenues go up quickly for Anthropic,
Elon Musk was actually one of the co-founders of OpenAI
back in the early days when I was there,
he would show up every other week,
trying to understand what's happening,
give some suggestions,
but he left at some point,
due to some disagreements with what OpenAI was planning to,
what he wanted to do,
and then he started a new AI company, XAI,
which was valued at 200 billion in his latest fundraiser,
and then he kind of said it's worth 250 billion
and merged it with SpaceX,
but this is the actual market valuation,
that was the latest,
and then Ilya Siskupin,
the chief scientist of OpenAI left about two years ago,
and a little after that, started a company,
and immediately raised that to 20 billion valuation,
so there's this strong belief,
at least with people with a lot of money,
that investing in AI is probably worth a lot of money
in the future,
because somehow it'll create a lot of value,
so you're willing to pay a lot already now
to get into these companies.
Now the most widely used system today,
generally the most widely used system is ChatGPT,
actually a quick question,
anybody here not using ChatGPT ever?
Never used it, do you use anything else instead?
Gemini, okay, equivalent,
that's kind of like the competitors
who popped up a little later to kind of imitate ChatGPT,
how does it work?
A bunch of words go on the input,
for example, what you type,
and then it generates a word on the output,
and it repeats that behind the scenes multiple times,
feeding this n plus first word back on the input
to go through that same neural net,
neural net, just think of it as something like the brain,
we don't know how the brain works,
but we know that it somehow takes inputs
and generates outputs,
and in between there's this massive amount of neurons
that is very flexible,
it can be adjusted as you grow up to internalize knowledge,
and we try to imitate that inside a computer,
and so it's learned from a ton of internet text books
and so forth that after a certain sequence of words,
another word is pretty likely,
and actually has a probability distribution over next words
and can generate, you know,
sample from that distribution
to generate a variety of responses as needed.
Now the question is how do you build it?
The way it started was what is now called pre-training,
you learn this next token slash next word prediction
on lots of text data,
essentially entire internet,
rough amount of training data used these days
is like probably like roughly 10 trillion tokens,
so it's more text than any human would ever read,
but of course not all the text is equal quality,
but it's still, either way it's a lot more
than any single person would ever read in their life,
it's used to learn to predict the next word
from all the previous words.
If you do that for long enough,
you get some kind of thing that can auto-complete,
let's say it was training a lot of emails,
you type email, be able to auto-complete your email
that you're typing.
What actually was found though is that
it didn't see a whole lot of use in that format,
this was in 2022,
people like type with it,
but it doesn't really respond in an interesting way.
The problem was that it had seen so much data
that was not exactly how you wanna have conversations,
it was other data, how books are written,
how internet pages are written,
and you would say like, how do I make a boiled egg?
And it would reply with, how do I make bacon?
How do I make hash browns?
And so forth, and you're just like, wait,
that's not the answer to my question,
but actually if you go on the internet,
you'd see that listed one after the other after the other,
and so you're like, okay, makes sense,
that's what it saw in sequence.
So something needed to change,
it had all this internal knowledge,
but it wasn't willing to expose it the right way.
Then John Shulman, Berkeley graduate,
shows picture here, one of my former students,
invented the second step, named post-training.
And the idea there was that,
instead of just launching the model as is,
you make it have a lot of conversation with humans,
where it has to generate multiple responses
to everything you tell it,
and then the person rates the responses.
And then it is trained to maximize its ratings,
something called reinforcement learning,
where you maximize your score,
in this case, the score is your ratings of your responses.
And so once you start doing that,
because it sees a lot of human ratings,
where natural responses are rated higher
than unnatural responses,
it starts generating those more often.
And remember, it's really a probabilistic system,
and so it's shifting its probability distribution
from kind of evenly over what all the text
on the internet looks like,
to the text that looks like conversational AI,
and now it starts doing that.
That's ChatGPT, that was launched in November 2022,
and immediately took off from there.
John's work here on step two is, of course,
what unlocked it, but not just that.
In fact, if you go back,
it's not just that you have to have a good technology,
you have to have the conviction to launch it,
and John also had the conviction,
even though a lot of people at the time would be saying,
oh, you know, who wants to chat with an AI?
Sure, sometimes it's funny.
Maybe we'll chat with it for five minutes
to get some jokes, and then stop chatting with it.
John said, no, no, no, no, no,
that's not the right way to think about it.
These things have the whole world of knowledge
that's written in text in them.
People are gonna wanna chat with them
to get information out.
In fact, the use case today
is almost never getting jokes out of the AI.
For example, 30% is healthcare cases
where you're wondering, like, I'm feeling this,
I'm feeling that, you know, what do you think it could be?
And it brings the entire internet to bear
to provide a response.
And John had the conviction, let's launch this as a product,
because until then, the OpenAI GPT models were just APIs.
It was just like an API for autocompletes
in various applications that people were building.
But he convinced, you know, he was convinced
and it made it happen to make it into a product.
By the way, at Berkeley, the same techniques
that were used for step two of chat GPT,
the same reinforcement learning techniques,
he developed them at Berkeley during his PhD
in the context of robots learning to walk.
And these robots, at the time,
this was the first time it was possible
to have a robot like this learn to walk on its own.
It shows something about the generality of these techniques.
Like the principle behind learning
to have a better conversation
is actually the same principle as the principle
in terms of learning to run better.
It's about optimizing a score
that you are given after every attempt.
Give a score, attempt multiple times,
compare the scores, compare the attempts,
and then optimize for a higher score.
And the beauty actually was that at the time,
like the mainstream in robot locomotion
was Boston Dynamics, like highly qualified engineers,
carefully engineering every aspect of their robot systems.
But then every new robot requires so much new work.
Here you have a new robot, sure it's in sim,
but the same has happened later in real.
You can run the same algorithm, no change,
and the new robot learns to run.
So it's pretty amazing how general these techniques are.
And in fact, the latest advances in LLMs
also use reinforcement learning.
So the next big wave, 22 was the chat GPT edition.
2024 was the reasoning edition
where models now can reason, think through,
multiple steps of kind of getting through,
trying to find a solution to a problem,
maybe backtracking if it doesn't look right,
try another path until they find a solution.
That's the reasoning.
OpenAI announced this in 24.
They said we use reinforcement learning,
nothing else, big, big headlines.
Chat GPT can now do so many things
like solve advanced math problems and so forth.
But nobody knew how they did it.
The big thing that happened a few months later,
OpenAI kept it secret, not telling anybody.
Little company out of China, DeepSeek,
released an open source reasoning model showing,
hey, our results match the OpenAI results
and we will write a paper
showing exactly how we did everything.
And how did they do it?
Well, for OpenAI, it was a lot of speculation,
but DeepSeek said, this is how you do it.
You essentially set up a specific conversation,
you tell the bot, this is the structure of the conversation
where you're an assistant and the user will provide a prompt
that will be inserted over here
and then you have to first think
and after you're done thinking, you provide the answer
and that's how you do things.
And then you run reinforcement learning
against the accuracy of the answer
and the formatting of the language.
You need to write in a reasonable language
and not just provide a number at the end that is correct.
So what does that mean for today's AI?
It's essentially done this way now.
First, you pre-train.
All the data you can get,
essentially that's reasonable quality,
next token, next word prediction.
Step two, post-training.
You learn to optimize human chat quality ratings.
That's ChatGPT 22.
And then since 24, there is more post-training.
Learn to optimize against verifiable rewards.
So have the bot try to solve a math problem.
It can think, then provide an answer.
If it's correct, that's a positive example.
That's a high score.
If it's incorrect, that's a negative example.
That's a low score.
And it tries to keep optimizing that score.
It allowed DeepMind and OpenAI to claim gold
in the IMO, International Math Olympiad.
Very hard math problems.
Very few people can solve these problems
yet somehow the bots can do it now.
Same for the main college programming level contest.
They won gold there also.
Revenue encoding has gone up like crazy.
Just a half year ago, it was big news
that Cursor hit one billion in ARR
in 24 months of existence.
But since then, Anthropic has just kind of
taken it through the roof,
recently hitting a 30 billion run rate
and expecting to hit a hundred billion dollars
in ARR by the end of the year,
which is kind of crazy to think that a company
can go from essentially no revenue
to a hundred billion in about two years
and might keep going up.
So pretty exciting and to me personally,
pretty excited how much a role reinforced learning
and John's work has played in enabling this.
Now one of the things that you might have noticed
with these models is sometimes they're incorrect.
So here's a great example from a court case actually.
So in this case, a passenger was chatting
with a chatbot from Air Canada and asking for a refund
and the chatbot said, sure enough, you can get a refund.
Little later, the passenger does not receive the refund.
They say, what's going on?
And the people working at the airline say,
well, actually the chatbot was wrong.
You should not be getting a refund.
They made a mistake.
The person takes it to court says,
well, I was told I was gonna get a refund
by the official communication channel from this airline
and the court said, yes, as an airline,
if you let chatbots have your conversations,
you're responsible for whatever they say
they will pay the passengers in refunds
and they were forced to pay.
I mean, for the airline, it's just, you know,
it's probably a couple hundred dollars
or something like that.
But the principle is very interesting here
that the court said, if you roll out a chatbot
and it promises things to people on behalf of your company,
you are responsible to honor that.
So very, very interesting precedent.
Now, let me tell a little about a company
that Arvind Srinivast started,
one of my other former students, called Perplexity.
Perplexity from day one was essentially a version
of chatGBT where you are always getting answers grounded
in context of the sources.
So it would be asking a question,
it would then say, let me go find sources to look at
and then instead of answering just from memory
like somebody might do, like chatGBT used to do,
you answer based on reasoning by looking at these sources,
looking at the question and reasoning through
how these sources are providing in some sense
the relevant content to answer the question
and then formulate an answer to the question.
Much more grounded that way, gives you more confidence
and it gives you the references.
If, for example, you say, hey, you know what,
this NerdWallet thing, I don't trust it,
answer again without relying on it,
you can just click it away, you get a new answer
that's just relying on, for example,
the Air Canada source in the front if that's what you want.
Tell me, I'll tell a little bit about,
actually, I forgot to tell a little bit about John.
John, very interesting case.
John Schulman, absolute genius,
he came to Berkeley, did his undergrad at Caltech,
came to Berkeley for a PhD.
Interestingly, he started working in my lab
as just a rotation student for three months.
At the end of the three months, I'm like,
my God, this guy is an absolute genius.
Never seen anything like it.
I feel so bad because he's not officially my student.
He's supposed to be in another program.
He's supposed to be in a neuroscience program
and I'm in computer science and I felt so guilty.
I'm like, I go to my colleagues in neuroscience,
I say, it seems like John wants to keep working with me.
I feel so bad because I'm like,
this must be like your best student possibly
and I think it's kind of pretty reflective on Berkeley.
They essentially said, no, if that's what the student
wants to do, it's what the student should do.
We're 100% supportive, they should pursue
what they want to do and we're very happy for you
and I'm like, okay, well, I'm definitely happy
and work with John from there onwards for PhD
and the first couple years at OpenAI,
which was really, really fun.
I'll also say one thing that's very interesting
about John that will stick with me.
The first time I took him to a lab meeting
of another professor, there's like some students
saying something and the professor's commenting
and John goes, after the professor says their thing,
John just goes, I think you're actually wrong.
I think you should do it the other way
and the professor's like, what just happened?
This John person, never seen him before,
he invited him to my lab meeting
because Peter said he could give interesting comments.
First thing he does is like telling me I'm wrong
and he's probably right but it still felt
very uncomfortable and I was like, yeah,
I'll tell John to phrase it a little differently next time.
Compliment sandwich these feedbacks.
So another interesting thing with John was like,
he essentially did a complete PhD in three years
and then he concluded that all this learning
from demonstrations which he had worked on,
which we all do, we learn from watching other people,
was never gonna be enough.
You needed reinforcement learning
but nobody could get reinforcement learning to work.
In fact, if you went to the machine learning AI conference,
they talk about reinforcement learning,
you're relegated to a small room
in the side of everything and just six people
and then the thousand plus people are somewhere else
and nobody comes and talk to you.
It's like, oh, these people are doing
reinforcement learning.
That stuff never works but John had the conviction
that the future of AI had to have reinforcement learning
and then just at some point needs to be figured out,
he might as well try to figure it out now
and spend all his time on that.
And he went from publishing like a paper every three months
in top conference venues to not publishing
for a whole year, cranking away at trying to find a way
to get reinforcement learning to work
and sure enough, after a year, he got it working,
he had a human learning to run
and everything fed into chat GPT and the current reasoning.
But it's a very interesting case
where somebody essentially just had conviction
and though most students like to write papers
every three months or four months,
he just was happy to not write any papers for a whole year
to then get to the result he was really hoping for.
Arvind, another interesting case,
he actually joined Berkeley, hoping to work with me
but my lab was pretty full.
I'm very happy he ended up working with me in the end.
Very, very thankful but I was like,
my lab is a little full, I don't know if it's gonna work.
Little out of time, he said, you know what,
I'm happy to take my chances.
I'm gonna come to Berkeley, let's work together,
see what you think after a half year.
And also the only big advice I ever had to give Arvind
was the question, did you sleep?
Like every time he showed up, he got so much work,
I'm like, Arvind, are you sure you're sleeping enough?
And he would say, I cannot afford to sleep
until I'm officially in your lab.
And I said, Arvind, don't think that way.
You really need to sleep, I would really appreciate
that you sleep while you do your work.
And he's finally started getting enough sleep
and did a really amazing PhDs, went to OpenAI,
then started Perplexity to ground answers in text.
Another interesting story, reflection,
you saw the DeepSeek story where actually right now,
the open source frontier of AI is largely in China.
DeepSeek, Quan are two of the top model providers
in open source, open weights really,
not fully open source, open weights.
There's nothing in the US, Misha saw the opportunity
for a postdoc of mine, he actually heard through the grapevine
that Jensen Wang from NVIDIA would love to see
a US strong effort on open source for AI.
So there was a counterpart of that here, why?
Typically when things are open source,
the whole ecosystem thrives more.
A lot more people can build on it.
And so the kind of AI industry could likely blossom more
on top of open source, even though maybe for a company
like OpenAI or Anthropic, giving everything away
is not very productive for their bottom line.
For the full industry, it's good to have strong
open source players.
So he actually, he went to talk with Jensen
and told him, I think I know how to do it.
And then sure enough, raised two billion in October
and are currently just a half year later,
looking to raise another 2.5 billion
and a 25 billion valuation, trying to establish
essentially the US counterpart of DeepSeek and Quen.
Misha is another interesting case of somebody
with very strong conviction.
Actually, he did his PhD at University of Chicago
in quantum physics, very strong PhD.
He was one of the top PhD students in the field.
He decided that actually he should instead
spend time on AI.
So he applied OpenAI.
I was not at OpenAI anymore at the time,
but still a lot of my friends there and they said,
hey, we have this applicant, Misha Laskin.
He has no background in AI.
We can't really incorporate him right now into OpenAI,
but maybe you wanna talk with him for,
maybe he wants to switch field in his postdoc at Berkeley.
I talked with him and I said, this is hard
because you have no background.
I don't know if you're gonna do well,
but clearly you're very smart.
Let's see what happens.
He said, no problem, I'll just volunteer.
And so he volunteered for two or three months
and after three months,
he was pretty much the most productive person in my lab.
I was like, okay, I cannot have the most productive person
in my lab be a volunteer.
I gotta hire him as a postdoc.
And then we wrote just a ton of amazing papers together.
Also really get great influence
on more junior students in the lab.
And then of course now started this company.
Another story I wanna highlight here
is one thing you've probably seen
is that AI can not just go text to text,
even though that tends to be the more like work
and use case that people use the most for productivity,
AI can also entertain in going from text to illustration.
For example, you can say, hey, a masterful oil painting
of Persian exotic cat discovering
their astounding crypto losses while checking their phone.
Like, okay, can somebody make a piece of art
that reflects this?
Well, surely that requires some creativity
and that's why it is an interesting prompt to use
because we wanna check, can the AI be creative
rather than just parrot things it's seen before
and sure enough, it can.
And this obviously has never happened in the real world,
but somehow the AI knows how to put these things together
into a new composition.
A Victorian man struggles with his addiction to TikTok.
The Victorian era in England is like 200-ish years ago.
There was obviously no TikTok at the time,
but it somehow still knows how to put this all together.
This man is holding a flask,
likely loves alcohol a little too much.
See the red cheeks, small eyes.
He's holding his phone and just staring at it
and that's kind of just all he's doing.
Then you can also do it for whole videos.
So this one here, the prompt is a man rides a horse,
which is on another horse.
It's even talking, but I don't know if that comes through.
An ostrich steals dad's hat and dad chases after it.
This is my favorite.
And so these things can now just be made
without having to actually put somebody there with a camera.
Google Nano Banana is also very interesting
because it's focused on how you can edit imagery
and combine different concepts together.
So it was a picture of a butterfly,
make it into a dress, what could it look like?
Picture of flowers, some functional boots,
but not the boots the way you want them to look.
And it can project for you what that would look like
if you combine things.
Same thing for rooms.
You can ask questions like,
maybe you're looking for a new apartment, it's empty.
You're trying to get a sense, what would it look like?
If you have pictures of your own furniture,
you could give that.
You can also just ask questions generically,
like put a bookshelf there and so forth
and see what everything would look like.
Makeup, I imagine some of you
sometimes spend a lot of time doing makeup
and deciding, did it look good or not look good?
Here, you can do it like,
at least give it a trial run very quickly
before having to put in any physical effort,
which might save you a bunch of time.
And the person who invented essentially
the key idea behind all of this
was a Berkeley undergrad, Berkeley PhD, Jonathan Ho.
And he did the work here at Berkeley,
it's called Diffusion Models.
And the key idea is that everything in AI these days
is there's an input, there's an output,
you get enough data of inputs and output pairs.
And at some point, it just knows.
For a new input, what should roughly the output be?
But for illustrations, it was proving harder.
And the reason it was harder is that
it's nothing is 100% deterministic in text,
but it's at least somewhat close to deterministic.
A lot of words are the natural follow
most of the previous words,
not a lot of options available.
But in illustrations, I mean,
think back to the example of the cat,
you could choose out of so many types of cats,
so many phones, so many background sceneries,
none of that is described.
So you have a very open-ended output here
that I need to generate, which is harder to represent,
and actually very hard to learn.
So the trick Jonathan invented
and made the mainstream practice
is to actually not just have the AI solve this problem,
but initially, when it's still warming up
and hasn't learned much yet,
you also take what you wanted to put on the output,
you put it on the input,
but you put a bunch of noise on it.
So it's kind of like a noisy, blurry version,
as well as a text describing what you want,
and all it has to do is denoise
the thing into the correct thing.
And now it's kind of already in the right vicinity
because it has a noisy version of what it needs to generate
rather than having a very open-ended problem.
If you do that, that's called diffusion,
things just kind of work.
And that's now widely used for image generation,
video generation, and in fact, AlphaFold3,
I'll say more about AlphaFold in a second,
is also diffusion, and most robotics uses diffusion.
AlphaFold is this thing where it was shown
that you can actually, from the sequence,
so the amino acid sequence that makes up a protein,
which is very cheap to measure,
you can actually compute the 3D structure with AI.
And the 3D structure is very difficult to measure,
very expensive.
Of course, people did this,
people paid the money many, many times
to measure 3D structures with matching sequences.
And then after they had a large enough data set,
AI could learn the pattern,
and for new sequences, generate a 3D.
And that is now also done with diffusion,
those competitions there,
and here you see kind of level of accuracy,
green is the x-ray crystallography measurement
called experimental result,
blue is the computational prediction,
you see they match very, very closely.
And AlphaFold is kind of the thing
that made it take off previous methods.
Well, kind of mediocre in performance,
AlphaFold kind of shot it up much closer
to 100% than anything before.
And actually, the creators of AlphaFold won Nobel Prize
not too long ago.
In fact, just a couple years after AlphaFold.
It's very rare to win a Nobel Prize
within a couple years of your invention,
but somehow they managed to do it
because for the longest time, biologists had said
the biggest unlock in biology would be
to be able to compute protein 3D structure
with a computer rather than having to measure it,
and they did it,
and so the Nobel Prize came very soon thereafter.
A lot of this is digital, then bio, how about robotics?
Can robots start doing things?
Turns out that a lot of my own time
is actually spent in robotics,
and a lot of my students are there.
I wanna share a couple stories of those students.
So I co-founded Covariant with Peter Chen, Rocky Duan,
and a year and a half ago, we moved to Amazon
as part of a deal we signed with Amazon.
Interesting stories there.
Peter and Rocky, both undergrads at Berkeley,
PhD students at Berkeley.
I'll start with Rocky, very interesting story.
Never seen anything like him.
It's kind of crazy.
He's a freshman at Berkeley, and it's like January,
and he applies for a research position.
And I look at his transcript.
I'm like, I'm not gonna work with a freshman.
It's too early.
They need to do other things, learn more, explore more.
But his transcript is like, he took six classes,
which is a lot for your first semester,
especially coming from abroad, and had five A pluses,
and then his English writing was a B,
but all his technical classes were A plus.
And so it's like, okay, well, let me see,
at least encourage him, rather than just ignore
his request to do research.
And I talk with him, and I say, look, Rocky,
it'd be better to take more classes,
and then maybe next year we'll do research together,
but I just wanna say I'm really impressed
with your record.
Let's reconnect next year.
And the screen is flickering a little bit.
I'm gonna make sure it's sufficiently powered.
Somewhere there.
You know, you're impressed with my work,
and he says, but I would really like
to do research this semester.
I'm like, okay, well, clearly he's gonna do research
with somebody else.
If I don't let him do research,
and then I'm never gonna get to do research with him,
so maybe I should do research with him anyway.
That's what's going on in my head.
And I'm like, okay, well, maybe we can do research,
but probably you should take less classes then,
because six classes is just not that compatible
with research, and I want you to do research seriously
if you're gonna do it.
He says, you know, what do you mean
with doing research seriously?
And I'm like, well, why are you avoiding
the question on classes?
He says, you know, well, let's say 20 hours a week
would be good.
He says, oh, oh, yeah, I can keep doing
my current classes then.
I say, well, what kind of classes are you taking?
Oh, I'm taking eight classes now.
I'm like, what, you're taking eight classes now?
I'm like, what classes are you taking?
He's like, oh, you know, it's okay.
I'm taking seven technical classes.
They don't take time, but I will admit,
I'm taking another English writing class.
It's really killing me.
I'm like, my God.
What's going on?
I'm like, he's definitely not gonna sustain that,
but I'm like, I should do research with him
because clearly he's one of a kind,
and I wanna work with him.
So we start doing research.
He's in my class, actually, 188.
We do the midterm, and he stands one standard deviation
above everybody else in the midterm.
I'm like, okay, this just doesn't seem right,
and sure enough, by the end of the semester,
he's A plus on everything, including English writing,
and he has done amazing research,
and so I'm like, this is like 2013 or 2014,
and still today, I'm working with Rocky.
I worked with him through his undergrad.
Worked with him through his PhD,
which he did in two and a half years.
Worked with him starting Covariant,
which took us six, seven years
to get to where we wanted to be,
and now we're working together at Amazon.
Absolutely amazing, and I would say
the most interesting thing about Rocky,
I think, India and his dad,
there's this moment where Covariant is still pretty small.
We're just 12 people, 10 folks from Berkeley
from my lab and two other people,
and we're going to retreat, and we go around,
and one of the new people, not from Berkeley,
who we didn't know, we just joined Covariant a month ago,
and we're asking, okay, what has mostly struck you
about Covariant?
What's been most good, or most bad, or whatever?
And he goes, what's mostly struck me is
the speed at which Rocky answers all my,
I would argue, very stupid questions,
and the patience with which he answers them,
and I think that's just so amazing.
It's like, he's somebody who can just do so much.
I mean, he types his code fast,
and back when people still typed code,
types his code fast, and I can type emails,
but he's ready to answer anybody's question
anywhere, anytime, just the sweetest person ever.
It's absolutely amazing.
Then Peter Chen and Rocky are good friends.
The story is Peter Chen, yeah, go ahead, perfect.
Peter Chen goes to a party at Berkeley
right when he arrives here as freshman.
He walks around at the party,
and he sees somebody at the party sitting on a bouncy ball
in the middle of the party with a laptop,
and somehow able to focus doing his homework,
and Peter's like, that's the guy I need to get to know.
He goes up to Rocky, and he's like,
gets to know him, and sure enough,
from then, they're best friends,
and they've done literally everything together.
Undergrad at Berkeley, they researched,
and OpenAI, Covariant, and so forth,
and so been working with Peter ever since.
Also, the funny thing is, Peter didn't wanna do AI.
So Rocky was like, I'm gonna do AI.
Peter's like, I'm not gonna do AI.
This is 2014.
He's like, AI doesn't work.
I'm do something else.
But Rocky's like, I'm gonna do AI.
So they were briefly on a split path, very, very briefly,
and then, after a couple months,
Rocky said, and that's how I really got to know Peter.
Rocky said, I have this friend, Peter Chen,
and he actually wants to do AI,
and he's not in the PG program.
He's not a student, but he's in industry now,
but he wants to do AI.
He wants to come volunteer in your lab.
I'm like, Rocky, do you like working with him?
Rocky's like, I love working with him.
Rocky, if you like to work with him, let's bring him in.
And sure enough, he came as a volunteer,
and it was another one of those scenarios
where he was one of the top two, three
most productive students in my lab,
but he was a volunteer.
Again, a broken system.
Quickly got him as a PhD student,
but I had to wait a whole cycle.
He actually started at OpenAI
even before he started his PhD.
Really amazing, super nice also.
It's been a real pleasure.
Those are the people I've worked
with the most in my career so far.
Two other people I've worked with a ton of ton
are Chelsea Finn and Sergey Levin.
Couple things I want to highlight here.
Again, people willing to make bets.
Sergey did his PhD at Stanford on graphics,
computer graphics.
He applied to my lab at Berkeley to switch to robotics,
saying, hey, I think robotics in the end
is gonna be bigger than graphics,
more important than graphics.
I want to make the switch,
and made the switch, of course, extremely successfully.
Was postdoc here, became professor here,
and started this company, Physical Intelligence,
that's doing amazingly well.
I'll show a video in a moment.
And Chelsea Finn, PhD student here at Berkeley,
undergrad at MIT.
One of the things that I want to highlight about Chelsea,
which I think a lot of the folks do,
but it's not always as clear cut,
and definitely I like to do it,
is she doesn't just work,
she makes her exercise regimen very high priority.
Chelsea would literally go swim every day,
like lap swimming, right?
She would go lap swimming every day,
unnegotiable.
We had a conference somewhere in some city
anywhere in the world,
major academic conference,
and Chelsea is like,
she found the pool that she could get access to
and was going swimming every day.
I think there's something to it, honestly.
I think good physical exercise,
better blood flow,
more oxygen to your brain,
I do think it really helps.
It can also help in other ways,
just like being fit is good in many, many ways,
but I think it actually also helps
in ultimate productivity.
Deepak, another student here at Berkeley.
The lesson learned from Deepak
in terms of what I experienced with him was,
I think the most unique thing about Deepak was,
he would do this amazing work.
And I'd be like,
my God, Deepak, that's so amazing.
And he wouldn't come to me
to hear that it was amazing.
He'd be like,
I know there are things,
he's like, Peter,
I know there are things you don't like about the work.
Let's just get to it.
Tell me what you think is still not good enough
about this work,
which in research essentially defines
the next directions you're gonna take on,
and then would push very hard
to get to the next phases.
In terms of what's possible,
here are a couple of videos.
This is in the hills behind Berkeley.
Humanoids walking around.
This here is some work we did
at Amazon recently.
Humanoid wants to climb on that table.
It's a little high,
so it puts the chair there,
gets on the chair,
gets on the table.
Should it jump?
I don't know.
Maybe it should.
And sure enough, there it goes.
Nice roll.
We decided to take it to another level of speed.
Got it to do a wall flip.
Pretty amazing.
An interesting thing with all of these
is that actually this is all trained in simulation.
Simulation environments are good enough today
that you do reinforcement learning in simulation
to train a neural network
that understands how to control a robot.
You train at very large scale,
and then you can make it do the same thing in real world.
In robotic manipulation,
it's a little bit different.
This is Chelsea and Sergei's startup Pi.
There, typically, you collect data in teleoperation,
so you make the robot do everything
under your direct supervision,
and then you take all that data,
you teach it to mimic what you did,
and then it can do the same things,
but on different items on its own.
So pretty amazing.
These robots can fold laundry now,
do all kinds of practical tasks in the home,
some cooking.
It's not all the way there,
or you probably have seen it in your home already,
but it's getting,
I would say robotics is in a phase
where it's starting to show the signs
that the methods we use today,
with a couple more years of work,
can likely solve a lot of the problems
we've thought about for decades in robotics.
Who here has ever used Gradescope for any of their classes?
So, well, almost all of you.
Arjun, Sergei, and Ibrahim built it.
I did some of the product vision with them,
but they built it all, ran the company.
This started, actually, in a very interesting way.
Arjun was an undergrad here,
loved being a TA, loved helping students,
hated grading, felt like the grading part
was not the thing a TA should be spending that much time on
because it's not fun.
Helping a student, that's fun.
Grading helps students in some way,
but most grading is more like getting through it
in a way that you get a grade back
rather than really teaching a student something.
He wanted to change that,
and we trial ran some of the ideas he had in my class,
and from there, built into a product
that is very widely used.
I think one of the things
that really stands out about Arjun
is that in my own experience working with Arjun,
I can tell you, as a professor,
sometimes you're just tired,
and you just want to take the shortcut to get things done,
and we have a TA like Arjun, he's like,
no, no, no, think about it again,
and that's not in the best interest of the whole class.
Let's do it this way.
I'm like, okay, you're right, Arjun.
We've already worked so crazy hard,
but let's work a little more
and make it even better for the class.
You're right.
Really amazing person.
Last student I want to highlight is Woody Holberg.
Again, very interesting story of inspiration.
So Woody is currently an astronaut.
He's stationed in Texas,
but when he came to Berkeley,
he had done his undergrad at MIT.
He came to the Berkeley visit days.
He's talking with me.
He says, hey, I want to do my PhD in your lab and so forth,
and I say, Woody, okay, sounds like a great fit.
He says, yes, it looks like a great fit
based on what I've done at MIT,
where you can see the research I did there,
but my real goal is not to keep doing research.
It's to become an astronaut.
I say, Woody, what's the probability of that happening?
He says, oh, probability is effectively zero,
but even though it's pretty much zero,
I still want to maximize it.
If I can make it from 0.1% to 1% by giving it my all,
that's what I'm gonna do.
That's my vision for what I want to achieve.
And I ask him, well, Woody, what does that mean
for your PhD and what you want to do in your career?
He said, look, research is definitely
my second most exciting thing,
and I'm gonna commit myself to that
because there's not that much you can do
to improve your chances,
but everything I can do to improve my chances,
I'm gonna do, and so he's like,
often astronauts are like professors
at top institutes in the aeronautics department.
He's like, if I can do a strong PhD,
maybe become professor at MIT Aeronautics Department,
maybe that improves my chances.
I'm like, sure, and that's pretty well aligned
with what everybody does for their PhD and tries for.
And the other thing he said, which I really stood out,
he said, you know what,
and sometimes I'm not gonna do research.
I'm like, oh, sometimes not.
My lab, I like it when people do research
pretty much all the time.
What's the point of doing other things?
He said, if ever I'm gonna be an astronaut,
I'm gonna be in situations that are very unpredictable,
and I need to react fast.
It might be a life or death situation
with a couple seconds to react,
and I need a way to simulate that on Earth ahead of time,
and he's like, okay, I'm gonna be a pilot.
I'll fly my own plane and make sure
that I'm very good at that
because maybe I'll be a pilot for a rocket later,
but also he said, I'm gonna go spend
the whole summer in Yosemite.
You've all been in Yosemite.
I hope.
If not, you should really go check it out,
but he wouldn't hike there.
He would actually be there, part of the rescue team,
and he would just sit there waiting for a call.
Essentially, we're doing research,
and then when a call comes in,
like all the other members of the team
would be hanging out, having fun,
like talking to each other.
He'd just be doing research.
The call comes in.
He closes his computer.
He runs out, jumps on a helicopter,
and he's the guy hanging off the helicopter,
grabbing somebody off the rock
who was stranded and couldn't make it to the top
and couldn't come back down safely,
and that's what he was doing all summer,
essentially rescuing people stuck in Yosemite,
and it's what, somehow, all of that combined,
it made it work.
Like three years ago, he was called upon
to become an astronaut, did the training.
Again, there, there's a lot of selection,
but he made it through,
and he actually spent six months on the ISS
from March 3rd, 2023 till September 3rd, 2023.
He was the pilot for the rocket that took him there,
so pretty amazing.
Very, very proud of him.
He's actually this year's
engineering commencement speaker,
so you have a chance to actually,
not just see a picture of him,
but actually hear him speak and see him.
Absolutely wonderful person.
Again, I think one of the things
that has stood out to me at Berkeley so far
in my career, and I hope it stays that way,
is all the people I'm highlighting here,
super accomplished, but super nice at the same time.
All of them super nice,
and what I mean with that is,
it's easy to be nice to your boss,
like you have to, but like super nice
to all their peers, everybody around them,
new talent that wants to join.
Not everybody's gonna be equally good,
but always super welcoming to everybody,
and Wood is another prime example of that.
Very, very proud of him.
Thank you.
Thank you so much.
That was amazing.
Thank you. Thank you, Oliver.
So we have time for one quick question.
I can also hang out.
I'll just say I can hang out a couple more minutes after.
If you can't get your question in,
I'll stay till 2.15 or something.
Thank you.
Hi, Professor.
Thank you so much for the presentation.
So your students, so looking across them,
what they did, what did they get right early,
before the PMF, before the funding,
and how did you choose the problem
that they wanted to solve?
Yeah, great question.
And I think that that's a very important question
for anybody starting a company, right?
What's the right way to do it?
I think there's something very interesting
to highlight here is I think traditionally,
the way startups were done was you target
a specific market customer, and from day one,
you're iterating with that customer
to best understand what they're looking for
and build your product around it.
That's still often the case,
but OpenAI kind of changed the playbook.
Because OpenAI, Sam Altman had enough money
at OpenAI to say we don't need to find product market fit.
We can just take a very big long-term direction,
namely if we build highly capable AI,
something will emerge.
And it happened to be ChachiBD.
It wasn't the goal to make ChachiBD.
It was never the goal.
It was just John, one of the many people at OpenAI said,
we should now do this.
It wasn't like a top-down, like this is our mission.
It's just like we do the best AI research,
something valuable will emerge.
And so a bunch of companies are founded that way now.
Skilled is an example.
Physical intelligence is an example.
Those two companies are found in a kind of OpenAI paradigm
of you take some of the world's best researcher
on a certain topic, and you assume
that if you make even more progress,
something will open up and investors
are willing to make the bet.
And you actually delay the product market thing
because you assume that today's technology
is not good enough.
If you try to do it today, you'll just be stuck,
and it will never be good enough for your product.
First do a couple years of research,
and then you might be in a place
where the product can be good enough,
and then you go start doing the product market fit cycle.
Because obviously OpenAI, ChachiBD,
there's a product market fit cycle
going on at all times also.
Perplexity was a little different.
There was like, let's look at limitations
in current products and take it to another level
at the product level.
If you look at perplexity, the grounded answers
was something that was missing from existing products.
Of course, it's been copied by others since,
but it's always still good.
If you're first on something,
you're often getting more adoption in that direction.
And since then, perplexity have, of course,
built computer agents that can do computer work for you.
But so, I would say you really need to know
which path you're on.
The first we need to do research,
or we go directly product market fit.
The other lesson learned, I would say,
from my own experience seeing so many students do startups
and being part of many startups along the way
is that there's a lot you need to do at a startup.
There's a lot of bits and pieces.
But the thing you need to avoid,
the biggest failure mode, I would argue,
is that you are, essentially,
there's like a highest order bit.
Are we actually working on the right thing?
And then there's like everything else.
Are we hiring the right people?
Are we ramping them up quickly?
Are we firing people who aren't performing well?
Are we being a friendly, productive work environment?
Are we marketing ourselves well?
Are we doing this and that and that and that?
But if you're not working on the right thing,
all of that, you can nail it everywhere else,
but it's not gonna matter.
And so, you get busy with everything else
because that's your day-to-day thoughts.
Like, today, I need to get all these things right.
But you need to frequently think back and like,
are we going in the right direction?
Your board is supposed to help you do that
every few months, of course,
but also, as a founder, you have to do that.
Because otherwise, you might feel like
you're firing on all cylinders,
but it's not on the right thing.
And you nail everything, but then the company as a whole
doesn't really move forward
the way you want to move forward.
So, please join me in thanking Professor Beale
one more time.
Thank you.