Meta's chief AI scientist says a provably safe AI is as impossible as a provably safe turbojet.
We build turbojets anyway. What replaces the proof is a system that cannot be jailbroken by construction, and the reason is not a safety patch.
Yann LeCun is Meta's chief AI scientist and a Turing Award winner. Please join my mailing list here 👉 https://briankeating.com/yt to win a meteorite 💥 We cover: why the search for provable AI safety is the wrong goal, what an objective-driven system does that a language model cannot, why human law is the closest thing we have to a working guardrail, and why the paperclip scenario does not frighten him. Yann LeCun: The Doomers Are Solving The Wrong Problem Those systems cannot be jailbroken. Not because we patched them, but by construction.
00:00 The paperclip argument runs out of iron 00:50 Why a safe AI proof cannot exist 02:36 The system that cannot be jailbroken 04:04 We already built guardrails for humans 06:47 What the printing press did first
🎯 Support Into the Impossible on Patreon — get my weekly M.A.G.I.C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeating
The speaker argues that AI safety should be approached through engineering and objective-driven design rather than impossible mathematical proofs. By implementing strict guardrails and mental models, AI can be made reliable and safe. He views AI as a transformative tool for amplifying human intelligence, comparable to the impact of the printing press on society.
Nick Bostrom was on the podcast and he's famous for this paperclip problem,
but I pointed out to him, you know,
you can't produce infinite paperclips because there's only so much iron inside
of the core of the earth or on any asteroid or on any solar system.
So it's kind of preposterous. They first have to have the agency, the desire,
the target teleology, the, you know, the purpose built.
And I don't want to say it's impossible because that's one of my rules.
I don't say anything's impossible,
but it seems like Max is obsessed with perhaps, you know,
proving something which can't be proven. He's asking you provably safe,
but what if, what if that's a proving a negative?
I don't believe, I mean, there also, uh,
Russell also wants to, is looking for, you know, provably safe AI system.
I think that's just as impossible as a provably safe turbojet.
You can't prove that the turbojet will be safe.
Yet we can build incredibly reliable turbojets, right?
They can fly you halfway around the world in complete safety with only a two
engine airplane, right? I mean, that's, that's my bargain in terms of technology,
but we can build those things. Like AI is going to be the same.
There's not going to be a magic bullet.
There's not going to be a proof that we can build, you know, safe systems,
but we're going to engineer safe systems.
And the way we're going to engineer them, I think, like you said,
that's the way I think, uh,
things are going to go is that we're going to build systems that are objective
driven. I call this objective driven AI. Okay. And it's the,
the fact that the output that is producing,
that is produced by the AI system is not the result of just producing a token
after the other is a result of optimizing an objective with respect to a set of
actions you take. So you have some mental model in your head of, in your mind,
in the mind of the system,
the system has a mental model of the situation it's it was to, uh,
or the environment it wants to act into.
It's imagining a sequence of actions is going to accomplish.
And through the sequence of action that is mental model,
it can predict what the outcome is going to be.
And now you can check whether this outcome satisfies a set of objectives.
So one of them is, did I accomplish the tasks that I set out to accomplish,
but making a thousand paper clips,
but then there might be other objectives that are more like constraints and there
would be guardrails. Okay. So, you know,
you pay a high price for killing someone or hurting someone, for example,
right. Or maybe for, you know,
taking certain actions that will consume too much energy or whatever. Right.
So you can imagine having a series of those, uh, objectives,
some of which are guardrails, some of which are task objectives.
And then the way the system produces its output is that it through optimization,
it searches through the space of action sequences for one that minimizes
all those objectives and guardrails. Now that's objective driven.
Those systems cannot be jailbroken unless you break them. Okay.
But you can't jailbreak them like you can jailbreak an NLM by giving it a real
prompt, which will kind of, you know, go outside of its conditioning if you want.
Right. So this system cannot be jailbroken.
The only outputs they can produce are outputs that satisfy the guardrails
according to their internal mental model of the situation.
And now the game to make a safe AI is going to be how accurate can those mental
models of the situation can be and, uh,
what guardrails do you have to put in to make sure that those things are not
going to go haywire and, and, you know,
try to sort the planet into paper clips. And that's really easy to do. I mean,
and, and we know how to do this for humans.
We've been doing this for humans for millennia. It's called making laws.
A law is a guardrail objective that tell people, okay,
maybe the act that you're planning to do here, um, seems good for you,
but if you do it, you're going to go to jail for five years. Okay.
So that changes your cost function for AI.
We'll blow a circuit in the GPU.
You know, humans can choose to ignore that guardrail,
but an AI system that is objective driven will not be able to ignore it by
construction because they will have to optimize it.
So it won't be able to escape those guardrails.
I always call it like the, the, you know, AGI of the gaps, you know,
it's kind of like this God of the gaps there. They act as if,
and I love Max and he's been very kind to me personally.
And I had him on many times, but, but the point is,
we're not just going to one day have AGI, right.
It's going to be an iterative process. We already see it, right.
We see Waymo, right? So right now I actually saw Waymo in LA last week,
but it was being driven by, it was very disoriented.
There was a guy in the front seat driving this Waymo. Uh, I was like,
what's the point of that? But I feel like, you know,
if we started to see that, you know, like,
so call that generation zero or full self driving, you know, Tesla,
I've been in them. They don't drive on the sidewalks. You know,
that would be the shortest way to get somewhere. You know,
the Waymo could just drive through the sidewalk to get past traffic.
It was a busy, but it doesn't do that. So we'll see evidence.
If there are guardrails, literal guardrails in the case of cars,
say driving off the rails, we're going to see that in iteration 0.1.
And then, you know, it's not just one day we're going to have how, uh,
and even how, you know,
I would say like the first operating system wasn't built with an operating
system. Like there's always precursor, you know, the first, you know,
computer wasn't designed on a computer. I mean, by definition.
So the first truly AGI is not going to be made by an AGI.
And so it's not possible to have this recursion,
but I want to give you the last word there before I turn to the final
questions. I want to ask you about being a professor in this new age,
but do you have any other thoughts about AGI?
I have millions of more questions, but we'll have to wait until another part.
I know you're very busy. Any other thoughts about, you know,
the dangers or that's the most common question I think you get.
And my audience asked you,
I have a very optimistic view, right? I mean, I think intelligence is,
you know,
one of the most desirable commodity that we're missing the most in society,
really, to make progress.
So I think the effect of having machines that amplify our intelligence
effectively might be as transformative as the effect of the invention of the
printing press in the 15th century and the dissemination of books,
which allowed dissemination of knowledge,
which actually gave a good reason for people to learn,
to read for which there was no reason before,
and then led to the dissemination of the enlightenment, science,
democracy, freeing people,
freeing themselves from the dogma of religion and becoming more rational and
things like that. So, so I think that had a profound effect, right?
It had temporarily really bad effects also,
like the printing press allowed the, you know,
ideas to be disseminated that, you know,
cause people to kill each other essentially for 200 years in Europe because of
religious dogma or something. But, but eventually really had, you know,
brought down the feudal systems and caused the American revolution and the
French revolution and things like that. So I think, you know,
AI will have similarly transformative effect of amplifying human intelligence,
assisting humans in accomplishing tasks that, you know,
otherwise would require other humans.
I think there's a bright future for humanity if we can, if we can do this, right?
And I'm not,
I'm not particularly scared of the kind of scenario that Max is talking about.
Hey everybody. I know that if you're enjoying these types of conversations,
you're going to love my Monday Magic mailing list,
where I explore the secrets of the guests that come on the show and other
exciting facets from around the world of STEM, science, technology,
engineering, and math. And best of all,
I enter each and every one of you into a competition to win one of these little
babies right here, a meteorite. Yes, that's right.
A fragment of the early solar system produced in a cataclysmic supernova event,
which ignited with as much intensity as I have for members of my Monday Magic
mailing list. I know you're going to love it.
So go to briankeating.com slash YT to join the mailing list and enter into the
competition to get one of these beauties each month.
But if you have a .edu email address, you're guaranteed to win one.
If you live in the United States,
go to briankeating.com slash edu to get your fragment, not of the metaverse,
but of the real early universe. Now back to the episode.
So you think that AI will literally be more transformative for humanity than the
Facebook poke feature of the early 2000s?
But speaking of Facebook, so you're an academic.
How do you think of yourself? Are you a professor? Are you a scientist?
Are you an employee at a top three firm on earth?
How do you identify yourself if you're woken up in the morning by a super
intelligent AI and they asked you it?
I'm a scientist. I'm also a,
I would also say I'm a educator, not just because I'm a professor in university,
but because also I, I do things like this, you know,
trying to talk to the wider public about certain aspects of science,
but I'm really a scientist.
So I spent about equal time during my career in industry and academia.
I started my career in industry at Dell labs,
which then became AT&T labs.
And then I worked briefly at the NEC research Institute.
And then I was in my early forties when I became a professor.
I'd never been a professor before. And then for 10 years, I was just a professor.
And then, and after that joined a Facebook at the time created a fair.
I run fair for four years. So I was a research manager,
I guess. And this was part-time.
I was also I kept being a professor at NYU, basically two-third, one-third,
more or less two-third Facebook or Meta now and one-third NYU.
After four years of running a fair and building it up and running it,
I, I, I stepped down from, from running it. So now I'm chief scientist,
I'm chief AI scientist at Meta and I'm what's called an individual contributor,
which means I don't, I'm not a manager. I don't have any people,
anybody reporting to me. I don't run anything. I hate running things anyway.
I would have a terrible administrator. Okay. I disorganized, you know,
I don't like doing it. You know, it's, it's torture for me. You know,
I did it for four years, but it's really torture. So I'm,
I'm much more interested in sort of intellectual impact.
So what the effect I'm having at Meta is that I'm,
I'm plotting a path towards human level intelligence because you know,
that's the scientific question of my life if you want. But,
but also it's like there's a product desire to have intelligent assistants in
your smart glasses that have human level intelligence.
So that's one of those rare cases where the lofty cosmic goal that you might
have lines up with what people will pay you.
I will never know.