Terence Tao has read more mathematics than almost anyone alive, and he uses AI tools every day. So when one of the most cited mathematicians on Earth says these systems still can't ask a genuinely new question, it's worth understanding exactly where he draws the line — because it isn't where the headlines put it. Watch the full conversation: https://youtu.be/ukpCHo5v-Gc If AI has absorbed every textbook ever written, why can't it discover anything new? Tao, a Fields Medal winner and professor at UCLA, separates what these systems do brilliantly from what they can't do at all, and the boundary turns out to be sharper and stranger than most people assume. We cover why reproducing a famous proof is less impressive than it sounds, what a neural network found hidden inside a million knots that humans had missed, why we still can't predict which tasks AI will actually be good at, the "Keating Test" — the benchmark that would actually demonstrate machine thought — and where exhaustive recall ends and real conceptual origination begins. AI can pass every exam. It just can't ask a question nobody has asked before — yet.
Chapters: 00:00 The question AI can't ask 00:48 Read every textbook, discover nothing 01:42 Why a reproduced proof proves less 02:39 A million knots, one hidden pattern 03:54 The competence we still can't predict 05:11 The Keating Test for machine thought 06:18 Where recall ends and discovery begins
🎯 Support Into the Impossible on Patreon — get my weekly M.A.G.I.C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeating
This discussion between Professor Brian Keating and mathematician Terence Tao explores the paradox of AI: while the underlying mathematics (matrix multiplication and calculus) is simple, the reasons why AI works so effectively remain a mystery. They discuss the difference between memorization and true reasoning, and how AI can serve as a tool for scientific discovery.
Once they have this black box, they could analyze it.
It gives you a new paradigm to access reality in traditional experiment or theory.
The greatest living mathematician told me the math behind AI is embarrassingly simple.
His words, not mine.
Matrix multiplication and a little bit of calculus.
An undergraduate could do it.
But then he said something that stopped me cold.
We have no idea why it works.
Not a clue.
Not even a good rule of thumb.
Here's Terry Tao explaining the deepest mystery in artificial intelligence.
The hope is that if you force the large language models to only put in some verified,
some language that you can verify and to filter out the hallucinations.
Has it been able to reproduce a Wiles, you know,
proof of Fermat's last theorem or your work, Navier-Stokes?
I mean, has it been able to actually just simply, quote-unquote,
reproduce what a natural intelligence person like you does?
It can, but often because of what's called contamination.
If a result is like taught in textbooks somewhere,
then it is implicitly in the training data that these AIs train on.
And so they're basically memorizing the same way, again,
that a student at the board may just reproduce from memory a proof that they saw in a textbook.
So AI has basically read all the textbooks in the world.
It's hard to discern when that happens, whether it was training data or whether they really
sort of thought it up.
If you ask the AI to explain their chain of thought,
they often give complete nonsense.
Like it's clear that they just didn't know.
Yeah. I mean, I've found that even we tried with my student and we tried to,
we just gave it the information about the orbit of Mercury over the past 3,000 years.
And then we said, well, if you observe this in this planet, you know, basically,
could you first discover this anomalous procession of the perihelion of Mercury?
And then could you predict it?
You know, and it was just completely unable.
It required us, we had to first discretize everything,
make everything Euclidean, which then totally ruins it.
Right?
So I've proposed, and I want to get your take on it.
Kind of a joke.
I call it the Keating test, but it's basically the Turing test.
We will know when it's true and actual AI can come up with new
and unknown heretofore unknown, you know,
predictions that can be verified by humans like you.
Yeah. No, I think that's a very promising use case of AI.
I mean, neural networks in general, I mean,
they're designed to make patterns, detect correlations and things.
So there have been a few examples in mathematics where, for example,
in knot theory, a neural network, not a fancy LLM so much,
or old school neural network, was used to detect correlations
between different types of knot invariants that was not believed,
people did not suspect existed before.
And initially this type of correlation was just sort of this black box relationship.
So knots have these loops in space, which some can be untangled and some can't be.
They come with all these numbers.
They're called knot invariants.
And so the neural network found that by feeding a database of like 1 million knots,
that there was one knot invariant called a signature,
which could be predicted with really high accuracy
from a whole bunch of other invariants called hyperbolic invariants.
So, but this neural network was this black box.
You just fed in these 20 numbers as your hyperbolic invariants
and it will spit out, the signature should be plus three
and like 90% of the time it was correct.
But once they had this black box, they could analyze it.
Suppose I change this input.
I modify this hyperbolic volume or whatever.
How much does this change the output?
It's like a box with 20 dials that they could play with.
And basically by running experiments,
they could see that three of these inputs were actually really important
and the other 17 were very peripheral.
And by doing those types of analysis,
they actually got some insight as to what the relationship was
and that they could actually make a formal mathematical prediction,
which they could then prove.
So once you have these neural network models,
you could actually program.
So in your astronomy example,
maybe a neural network might not be able to tell you
exactly what the new law of physics would have to be,
but it can say, well, I can at least predict
the orbit of Mercury over the next thousand years.
Here's my model.
And then you can just try to tweak it.
Now, suppose I change the period of Mercury or the Mars, whatever,
what happens to it?
And maybe you can work out laws of nature experimentally.
It gives you a new paradigm to access reality
in traditional experiment or theory.
Yeah, I've used that example.
There's a lot of AI doom people that think AI's going to run amok
and turn us all into paperclips and all sorts of nonsense.
But because it seems to have this feature that you mentioned,
that it's sort of averaging over all of human knowledge
and so it'll have errors,
but it's bounded by the amount of human knowledge
it's used in some level in its training set.
But then there's something magical about it.
And I wonder, the mathematics, I mean,
I'm in the presence of greatness, right?
So the mathematics though aren't that common.
It's matrix multiplication at a massive scale,
high dimensions and huge volumes.
But is it really that complicated and intrinsically?
So the mathematics to train and run a large language model
or any other modern AI is not that complicated.
So an undergraduate math major would have all the prerequisites.
Basically, you need to know how matrix works in a liberal calculus.
The area where we don't have a good method of theory
is how to evaluate, how to predict the performance of these models.
So the mystery is not so much how they run.
We know how to make a large language model
and how to train it and how to run it.
But what is surprising is that it works really well for certain tasks
and it doesn't work well for others.
And we don't know in advance, we don't have good rules.
Even heuristic rules of thumb for predicting which tasks are good,
which ones are not, can only just make empirical experiments.
Part of the reason is that the data that you train on,
so on one level, it's just strings of 0s and 1s.
And mathematically, we understand sort of very, very random data.
So like complete noise, completely random 0s and 1s,
we have the mathematical probability theory,
which explains how we can analyze this situation very well.
And then we have very, very structured types of data,
like a sequence which is all 1s or all 0s,
alternating 1, 0, 1, 0 in a very periodic fashion.
Very structured data we understand very well.
But the type of data that is natural data, like English text,
you can digitize that as strings of very specific 0s and 1s,
but not so specific that they're completely predictable.
But they still seem to be somewhat predictable.
And so we don't have good mathematics for partially structured objects.
In physics, we have continuum mechanics,
where everything's sort of averaged out.
And then we have atomic level physics.
You can look at individual molecules and particles.
But at the meso scale, there's lots of intermediate structures,
like cells, for example, biological cells.
— It's emergent.
— Yeah, it's emergent.
We don't have good mathematics for this.
I mean, in principle, you could break down the atoms,
but you can't possibly analyze.
— Yeah, it's not mathematically impossible,
but in practice, it might be physically impossible.
Terry and I went much deeper on this,
including whether AI can actually do original mathematics.
Full conversation is linked right here.
I'm Professor Brian Keating.
I don't dumb things down.
I'll be talking to Terry again soon, so subscribe for more.