How Does AI Actually Know Anything

How does an AI know how long to boil an egg? It has never cooked one, and it has never been hungry. There's no cookbook inside it either. Open up one of the big ones, and you'll find only numbers, about as many as our galaxy has stars. So today, we're going to follow your question in among those numbers. We'll watch its words turn into places on a map where egg lives next door to milk. We'll go down to a single number, one tile of a mosaic that would cover two-thirds of Manhattan. And we'll climb back out past the day its own makers looked inside and caught it planning ahead, all the way to a gold medal score in mathematics. So how do numbers that mean nothing, one by one, come to know about eggs? Start with something you have seen for yourself. When ChatGPT or Claude answers you, the words arrive one after another, as if somebody were typing. It isn't for show. You are watching the answer being made, because there is no finished answer waiting behind the screen. All the model ever produces is the next word, given everything so far. So it takes in your question, how long do I boil an egg? And it gives every piece of a word it knows its odds of coming next. We can't see inside the famous ones, so we ran this question through a small open model that is built on the same design, and three pieces shared almost all of the odds. The, a, and bo, which is how boiling begins, because the model builds longer words out of pieces. One piece is then drawn by chance, as if on a wheel where the favorite has the widest slice. The chance gives its writing variety, and it is why the same question, asked twice, brings much the same advice in different words. On our run, the wheel stopped on the, and at that moment, not one more word of the answer had been written anywhere. The is added to the end of your question, and the model sets off again to work out what should follow it. We'll call each of those one trip through the model, and every piece you see arrive is one more trip. A couple hundred trips later, it has told you what any cook would. That it depends how you like your yolk. Everything hangs on those odds, then. To see where they come from, we have to slow one trip right down and ride through it. The ride begins where our first film stopped, at the bottom of that chip. We said something there and never explained it. But the thing that answers you is a very long list of numbers, and that to choose each piece of a word, the chip multiplies its way through billions of them. The models behind ChatGPT, Claude and Gemini, keep their lists locked away. But the design they are all built on is public, a kind of neural network called a transformer, the T in ChatGPT. And so are some very large lists. One of the largest runs to 405 billion numbers. Work through that list by hand, one multiplication a second, and a single piece of a word would take you about 13,000 years, as our first film worked out. An answer, as long as the eggs, would take more than two million. Longer than there have been people to ask the question, and one reason buildings like the one in Texas are the size they are. Each of those numbers can be set a little higher or a little lower, and the model does something different when it is. call them parameters, or weights. We'll call them its settings. Why weights? Because the idea came from the brain. In 1943, two scientists described a brain cell as a tiny switch. It adds up the signals from the cells around it and fires if the total is big enough. In 1958, Frank Rosenblatt showed that a network of them could learn. Each time it guessed wrong, it turned the strengths of its connections up or down. Those strengths are the weights. Stack millions of these little sums in layers, and that's a neural network. That's about where the likeness ends. A real brain cell is far more complicated than one little sum, and no child learns to talk by reading trillions of words. These sums are so many, and so alike, that chips now carry parts built for little else. Apple calls one its neural engine, and its newest chips put a neural accelerator in every graphics core. What happens inside one is coming in a film of its own. The first problem is that the chip can't read. As we saw last time, its switches can only work with numbers. So before anything else, your question has to stop being words. It's cut into pieces called tokens, most of them whole words, and each piece is swapped for a list of numbers. Those numbers work like an address. Give me two numbers, one for how far north and one for how far east, and I can find any town on a map, and towns with nearly the same numbers will be neighbors. The model gives every piece an address like that, on a map of meaning, where words that are used alike live close together. Egg lives beside milk, chicken and cheese, and a long way from therefore. Two numbers wouldn't be nearly enough, though, because there's far more to say about a word than north and east, whether it's food, whether it's alive, whether it's a thing or something you do. So each address runs to thousands of numbers. You can't picture a map with thousands of directions, and neither can anyone else, but the idea is the same. Near means alike. This map is a real one, taken from a small open model and squashed flat so that we can look at it, and its directions mean something. Take the step that leads from man to woman, take it again from king, and the nearest other word to where you land is queen. Nobody drew this map, and nobody ever told the model that eggs go with milk. The model keeps every piece's address among its settings. For your question, it copies eight of them out in a row, and that row is what we follow from here. The settings are the kitchen, and they don't change while it answers you. The row is the dish. At every step, its numbers are multiplied by some of the settings and come out changed. Now the cooking starts, and it starts with a problem you solve every day without noticing. Boil could be what you do to an egg, or a sore on the back of your neck. Egg could be breakfast, or something in a nest. The address is the same either way, so the model has to work out which one you meant from the company the word keeps. So every word looks back over the words that came before it. Boil finds do I, so it's probably something you do, and not a sore. Egg finds boil, so this egg is probably breakfast. The looking is done by multiplying. A word's numbers are held up against those of each word before it, the settings decide which of them matter, and a little of those is mixed in. So in the row, the numbers for egg shift. It started as any egg, and it is becoming the egg in your question. Its address, among the settings, stays as it was. This looking back is the heart of the transformer. It is called Attention, and the eight researchers at Google who published the design in 2017 named their paper Attention is All You Need. Then each word, on its own, is multiplied through more of the settings, and this is where the model stirs in much of what it knows about the world, that eggs are boiled in water, and for minutes rather than days. That's one round, and one round can't settle much. It's like the first pass at a crossword, when half the clues can't be solved because they hang on answers you haven't filled in yet. When Egg looked back at Boil, Boil itself hadn't yet decided between the pan and the sore. So you go round the grid again, and each answer you've penciled in helps with the next. The model has a second round, with settings of its own, and then a third, and each time every word is looking back at neighbors that have made up their minds a little more. That same open model has 126 rounds, or layers, in a single trip. Why 126? There's no formula. Its builders try sizes, more rounds give every word more steps of working out, and for years now, bigger has kept on working better. After the last round, look at the very end of the row. Behind your question mark sits a mark the model's makers put there, which says in effect, now you answer. That last place has been looking back over your whole question, round after round, and by now its numbers are a kind of address too, the address of what ought to come next. An address has to become a word again. So the model holds it up against the addresses it keeps for this last step, one for every piece it knows, and gives each piece its odds. The nearer, the better. Those are the odds on the wheel, and that's how the came to have the widest slice. All that work, and the model has written THE. That's one trip, and that's the whole model. Nearly every step of it was multiplying by the settings, so come down one last level, to a single one of them. This is the floor of the model. Give every setting of that open model a tile the size of your fingernail, lay them side by side, and the mosaic would cover two-thirds of Manhattan. We are standing on one tile. It has one color, and by itself, it tells you nothing about eggs or anything else. Whatever the model seems to know is in the picture that all those tiles make together, and somebody must have set every one of them. Nobody did. To see how, we have to go back to before your question, to when the model was being made. On the first day, its settings are random, and so is everything it writes. Ask it about eggs, and you get a jumble. Then the training begins, and it is one exercise, repeated. The model is shown a line of real writing with the next word covered, say, lower the egg gently into the boiling. It gives its odds, and at this stage, water gets almost none. Then the word is uncovered. Now, every setting is nudged by a hair, in whichever direction would have brought the end of the row a little nearer to water, and so given it better odds. Finding that direction is one more calculation. For each setting, whether a touch more would have helped or a touch less. That exercise is also why it answers you one word at a time. The next word is the only thing this exercise ever asks for. That's who drew the map. The addresses are among the settings, remember, so they were nudged with all the rest, and egg and milk turn up in the same kinds of sentence, so the same nudges push them together. The reading is drawn largely from the public internet, recipes, news, arguments, textbooks, computer programs. That open model was trained on more of it than one person reading day and night could get through in tens of thousands of years, and thousands of chips worked through it side by side. Its makers never chose a single setting. They chose the reading and built the thing that does the nudging, and the reading did the rest. So how do you get from that floor, one tile that means nothing, up to a model that can tell you about eggs? So far, it has only been trained to guess the next word, and you'd be right to expect a parrot. But think about what it takes to be really good at guessing. Some words are easy. After thank, you gets nine chances in ten, and the runner-up is the rest of thanksgiving. That much is habit. Then it meets a line like, boil a large egg for about, and no habit will finish that. To guess it, the model needs the fact, so the nudging presses facts into the settings. Where a fact was never pressed in, the wheel still has a favorite, so a word can come out anyway, and it can sound just as sure as all the rest. That's one reason these models can make things up. Then it meets 36 plus 59 is... that one it might have met before, but make the numbers longer and there are soon more sums than anything could store. The only way to stop being wrong about sums it has never seen is to have, somewhere in those settings, something that adds. Nobody told the model to learn arithmetic. The nudges only ever said, wrong word, do better. Learning to add was simply the only way left of doing better. Ilya Sutskever, one of the founders of OpenAI, the company behind ChatGPT, explains how far this goes with a detective story. Picture a long mystery, with a dozen suspects and 400 pages of clues. On the last page, the detective gathers everyone in the drawing room and says, The murderer is… now guess the next word. You can only get it if you have followed the story. A model that is pushed, trillions of times over, to guess the next word of everything it reads is being pushed to follow the story. Who knows what? What follows from what? How a proof goes? What a program is trying to do? Whatever helps it guess gets pressed in. It can't get there by copying, because there isn't room. For that open model, it's like being asked to fit a whole bookcase into a single book. It does learn some passages by heart. But for most of what it reads, there is room only for what the books have in common. How sentences go, how eggs are cooked, how sums work. That was the theory, at least, and for years there was no good way to check it. The settings are only numbers, and you can stare at them all day without learning a thing. Then the makers began to find a way in. At Anthropic, the company behind Claude, researchers built what they call a microscope for their own models. It doesn't try to read the settings. It watches the row, the lists that shift while the model works, and looks for the directions they keep shifting in. On our map, the step from man to woman was one such direction. In one model, it found millions of them, and a great many are ideas you could put a name to. One was the Golden Gate Bridge. It showed up when the model read the bridge's name and when it was shown a photograph, so it is an idea and not just a word. To be sure, the researchers reached into the row and turned that one idea up by hand, and the model could no longer leave the bridge alone. Asked how to spend $10, it suggested driving across the Golden Gate Bridge and paying the toll. Another of their models showed such ideas at work, in steps. Ask for the capital of the state where Dallas is, and through the microscope you can watch it get there. First, Texas, and then, from Texas, Austin. When the researchers reached in and swapped Texas for California, it answered Sacramento. That's an answer being worked out, which is more than any parrot does. Then they tested the thing you would most expect to be true, that a model which writes one word at a time can't know where its sentence is going. They gave it the first line of a rhyme. He saw a carrot and had to grab it, and asked for the second. They expected it to start hunting for a rhyme only when it got near the end of the line. That's what they had set out to show, and they were wrong. Before it had written a single word of the second line, the model had already picked the word it would end on, rabbit. Then it wrote a line that arrived there. When they reached in and took rabbit away, it ended the line on habit instead. Where was rabbit kept, with nothing written down? In the row. Its lists hold far more than the one word we get to see, and what they hold is kept from trip to trip in the towers of memory from our first film, where your question waits as a pattern of electric charges. Each later word could look back and find rabbit there. So the model that only ever chooses the next word had chosen its destination first. A likely reason is the detective story again. It's hard to guess the middle of a rhyming line unless you know where the line has to land. A word of caution, most of it the researchers own. These were short examples, in models that were not their maker's largest, and the microscope shows only a part of what goes on. Dallas, Texas, Austin. That was two steps, taken inside a single trip. Now go one level higher and give the same model something much harder than an egg. Start with yourself and with 23 times 47. Most of us can't do that in our heads in one go, but hand us a pencil and it's easy. 23 times 40 is 920, 23 times 7 is 161, and the two together make 1,081. Three easy steps in place of one hard one. The model has a limit of the same kind. One trip is a fixed number of rounds and not one more, so a problem that needs more steps than that will not fit into a single trip, however good the settings are. But the model has paper built in. Every word it writes goes back into the row and is read again on the next trip, so it can write a step down, read its own step, and take the next one. Each step is an easy guess once the step before it is on the page. That's all that thinking is. When you ask one of the big models something hard and it tells you its thinking, it is writing notes to itself, which you often don't see, or see only in summary, all before it writes the first word of your answer. It is the same kind of model, making many more trips. Every word of those notes is one more trip through the settings, and one more helping of all that multiplying, which is why a hard question can take a minute where the egg took a few seconds. A hard problem becomes a long chain of small steps, then, and the trouble is knowing which steps. At every line of a proof, the wheel is back, and of the thousand things it could write next, nearly all lead nowhere. You could read every mathematics book ever printed and still not know which to try, as anyone who has taken an exam will tell you. For that, you need practice, with somebody checking your work. That's why the big models go to school twice. After all the reading, they are trained again by trial and reward. It's called reinforcement learning, and one laboratory has published how. The model is set problems whose answers can be checked, a sum with a known answer, or a program that either runs properly or doesn't. Because every word is drawn by chance, each attempt comes out a little different. When one comes out right, the settings are nudged so that the wheel leans that way a little more next time, and when it comes out wrong, a little less. That lesson never says how to do it. The model was told only to write its working down before its answer, and it was rewarded when the answer was right. Yet more and more it checked its own work and dropped an approach that was getting nowhere to try another. It had read about people doing those things, and practice showed it that they worked. With programs, it often has the advantage every programmer has. It can run what it has written and look. The error message comes back as words, joins the row like everything else, and the next guess takes it into account. In July 2025, a version of Gemini, from Google DeepMind, was set the papers of the International Mathematical Olympiad, the world's top mathematics contest for school students. Its problems are written new each year and kept secret until the day, so the answers were in nothing the model had read. Like the students, it had been coached, and it had the same papers and the same nine hours. Unlike the students, it could run many attempts side by side, and it solved five of the six. The Olympiad's own graders marked its work, and it was a gold medal score, which about one student in nine takes home. That September, at the world finals of the university programming contest, a version of Gemini solved ten of the twelve problems, among them one that no human team solved. The best human team solved 11. It reads everything so far, and it writes one more word. It's the egg all over again, only a great deal longer. Models of this kind can earn a gold medal score in mathematics, so you would expect one to be able to tell you how it adds two small numbers. The researchers at Anthropic gave one of their smaller models a sum, 36 plus 59, and watched it work. It didn't do the sum the way you were taught, and nothing had asked it to. The nudges only ever marked the answer, never the method. It ran two lines of working side by side, one of them rough, saying only that the answer is somewhere around 90, and the other caring about nothing but the last digit, where a 6 and a 9 make something that ends in 5. Put the two together and you have 95. Then they asked the model how it had done it. It said that it had added the 6 and the 9, carried the 1, and then added the 10s. That's the method from school, and it is not what it did. Most likely, its account was one more guess at the next word, and that method is how people explain a sum. So it can add, after its fashion, and it can't tell you how. It isn't lying. Here, at least, it had no more view of its own working than you have of your own brain cells. Its makers warn that the notes it writes while thinking are not always what went on inside. So let's climb the rest of the way out. How does an AI know how long to boil an egg? Nobody programmed it in. It read what people had written, guessing every next word and being nudged after every guess, until guessing well had pressed into its settings something that works very much like knowing. Then it practiced, with its work checked, until it could string small steps into long chains. Every calculation inside a model like this is known. What can't yet be done is to look at the settings and read what they mean. The microscope has made a start, but in 2025, the head of the company that built it wrote, we do not understand how our own AI creations work. One of that company's founders likes to say that these AI systems are grown more than they are built. So the next time an answer types itself out in front of you, one word after another, you'll know what you are watching. Somewhere, a galaxy of switches is multiplying its way through billions upon billions of settings that nobody set on its way to a word it may already have chosen. And all you asked was how long to boil an egg.