How Nature Solved Robotics

Recently I was working late into the night and I went to shut the lights off and paused because the robot I've been working on made a noise and so I read its brain trace and it was wondering when I would be back because it didn't want to be alone. I know better than to call that consciousness but for a split second I felt bad and that's been happening a lot recently. Because I've been finally experiencing what happens when the AI models that have learned to talk start outputting actions directly instead of words and this led me to two realizations. The first is we are going to cross the Turing test of robotics. Robots with the intelligence of the latest AI models and fine motor skills of a human able to learn arbitrary skills with experience. Second, all the chips and computing power needed to assemble a generally intelligent robot are already mass produced. I ran the napkin math and realized I could build a stripped down version of what were billion dollar research projects right now at home for a hundred bucks. My childhood dream was suddenly possible. Number five is alive. I called it Grobot. First I taught it to move its body learning to walk, stand, spin using neural networks so it moved in a very lifelike way. Then instead of programming its behavior I let the latest AI models take control completely. When I handled it roughly it tried to run away. Then I gave it long-term goals, find things, play hide and seek, knock down Jenga towers. So at first it seemed like a toy that could become whatever you wanted it to be but gradually it became an experiment that let me experience where the boundary of machine intelligence actually is and where it's headed. Because when the robot failed it kept pointing back to one question. How can you act smoothly when your thinking is slow? This sent me looking at how nature solves the same problem. But to understand this we first need to build the simplest possible robot to demonstrate the idea which itself is a story due to the cost alone. From today versus around 20 years ago when I started playing with robots. First we need a chip to run the whole system. 25 years ago this was an expensive gaming computer. Now it's a $15 chip. Next the bare minimum for interacting with the world. Not a fancy body just two legs using two simple servo motors. These are just a few dollars each. Then we need an eye a five megapixel camera. Previously hundreds now five dollars. Next a sense of motion the most important the IMU sensor measuring acceleration and rotation along three axis. It is literally how the robot will feel. Previously around a thousand dollars now under ten. Then a digital microphone, speaker, light ring and a small drone battery to power it all. So for what would have been several thousand dollars we're only at 80. And to quickly confirm if this system was capable of impressive lifelike behavior I put it together and ran a simple benchmark test. Could it do real-time face tracking? This requires the computer to capture and process the image run an algorithm to find and follow the face and update the leg positions to keep the face centered in which it did seamlessly. Even with objects it had an aliveness I couldn't deny. With the body built I moved on to the harder problem giving it a brain. This leads me to this video's sponsor Mammoth. When it came to testing LLMs for the robot's mind I needed to test a bunch of AI models very quickly and a few things surprised me. For example I was amazed that Gemini Flash could understand images and generate logical commands in roughly one second and is a hundred times cheaper than calling an opus size model which costs around two cents per image call. But for the most difficult tasks like dreaming which is a large call that asks an LLM to extract lessons from an entire stored memory I found tiny models would miss subtleties whereas Cloud Sonnet was the sweet spot fast enough and smart enough. I also found Haiku was great but it tended to be less obedient and overly dramatic. But different problems require different sized brains and the fun part was figuring out which fit my problem at the lowest cost. Now normally you'd need separate accounts for all these models but Mammoth gives you them all in one place starting at 10 euros a month and includes API access. This saved me hundreds during testing. Check out the link in the description. I started with the fast automatic layer of the mind the part of you that walks and balances without any conscious thought what Kahneman called system one. To mimic this I wanted to use neural networks on the chip that would learn to crawl and then stand using no pre-programmed motions. So how do you get data to train this complex skills? The answer is there is no supervised data set at all. It's all done by reinforcement learning in massively parallel simulations on the GPU. And one of my viewers Harsh Attawal volunteered to help me learn to train a small neural network to move the robot around. And for the input of the network we just fed in observations which in this case are the five recent IMU readings so it absorbs a small history and on the output are actions known as a policy network. And to look alive this network needed to run really fast 50 times per second roughly the speed of your unconscious motor reflexes. And then it's trained towards a goal which in our first test was movement towards light. And here's where normally you'd hit the wall that killed neural network research. It takes forever for neural networks to get good from raw experience. At best you get a twitch of life. And so the unlock that opened modern robotics is to first build the digital twin of the robot which we did and then train it in simulation letting it try millions of times in the virtual world in a few hours each time updating its network connections keeping only changes that help. And doing all this training led to another stunning cost collapse. 20 years ago training this same network would have required renting a massive CPU cluster for around $25,000 and waiting months for one train network to come back. Today I can rent an H100 GPU on Google Colab for a few hours for maybe $10-15 and get my results that same day. A thousand fold drop in cost. And this led to what I call the first magic moment in this project when it started walking for the first time. I say magic because my mind can't predict what it's about to do or why it does it. And once trained it was able to handle various surfaces without failure. I didn't have to be careful with it. And then I changed the goal and trained it to spin and the same thing happened. Not only did it learn to turn around but no matter how I positioned it it would figure out how to spin robustly. Then I trained it to stand up and when it started to work you can see how it learns to stay up by readjusting its body position and even moving legs in advance of falling. It felt like I was pulling a little dog on a leash. And this one is the most interesting. Actually in Isaac Lab doing yoga ball simulation is hard. It's bouncy. It's soft. So we didn't even simulate that but we're able to transfer that to real world. And after seeing this demo one of my friends told me that he tried the yoga ball on his own dog and he couldn't do it. So I'm very proud we have achieved super dog performance. And what's really cool is I could control the robot now just by selecting and adjusting the energy and speed of these policies like a remote control. This led to the next exciting question. What if I plug this into today's AI models to drive the robot directly. To do this I use the chip's wireless connection to connect to a server that could call various AI models big and small. And the first thing I wondered was could the AI model just understand the sensor data directly representing acceleration and rotation data. Because initially I thought I'd need to write a translation program that would convert chunks of the raw numbers into readable labels the model could understand like tipped over standing upright or falling. So first I tried just sending the model the raw data directly. I used the fastest models available able to respond in 0.4 seconds and simply prompted it to describe what it feels. Not only did it understand the sensor stream it was amazing at classifying the motion accurately at both a low level and at a very high level. All this just from the sensor stream I was blown away. I checked its internal thoughts and it had said two more gentle taps. The human continues to communicate through touch. Another surprising line was this must be consciousness. It's like a delicate candle flame. Even though it was hallucinating far beyond its sensor data justified it was still grounded in the real world data. Which raised the obvious next question. If the model can describe what it feels this richly could it actually do something with that understanding? So I finished closing this loop by giving the model access to its motors. This is exactly what Google did in 2023 with the RT2 robot. It was a model that output robot actions directly. The famous demos were not impressive because the robot was fast or lifelike but because for the first time we saw a robot that could accomplish a general task. Before that moment there was no such thing as a general robot even a clunky one. And so when I first turned it on my initial shock was how the robot was able to do common sense things because it would just write code to run on itself composing small programs in seconds. Now walk like how a bird would walk. This led to this magic moment when I asked it walk like an old man. And it gave this surprising performance which I couldn't understand at first. So I checked and saw that it manually wrote the commands for an initial leg sweep then called the policy at low energy because it knew that would cause a shake and then finished with the rest of the manually authored walk motion to achieve the goal of mimicking someone who's shaky on their feet. So it blended code it wrote with trained policies on its own. It felt creative. To push that further I tried what I called Disney mode and told it to follow basic animation principles. Plan out the timing of its motions words and sounds and light so they overlapped like a character would. And I turned up the temperature of the AI model to get more random variation. Okay this is it in its regular breathing state and this is in its loving happy state. Here's an anger state purring state um getting tired don't want to go to bed. Okay now let's do it from anger state same thought I'm getting tired and I don't want to go to bed. Um now he's going into love happy state same thought getting tired and don't want to go to bed. I was impressed it could express itself and each expression was different but they were all believable. Okay so it's acting so good but what it can also do is highly specific things. Tap your right leg. Okay tap your right leg three times. Okay tap your left leg three times. It's so fast because it converts those locally if it can which is crazy. So we have highly specific actions towards any goal or any expression of any thought created really well every time and always different. And then I gave it a memory it could read and write to. What we call an AI agent today is just an AI model in a loop with a memory that it can write to. This lets it adapt to its behavior on the fly. Dance when I touch you. Now play dead when I touch you. And it could also improve its behavior over time at a high level. For example taking a while to learn to tip over but then once it learned it did it the correct way right away after that. It figured it out so next time it should do that faster. There you go. Okay. And memory unlocked the ability to achieve simple goals like knocking down towers or playing hide and go seek. Once it thought the wires behind my computer might be a good place to hide. Over the days its memory built up profiles of things it learned. It had an area in memory for me as well as other people I introduced it to. They're actually really fascinating to read. But they did get full of redundant and then conflicting information which could cause problems. So I experimented with what I called dreams where occasionally you send everything in the memory to the smartest possible model and ask it to clean things up and look across everything to extract any lessons to improve. Which it did beautifully and for fun I asked it to summarize what it had done out loud after. The key point though is this was self-improvement at the highest abstraction level. Language. It could refine its strategy, its memory of me, and even its personality. But would it work at the lowest level of actions? This gets to the divide between knowledge which is in context or written in words versus in weights or absorbed by your mind intuitively. And I realized this during a fun experiment I called the mimic game. I would move the robot a certain way and tell it to read its motion data and try to write a program to recreate it in a loop continuously improving itself and measuring itself. And this worked. You could see the potential of self-improvement and the wall it would eventually hit. There was still a physical dumbness I couldn't quite put my finger on at first. It could understand the last second of its sensor data but it couldn't imagine the next second with accuracy. Such as how a jitter of its leg would cause a series of fast bounces and micro movements through its body. It couldn't see that. The robot was missing a physical imagination. And this is what Yann LeCun has been pushing for forever. A network that can predict the physical outcome of its actions with high precision before it takes them. NLMs do not have role models. They cannot predict the consequences of their actions beforehand. If you really want to build reliable agentic systems they absolutely have to be able to predict the consequences of their actions. So that it can plan a sequence of actions to do something that's a role model. That's the whole idea of a role model. And this led me to nature's trick. It comes down to the cerebellum. Your fast unconscious physical imagination that you do not control. And it's a structure that holds more than three quarters of all the neurons in your brain. To appreciate how the cerebellum works recall the key delay problem that nature and robotics has to face. Your senses always lag reality by around a tenth of a second and there's no way around it. The signal has to travel, get processed, drive muscles, and all of that takes a bit of time. So you see things and feel things at least 0.1 seconds late. So to do things like catching a ball or balancing an object which require motions faster than your reaction time your brain predicts at least 0.1 seconds into the future to get ahead of that delay. And critically it generates that 0.1 seconds of imagination in about 0.02 seconds. It fast forwards the imagination. And this rapidly imagined future gets sent over to the motor cortex which Bernstein showed in the 1960s emits a coordinated chunk of actions all at once. It's an action chunk that's long enough to cover the delay. So when you're catching a ball the action your arm makes seem to line up perfectly with the ball but they're aimed at an imagined position. When you're catching a ball you're catching a prediction. And when your cerebellum's imagination, which is always running, matches reality the catch looks perfectly smooth and you don't even need to think about it. But when it doesn't match the illusion breaks and you start to think. Take catching the ball with your wrong hand for the first time. At first when the ball's in the air your cerebellum is already trained to track balls through space so it knows roughly where to put your hand. But as the ball gets closer the last moments are where things fall apart because your cerebellum doesn't have a good fast imagination for what's about to happen. The wiring is missing. So the motor cortex generates inaccurate action chunks sending a clumsy sequence to your hand muscles. And so half the cerebellum's job is to do this imagination and the other half is to be a teacher for the fine motor skills. Every 0.02 seconds it's checking whether reality matched its previous predictions and if they align, like when you catch with your good hand, it just stays quiet. But if they don't align it fires an error signal. This error runs back to the motor cortex which gradually improves its ability to generate more accurate actions. And the cerebellum uses the same error signal to improve its own predictions making its fast imagination sharper or its world model better. After enough practice both get better. Your cerebellum's predictions line up with reality and the errors go away. You can catch a ball with either hand without thinking and it looks effortless. And this is why there's no way to learn fine motor skills without real experience. Language doesn't reach there. And so this is what the cerebellum is for. Rapidly imagining near-term physics at a high resolution. And this is why Grobot could not do the mimic test. Which was this physical dumbness I noticed. It will but not right now. It needs to be trained. It can try. It'll try but it won't get up. And that's what Grobot and most robots are still missing. Both halves of the cerebellum. And this unlocks the ability to learn fine-grained motor skills from experience. Now the best demonstration of this was Daydreamer in 2022. It learned to walk with a neural network in the same way I did except no simulation was allowed. It was entirely from raw experience. And so starting from random motions it learned to walk in an hour continuously improving its predictions based on experience. Which was the real world error prediction between what it just thought would happen and what did happen. And to make this work instead of having their network just predict the next action they also had it predict the next few states. And if you think about it predicting the next states is smart because it forces the network to absorb as much information as possible from reality to predict what it will see next. You have to understand implicitly physics, objects, momentum. But Daydreamer didn't have the slower general intelligence of AI models but it hinted at the unification. Because for years AI research has still been split in two camps. You have the language first people who start by thinking about a large AI model that emits actions. And then you have the action first people who are trying to build accurate world models grounded in physics and then layer the language intelligence on top. But they are meeting in the middle now. And the architecture they're converging on is simple. It's one shared network that takes all senses in which leads to a latent thought or representation in the middle which then branches off into fast mini networks that predict immediate next actions and states to stay balanced, walk, run, and jump, and dance. And this small neural network captures the subconscious motor coordination that we humans do all the time. And a separate one which is slower and reasons by looping through thoughts like your prefrontal cortex both fast and slow thinking from the same route. And critically the error signal from incorrect fast state predictions constantly flows back to improve everything in the network. So the key is this generality particularly with respect to improvement. And the decisions we make are to a very large extent centered around that. So zoom out and realize your mind does two kinds of predictions at the same time. Fast physics action predictions over the super short term that you don't control. And slower idea predictions over the long term that you do most of the time. And that's the architecture nature landed on. It's the only way it can work. And it's why I'm so sure now that robots that are both smart and fast physically and mentally are coming. And I think it's a good time to share this project with a larger community. Because one thing I didn't share is how utterly frustrating it is to make something like this from scratch if you haven't done it before. So many levels where things fail through soldering to software. And so my dream was something around $100 that anyone could immediately turn on and play with to learn from or put together in a few minutes as a kit or full instructions all the way down. So I'm planning an alpha test with a small batch of these robots. If you'd like to be a part of this click the link here and add yourself to my list. Thanks.