Boston Dynamics’ Atlas humanoid robot, now powered by Toyota’s cutting-edge Large Behavior Model (LBM)! In this video, we dive into Atlas’ groundbreaking 50 Degrees of Freedom (DOF), advanced AI capabilities, and its ability to tackle complex tasks like folding Spot robot legs, handling ropes, and even flipping barstools. Watch Atlas adapt to unexpected challenges in real-time, thanks to a 450M-parameter diffusion transformer that processes images, proprioception, and natural language prompts. Learn how Boston Dynamics and Toyota are pushing the boundaries of automation with data-driven learning and teleoperation.
Boston Dynamics and Toyota Research Institute just unveiled several breakthroughs,
but how close are they to realizing real-world, general-purpose humanoid automation?
To start, these newest advancements center around developing large behavior models
and deploying them on Boston Dynamics' Atlas humanoid.
At the heart of this new large behavior model is a 450 million parameter diffusion transformer
architecture, which allows Atlas' AI to ingest not only image streams at 30 frames per second,
but also proprioceptive signals to track body position in space,
as well as natural language prompts to specify the robot's goals.
With all of this information, the model then predicts sequences of its own future actions
with timestamps for each portion, allowing for flexible and adaptive behavior.
But yet another breakthrough feature is the model's ability to adjust the speed of execution
at inference time, all without the need for retraining. For example, the same policy can
be run at regular speed, or even three times the normal speed, with minimal loss of performance
in tasks. Importantly, this kind of flexible timing control is what will allow robots to
work faster than human teleoperators, especially for repetitive or highly optimized tasks.
And to demonstrate the practical value of large behavior models,
Boston Dynamics demoed a complex, multi-step task known as the Spot Workshop,
where Atlas is asked to do a three-step task.
First, Atlas grasps and folds the legs of spot robots from a cart and places them on a shelf.
Second, the robot retrieves face plates from the same cart and then opens a bin on a lower
shelf to deposit them inside. Third, once the cart is cleared,
Atlas turns and transfers all remaining spot parts into a blue bin behind it.
But the twist is that each subtask is actually initiated by a high-level language prompt,
and Atlas coordinates its locomotion, body posture, and fine manipulation to complete
the entire sequence. Incredibly, the policy is so robust that it even handles unexpected events,
including parts falling on the floor or the lid closing unexpectedly.
This is achieved by incorporating examples of failures and recovery into the training data,
allowing the model to learn to estimate the state of the world from its own senses,
and then react intelligently, all without requiring explicit programming for every
possible contingency. And Atlas's range of abilities is only expanding,
with the robot no longer being limited to rigid object manipulation.
Instead, the team has already benchmarked the robot on dozens of complex tasks,
including rope tying, flipping barstools, unfurling tablecloths,
and even handling a 10kg car tyre. Impressively, tasks involving rope,
cloth, or flexible materials are particularly challenging due to their unpredictable shapes
and dynamics. But because the learning process is driven by demonstration, Atlas can simply
leverage its large behaviour model to bridge the gap. This is thanks in part to Atlas being
equipped with 50 degrees of freedom to specifically focus on manipulation. Each gripper features 7
degrees of freedom for diverse grasping, including power grips and pinch grips.
Plus, a pair of high-dynamic range stereo cameras integrated into the robot's head
provide critical visual input for situational awareness and policy execution. In fact,
until now, robots built for generalist capabilities have faced steep challenges
because of the need to handle a wide range of objects, adapt to unpredictable environments,
and recover gracefully when things go wrong. But Boston Dynamics believes the most promising way
to achieve general competency is to simply train Atlas on vast and varied experiences,
much like how large language models learn from massive text datasets. In fact, their goal is
to build an automation platform where robots can not only learn but also adapt and even generalize
across a multitude of manipulation and locomotion tasks, ultimately scaling up for real-world
general deployment. So to achieve this, Boston Dynamics and Toyota Research Institute have
officially joined forces to focus on the development, training, and evaluation of large
behaviour models. They believe these neural network policies will allow Atlas to perform
long-horizon, end-to-end tasks, all conditioned on images, the robot's internal sense of joint
positions and forces, and natural language commands. And to do it, Atlas learns step-by-step
in four key stages, starting with data collection. High-quality data is gathered using both real-world
robot teleoperation, where a human operator guides the robot through complex interactions,
and simulated environments. This dual approach ensures a diverse and robust training dataset,
capturing everything from fine finger movements to full-body reconfiguration and whole-body
locomotion. Once collected in the second stage, the data is carefully processed and annotated,
making it suitable for feeding into large-scale machine learning pipelines. In stage three,
neural network policies are trained on all of this data, learning to map sensor inputs,
including vision, proprioception, and language prompts, to a sequence of control actions for
the robot. In the fourth stage, the trained policies are tested across a comprehensive
suite of tasks. Insights from these evaluations inform what new data should be gathered next,
and which architectural tweaks could further boost performance. Incredibly, this rigorous
loop of data collection, model training, and iterative refinement has allowed the research
team to rapidly improve the capabilities of Atlas, enabling it to handle tasks that would
have been considered out of reach for humanoid robots just a few years ago. On top of this,
Boston Dynamics has developed a state-of-the-art teleoperation system to collect demonstration
data for model training. Operators use a virtual reality headset and body trackers to immerse
themselves in the robot's workspace, directly controlling Atlas with precise, intuitive
movements. Earlier versions of the system allowed operators to perform a variety of tasks while
standing still, but later upgrades introduced full-body tracking, including the feet,
allowing Atlas to step, crouch, reach, and assume wide stances as needed.
This teleoperation interface is not just a tool for demonstration, but also an important
benchmark for evaluating the performance of the policies. Furthermore, the neural networks that
power Atlas are designed to use the same control interface as the teleoperation system, ensuring
a seamless transition from human-guided demonstration to fully autonomous execution.
And looking into the future, this data-driven approach to learning is general enough to be
used for almost any downstream task that can be demoed via teleoperation.
While Boston Dynamics has a strong start, there's still a great deal of work to do.
Boston Dynamics will use their established baseline of tasks and performance to focus
on scaling their data flywheel to increase throughput, quality, the number of tasks,
and their difficulty, all while also exploring new algorithmic ideas.
In fact, Boston Dynamics is pursuing multiple research paths,
such as robotics performance areas like tactile feedback gripper control and rapid dynamic
manipulation, integrating varied data sources like cross-embodiment and human-centric ego data,
advancing reinforcement learning for vision-language action models, and implementing
vision-language and vision-language action architectures to support more intricate long
horizon tasks and flexible reasoning. Altogether, Atlas has just unlocked a new
level of AI automation in the real world to work in industrial settings, but how long do you think
it will take for the company to make the transition into the home robot market?
Tell us in the comments below what tasks you'd want your Atlas to be capable of out of the box,
and how much would you be willing to pay for it? Anyways, like and subscribe for more of the
latest AI news, and make sure to check out this video here for more on Atlas' NVIDIA hardware.