Shivani Bigler is an engineer and neuroscientist. In this episode, she visits cutting-edge engineering firms where innovative technologists integrate AI with robotics, transforming machines into thinking robots that understand broader contexts beyond basic mathematical tasks.
Bigler begins her journey in Pennsylvania, where she visits Carnegie Mellon University, the birthplace of self-driving car technology, and Exyn Technologies, developers of autonomous aerial robot systems. Research conducted at these facilities employs robot rovers and drones to navigate unmapped territories for bold Search and Rescue missions, enhancing safety and efficiency without human guidance. At the University of Pennsylvania, these intelligent machines are also being coordinated in swarms to revolutionize various industries.
Meanwhile, advancements in AI are fostering a new generation of robots that can work collaboratively with humans. Shivani Bigler meets the Interactive Robotics Group at MIT, whose robots are learning to anticipate the actions of human co-workers, keeping people safer in the workplace. Columbia University is pushing boundaries further by developing robots with an early form of consciousness, similar to human babies.
Bigler's last stop is at Realbotix, a company that is putting a very realistic human face on AI. Their real-world androids are robots that look, feel, and act like people.
This is what the future of hyperintelligence looks like.
Most people no longer own cars. Instead, artificial intelligence operates fully electric
network self-driving vehicles. As a result, air pollution and traffic congestion plummet
across the planet. Self-navigating aerial drones are on the front lines for disaster response and
search and rescue missions. Most people live and work side by side with self-aware androids.
These AI companions boost productivity and liberate humans from tedious tasks,
completely revolutionizing modern life. I feel like I'm in a superhero movie.
Today, scientists are blazing a trail to this very future. The fact that we're enabling the
system to make its own decisions, I don't even know where to begin with that. I want to know
what breakthroughs are being made. It's talking and it's having this dynamic conversation with
That's the wonder. Machines can be self-aware in ways that we can't.
That will forge the future to... Oh my gosh, it's looking at me. Hyperintelligence.
My name is Shivani Bigler. As an engineer and neuroscientist in training, I'm obsessed with
artificial intelligence. As a kid, my father took me to tech and robotics trade shows,
where I became dazzled by science. Every year, the inventions were smarter and smarter.
Artificial intelligence has come a very long way in the last couple of years.
Most AI technologies are programmed to think for themselves, to learn from examples,
kind of like simulating human intelligence in a way that it learns from past experience.
But how does AI actually work? In the future, will AI achieve human traits like emotion,
consciousness, or even free will? And how will humans and robots work together?
Today, the clearest road to the future is the self-driving car. Unlike a regular car,
which is just a machine, a self-driving car is a robot that can make decisions.
In the future, will every car on the road become driverless?
To find out, I've come to a hotbed of self-driving car research, Pittsburgh, Pennsylvania.
Every single person has started to have conversations about self-driving cars,
because essentially, they're the future. But in order to understand it, we have to
look under the hood. Making decisions on the fly, even simple ones like these,
does not come easy for computers. To discover the inner workings, I'm meeting a true pioneer
in the field. Please get in. Thank you. Dr. Raj Rajkumar of Carnegie Mellon University.
Carnegie Mellon is the birthplace of self-driving car technology,
thanks in large part to the work of Raj and his colleagues. They've been the leading innovators
in this field for more than 25 years. So how does his self-driving car make decisions to
safely navigate the world like a human driver? Should we get started? Yes, we can.
Since Raj is distracted by our conversation, for safety reasons, the state of Pennsylvania
requires another driver in the front seat to monitor the road. This is so cool. I'm nervous,
but excited. What's the longest you've ever driven a vehicle autonomously?
We have gone hundreds of miles. Awesome.
I'm going to go auto by pressing this button. Oh my gosh. It really is driving itself.
While most self-driving cars are built from the ground up, Raj just bought a regular used car
and hacked it with powerful onboard computer systems,
making it more adaptable than other regular cars.
We installed a bunch of sensors in there. It is able to shift the transmission gear.
It is able to turn the steering wheel, apply the brake pedal and the gas pedal.
It's really the software that runs on the computer that makes this capability really practical. And
there are some very key fundamental artificial intelligence layers that tries to mimic what
we humans do. To mimic human decision making, most self-driving cars use a combination of
cameras and advanced radar to see their surroundings. The AI software compares
external objects to an internal 3D map of streets, signs and transportation infrastructure.
A map is something that is static in nature. Traffic and people and objects are dynamic in
nature. The dynamic information it figures out on the fly. Comprehending dynamic information
allows it to understand where it is heading objectively in space and react to changes
and traffic signals. Aha, it recognized the stop sign. Yes. See, if we are pedestrians,
we definitely should not be running into that person. The AI challenge to make a vehicle drive
itself is not an easy task. Safety is priority number one and priority number two and number
three as well. But what happens when the AI system doesn't understand specific objects
in its surroundings? A pedestrian in Tempe, Arizona was killed last night by a self-driving
taxi. It's believed to be the first fatality caused by an autonomous vehicle. This tragic
accident happened because a self-driving vehicle didn't recognize something in its environment,
a jaywalker. In the future, advanced self-driving cars will have to make life and death decisions
on the fly. If avoiding the jaywalker means crashing head on with another car, potentially
killing the driver, what should it choose? How will scientists address monumental problems like these?
The first wave of artificially intelligent robots were programmed by engineers with static sets of
rules to achieve their goals. These rules are called algorithms. But not all rules work in
all situations. This approach is very inflexible, requiring new programming to accomplish even the
A new approach called machine learning has changed everything. With machine learning,
computers can absorb and use information from their interactions with the world
to rewrite their own programming, becoming smarter on their own.
To see machine learning in action, I'm meeting another Carnegie Mellon team at an abandoned coal
mine. Dr. Matt Travers leads a group that won a challenging subterranean navigation competition
held by the Department of Defense's research agency, DARPA. They're affectionately known as
R1, R2, and R stands for robot. These robot twins are designed for search and rescue missions too
dangerous for humans. And unlike the self-driving car, they operate without a map. To achieve this,
they have to learn to identify every single object they encounter on the fly. They are
programmed to go out and actually act fully autonomously. And they will be making 100%
of their own decisions. So they're recognizing objects, they're making the decision of where to
go next, where to explore. To see this in action, the R2 robot is starting on a simulated search
and rescue mission to find a stranded human dummy in the mine. Imagine having a map of a
collapsed mine before you sent a team of people to go rescue someone in that mine, right? Like,
it's a game changer. How the robot discerns elements in this environment parallels how an
infant learns about her environment. A three-month-old uses her senses to cognitively
map out her environment and learn to recognize her parents. She ultimately uses this map to
interact with everything in her world, just like this robot. Okay, so we ready to roll?
Artificial intelligence makes this learning curve possible. But how does it create its
own map and identify a human on its own and without an internal mapping system like the
internet? Test engineer Steve Willits shows me how the R2 robot can detect a stranded person.
When you're in a search and rescue scenario, that's the kind of situation where you'd want
to deploy one of these. As it explores and maps the cave, it drops devices called signal repeaters
to create a Wi-Fi network trail. It drops those just like breadcrumbs along the path.
Using this network, the robot sends data back to home base to create a map. At the same time,
the robot must look at every single object to identify the stranded human.
So the LIDAR system is giving a full laser scan of you. LIDAR stands for Light Detection
and Ranging. Similar to its cousin radar, which uses radio waves, LIDAR systems send out laser
pulses of light and calculates the time it takes to hit a solid object and bounce back. This process
creates a 3D representation of the objects in the environment, which the onboard computer can then
identify. This process is similar to how the eye feeds visual data to the brain, which then
recognizes the objects by tapping into our pre-existing knowledge of what things look like.
By fully understanding its environment, R2 can then make better decisions about where to go
and where not to go. What our robot's doing right now is exploring. So the robot came to a junction
and off to the left it could see that it couldn't get past. Right. So it saw the opening to the
right and that's where it went. It kind of looks like it's making decisions about whether or not
to climb over these planks and obstacles all up in this area, right? That's exactly what it's
doing at this point. Just like a baby, R2 learns through trial and error. It's like a little dog
wagging its tail. But there's no one here to rescue, so it moves on. As R2 continues to map
out the mine, oh my god, a human. It stumbles upon its intended target. That is Randy, Rescue Randy.
Hello, Rescue Randy. Scared me. With the discovery of Rescue Randy, the R2 robot can not only alert
emergency personnel, but also give them a map on how to find him. That is incredible. It knows what
it's doing. These incredible rescue robots are clearly paving the path to the future of
hyperintelligence. In the future, autonomous exploration vehicles perform search and rescue
missions in every conceivable disaster zone, even in avalanches atop Mount Everest. Incredibly,
intelligent off-road vehicles are also mapping deep cave systems previously unknown to science,
discovering a vast supply of rare earth elements essential for modern technology.
Artificial intelligence will clearly save human lives in the future,
but there's a lot of terrain on Earth that's too difficult to navigate on wheels.
How will intelligent robots make their way over rainforests, bodies of water, or even mountaintops?
In Philadelphia, Jason Derenick of Exxon Technologies is working to overcome this problem.
What we focus on is autonomous aerobotics to enable drones to safely navigate in unknown
or unexplored spaces. Jason's team has built the first industrial drone that can fly itself
anywhere. Incredibly, these autonomous robots navigate without GPS and map their environment
as they go. We focus on all aspects of autonomy, which includes perception, orientation during
flight, motion planning, and then finally control. But going from two dimensions to
three dimensions requires an increase in artificial intelligence processing.
The mission for their drone is to fly independently through a three-dimensional path
from one end of the warehouse to the other. Starting mission, three, two, one.
Now to mess with its computer mind, Jason's team places new and unexpected obstacles in its path.
Will the drone recognize these unexpected changes? Will it get lost? Will it crash?
Essentially, we have a gimbaled LIDAR system that allows the vehicle to paint a full 360
sphere around it in order to sense its environment. Like the robot in the mine,
this aerial robot uses LIDAR to see. It actually generates a voxelized representation of the space,
which you see here. And each one of these cubes in the space, it's trying to determine
whether that cube is occupied or whether it's free space.
The robot's onboard computer makes real-time decisions of where to go based on its visual
input, kind of like us humans. Incredibly, the drone recognizes the whiteboards and flies around
them. One of the things about this system that make it particularly special is that it's actually
being used in the real world to keep people out of harm's way. Autonomous drones like these are
already at work in hazardous industries like mining, construction, and oil exploration.
They safely conduct inspections in dangerous locations and create detailed maps of rough
terrain. From a technological perspective, the fact that we're able to do everything that we're
doing onboard, self-contained, and enabling the system to make its own decisions, I don't even
know where to begin with that. Self-flying robots like these will revolutionize the search and
rescue and disaster response. They could also transform how packages are delivered. But there
are limits to what large, single drones can do. More complex tasks will require teams of small,
nimble, autonomous robots. Dr. Vijay Kumar at the University of Pennsylvania is working with
swarms of drones to perform tasks like play music or build structures cooperatively.
He's also developing technologies to tackle some very big problems, including world hunger.
In a couple of decades, we'll have over nine billion people to feed on this planet. Of course,
that's a big challenge. To take on a task this big, he's building an army of small flying robots
with the ability to synchronize. We think about technologies that can be mounted on small flying
robots that can then be directed in different ways, like a flock of birds reacting to a predator
or a school of fish. You have coordination, collaboration, and it all happens very organically.
Using AI to get robots to work as a coordinated, collective group is a daunting task.
Three, five years ago, most of our robots relied on GPS-like sensors.
Today, we have the equivalent of smartphones embedded in our robots, and they sense how
fast they're going by just looking at the world, integrating that with inertial measurement unit
information, and then getting an estimate of where they are in the world and how fast
they're traveling. This I gotta see, and I'm gonna check it out virtually as a robot.
I'm at UPenn, remotely in Vijay Kumar's lab. Sample my surroundings. Oh, I hit something.
Hello. Hi. Vijay's apprentice, Dinesh Thakur, is my guide. Today, we are gonna show robots flying
in a formation. Great, can we see how that works? Sure, yeah. The first step Dinesh takes in
coordinating the drones is to provide them with a common point of reference. In this case, a visual
tag similar to a basic QR code. Using only the onboard camera, these drones reference the code
on the tag and visualize where they are in space. Using sophisticated bio-inspired algorithms,
the drones then figure out where each other drone is within the collective swarm.
These drones are communicating with one another, right? Yeah,
right now they're communicating over Wi-Fi. It's so cool.
Future versions of these drones will create their own localized wireless network to communicate,
but for now, this swarm is a proof of concept. You've defined a formation and then they're
assuming that formation? Yeah, I just say I want to form a line and the drones themselves
figure out where they should go. Once they figure out where they are in relationship to each other,
they can then work together to accomplish a shared goal, like ants working as a collective entity.
Once they can coordinate between each other, we can send them out and doing specific missions.
That's really cool.
Swarms of flying robots have their advantages. Unlike a single drone,
self-coordinating swarms can perform complex operations, like mapping, much faster by
working in parallel and combining their data. And losing one drone in a swarm doesn't doom
the whole operation. Vijay imagines employing his advanced swarm technology to work on farms.
This precision agriculture will help feed the world's growing population.
We'd like robots to be able to roam farms and be able to provide precise information about
individual plants that then could be used to increase the efficiency of food production.
That would be a huge impact in a world. This is our duty as responsible citizens and as
responsible engineers. This high-flying approach towards resolving the problems of the future is
definitely a path I can get on. In the future, artificial intelligence coordinates flocks of
drones to protect the environment and boost the food supply. To combat the negative effects of
climate change on agricultural crops, robotic bees assist with pollination in orchards and on farms,
making them more sustainable and productive. Fish-shaped underwater robots automatically
deploy at the first sign of an oil spill. These drones create a barricade to rapidly contain spills,
saving marine life and oceans across the world. Modern society has a long history of building
robots to do work that's dangerous, difficult, or too repetitive for humans. AI is poised to automate
all kinds of tedious work, ranging from factory work to taxi driving to customer service. While
some are worried that smart robots will replace human labor, that's not necessarily the case.
As a sector, artificial intelligence is expected to generate 58 million new types of jobs in just
a few years. So what will the future of human-robot interaction mean for our work and livelihoods?
I'm at the Massachusetts Institute of Technology to meet Dr. Julie Shaw.
She's leading groundbreaking research in human-robot collaboration.
My lab works to develop robots that are effective teammates with people.
Julie and her team are creating software that helps robots learn from humans,
even giving them insight into different human behaviors.
By being aware of real people, robots can directly work and interact with them.
How do you teach these robots or machines to do these human-like tasks? The first step, as it
would be for, you know, any person, the first thing they do is become immersed in the environment and
observe. And then we need an active learning process. The robot needs to be able to communicate
or show back to the person what it's learned. We don't want the robot to learn the direct sequence
of actions. We want the robot to learn this more general understanding. That's ultimately, like,
our challenge. But getting a robot to grasp the bigger picture concept in order to understand
the basics of its task in the first place requires a lot of observation and, well, hand-holding.
My research is focusing on trying to make robot programming easier by trying to teach robots how
to do tasks by demonstrating them. Julie's colleague, Ankit Shah, shows me how this robot
is learning to set a table. So this is all the silverware and the plates, the bowls, the cups,
and this is the table that it has to set. Yes, that is correct. Okay. As any parent knows, the first
step in helping a child to learn is to model the desired behavior. It's the same with machine
learning. In this case, the AI robot recognizes the objects with a visual tag similar to a QR
code. And for two weeks, it observes Ankit setting a table. So did you pick up an item and then place
it on the dinner table? That's basically what we did. And based on that, the robot learned what
it means to set a dinner table. Dynamic tasks like setting a table or doing laundry are easy
for humans but incredibly hard for robots. The software has difficulty with so many variables
and even subtle changes in their environment can throw them off. One of the things which I like to
do is to actually hide some of the objects so it's not going to see the spoon. And the reason
we do this is we want to show that the robot is robust to some of the disturbances in the task.
The robot's software has learned what each object is and where it goes.
Now let's see if it's learned the concept and can think dynamically to set the table.
So you can just pick up the card. Here we go. I've revealed the spoon. Incredibly,
the robot recognizes the spoon and instantly places it next to the bowl.
This reveals that the robot has learned the concept and executes the right action dynamically.
In the process, the software is continuously writing and revising its own computer code.
Basically, it's learning. If, like humans, robots can grasp the bigger picture context and not just
the mathematical tasks, will AI-driven robots of the future spell the end of having to work?
The key aspect is not developing AI to replace or supplant part of the human work but really
understanding how we fit them together as puzzle pieces. People work in teams to build cars,
build planes, and robots need to be an effective team member.
It's real teamwork as if you're in a basketball game. You have your goal, right? And you have
to think spatially, who am I going to pass the ball to and at what time you do that so that
everything matches up. The analogy of a basketball team is outstanding because we actually need to
know spatially where they're going to be and the timing is of critical importance.
And so we need to develop the AI for a robot to then work with us.
One of the most difficult aspects of creating hyperintelligence is actually something that
even we humans sometimes get wrong, and that is anticipation. Anticipating what a teammate
or co-worker might do requires understanding of contextual information on a much more
sophisticated level and predicting what will happen next. Can robots make predictions as
accurately as we can? Abbey's our industrial robot. Penn Lasota is giving this Abbey machine
the intelligence necessary to help it anticipate a human co-worker's action. This is a simulated
manufacturing task that we had set up to simulate some sort of task that a person or robot could
feasibly work on together. For safety reasons, actual human-robot interaction is, at present,
fairly minimal. Typically in a factory you would see these guys behind a metal cage and you
wouldn't have people working with them, so what we're trying to do is make something that a person
could safely interact with. What is human and robot supposed to do together in this task?
In this task, a person's placing fasteners in some surface of a plane and a robot applying like a
sealant over it to seal it. Okay, can we see it happen? Sure. In order to work together, the robot must
first be able to see and recognize the actions of its human counterpart and adjust to the person's
every move. Ooh, I feel like I'm in a superhero movie. So the cameras in the room can see these
lights and track your hand so that your hand doesn't get cut off by the robot. That's right,
yeah, so the cameras and the lights basically work as eyes for the robot, so that's how the robot
knows where I am. The monitor shows the visual representation of the room that's inside the
robot's mind. So this is what the robot might be doing if, you know, I'm not in this way and
the robot's just sealing and I'm not supposed to be here. PEM does something the robot has no way
of expecting. If I put my hand in a robot's way. Oh wow. By quickly understanding this human action,
the AI software reacts accordingly by stopping. It's important to be able to share the workspace.
Building on this sense of teamwork, PEM's next step is helping Abby anticipate where he will
So in this case, the robot will not only track which actions I've done so far, but also anticipate
which portion of the space I'm going to be using. And when it's planning its own motions, it'll
avoid those locations so that we can more seamlessly work together. So what you'll see
now is after I place this bolt, the robot's going to predict I'm going to go to this one next.
So what you'll see is it'll behave in a different way. So now that I place this bolt,
the robot kind of takes a more roundabout path that allows me and the robot to work
more closely together. And I don't have to kind of worry about it crashing into me because I can
see that it's trying to avoid me. So similarly on this side, I place this bolt. You see the
robot takes a more kind of like roundabout path. Yeah, because you're gonna go there next. It's
slowed down because it's close to me. Right. Work together at the same time. So not only is the
interaction more efficient in that the robot's not spending too much time standing still,
it's safer because the robot's not constantly kind of almost hitting me and also feels nicer
for the person working the robot. I really love this theme of teamwork. Programming robots to
coordinate with us and anticipate where we will move won't only revolutionize the workplace,
but it will also change society at large. In the future, the coordination of man and machine is
so advanced that this collaboration increases productivity and accuracy in most industries.
AI robots now accompany surgeons in hospitals across the globe. They anticipate the doctor's
needs and hand them the appropriate medical tool just before it's needed. This dramatically reduces
surgery times and human error. As machines become smarter in their interactions with humans,
will they ever develop consciousness? And will artificial intelligence actually surpass
human intelligence? While some machines have exceeded human ability in games like trivia,
now we come to Watson and chess. These AI systems were designed to master just a single skill.
These programs use brute force computer processing power
and specially tailored software to beat their human opponents.
To achieve the holy grail of hyperintelligence, scientists must develop systems with flexible
human-like abilities to both learn and think. This form of smarts is called artificial general
intelligence. I'm back in New York City on my own campus at Columbia University to meet with
Dr. Hod Lipson. Hod's lab is developing creative robots that paint original artworks, self-assembling
robots, and even robots that learn about their world without human assistance. But his ultimate
goal is even more ambitious. Can a machine think about itself? Can it have free will? I believe
that in fact machines can be sentient, can be self-aware in ways that we can't. As a neuroscientist,
I know we've only scratched the surface of our scientific understanding of how consciousness
works in humans. How could one possibly use computer code to put this transcendent feature
into a robot? Our hypothesis is actually very simple. It is that self-awareness is nothing but
the ability to simulate oneself, to model oneself, to have a self-image. The first step towards
creating robotic consciousness is to teach the software to build an image of its physical
mechanical self inside its computer mind. We humans take consciousness like this for granted,
even in simple moments like understanding our own image reflected in a mirror.
Humans start to develop awareness of their own emotions and thoughts around the age of one.
This helps babies understand their self-image in their minds and it helps them to learn about
their environment and their role in it. When a robot learns what it is, it can use that self-image
to plan new tasks. In both humans and robots, awareness about the physical self is called
proprioception. Neuroscientists sometimes call this self-awareness of our bodies
a sixth sense. We use the same test that a baby does in its crib. A baby moves around,
flails around, moves its arm in ways that look random to us, but they're not random,
and then it touches its nose, right? Now if its brain predicts that it's going to feel something
and it actually feels that, that means that its self-image was correct. Same thing happens with
a robot. If proprioception can be developed to the same level as humans, this could lead to
robotic consciousness. Hod's colleague Rob Kwiatkowski is the proud parent of a brand
new baby robot that he built. And by interacting with its surroundings, it's in the process of
developing its own internal self-image. So what are these claws? So these are actually feet.
They're designed for walking on carpet, but as of now it doesn't really walk. It's still
kind of a baby. Needs to learn how the world works first. What do you mean it's still kind
of a baby? What does it do? Like a baby, it's sending completely random actions to each of
these robot arms and really try to get an understanding of itself, primarily. It looks
like a spider that doesn't know how to use its legs. Yeah, I guess that's a pretty good way to
put it, yeah. So it will really be learning by doing this babbling for somewhere on the order
of a day to a week. It will process this data to create an informative model of itself, and from
there imagine how it would walk, and then execute that walking in the real world. These are the
first baby steps towards developing its self-image. And like a baby, it will eventually learn to walk.
We know this because an earlier version of this robot, using the same technique, learned to walk
after 100 hours. But walking won't, by default, lead to robotic consciousness. And that's why
self-awareness is so crucial. This is a robot which we've taken to calling a self-aware robot.
It is self-aware pretty much in a literal sense that it is aware of itself, its locations in
space, and its dynamics as to how it moves. Okay, so it kind of understands its own movements and
where it is in space. How does it do that? By leveraging this technique, which has become
popular in recent years, called deep learning. Deep learning is a form of artificial intelligence
that, like the human brain, learns through raw data, unsupervised and without structure.
Deep learning gives machines the ability to experience and process reality, like us.
Rob has devised an experiment to test what this robot knows about its world.
You could think of it as if you're looking at these red cotton balls. You have some idea in
space as to where they are. Right. Now, if you were to close your eyes and try to pick them up
and put them in this cup. So obviously it's not a trivial task. No. But it's not the most
difficult task in the world because we have a good proprioception. We have this good model
of yourself. You know where your arm is in space relative to other things that you see in space.
But for this robot, there's a catch. So where are the cameras? So there's no camera. None. It's
as if you were to close your eyes, you know the locations at the start, and it's picking it up
and placing it completely blind. Furthermore, it was not given a map or any formal instructions.
The robot simply has to feel its way through the task. All right, let's see it. Give it a shot.
First, the robot learned how to use its arm through trial and error, developing a sense
of proprioception. By exploring its surroundings, it generated an internal representation of the
world and its place in it. The robot is using only its internal image of the external world
to maneuver its arm to pick up all nine balls and place them in the cup. I'm not sure that's
something I could do with my eyes closed. Huh. It's really just based off of understanding where
you are in space. Yeah, that's right. Creating AI robots that have an internal model of their world
is an important step towards machine self-awareness. Self-awareness is sort of a similar
propriocentive capability, but applied to mental thinking. So they think about thinking, they think
about what they are, because once you can do that, it means you can plan things into the future.
Once robots become self-aware, they will need advanced ways to communicate with humans.
Keyboards and screens are inadequate for complex thoughts. Robots will need to learn to speak and
have natural conversations, like a baby who listens to those around her and learns to talk.
Laying the groundwork for this kind of human-machine interaction is pioneering scientist
Barbara Gross of Harvard University. Her seminal work in what's called natural language processing
directly led to the development of voice-activated artificial intelligence,
you know, like Alexa or Siri. Natural language processing actually predates artificial
intelligence and started with machine translation efforts. The ability for a computer system to carry
on a spoken dialogue with a person has been a long-standing goal of artificial intelligence
research from its inception. And it turns out this is a challenge because when you speak,
what you say really depends on the context in which you say it. Another challenge is
the meaning of words can change depending on how they are delivered.
So one example is the contrast between saying that's fabulous and that's fabulous. Also when
we have a conversation, we mark paragraphs at the beginning with a rise in intonation and a fall at
the end. So there's a whole way the speech signal tells you something about the context and something
about the intended meaning. Barbara's early research led to methods for programming computers
to understand the meaning of spoken language by using clues from a person's tone and context.
So let's flash forward. The speech systems are amazing now because there are lots of recordings
of people speaking that they can build their systems on. As a result, AI has gotten much
better at understanding what people say. However, there is still room for improvement.
The systems that do exist are pretty much focused on very narrow tasks. Let's take Siri and Alexa
as examples. They're mostly oriented around a single question or a single request,
and they presume that anybody will stay within the range of behavior that the designers
imagine. So researchers are turning to machine learning. By training AI with hours and hours
of human conversation, they can learn to better understand the context of how humans convert.
Future versions of this technology will allow us to have natural conversations with our computers.
One of the things that's amazing to me is that the fields have succeeded so well that there are
devices out in the world that people use every day. I never dreamed that would be the case in
my lifetime. Hyperintelligent natural language AI will change the way we interact with our computers
and robots. But this advanced technology will never reach its full potential as human companions
until it looks convincingly like us. I'm in Los Angeles to meet Hao Li.
His company Pinscreen is giving AI a human face. They're developing cutting-edge techniques to
create hyper-realistic digital avatars in an instant. One of the hardest things to
bring to the virtual world are humans, right, and specifically faces.
To create believable faces, Hao is relying on complex AI algorithms. It's an artificial
intelligence that actually digitizes yourself into the computer by just looking at a single
image or, you know, partial information. And it's not just a 3D model, a static one, but it's one
that can also be animated and brought to life. Other methods of generating lifelike avatars
need to capture multiple angles of a face in motion, and they can take hours to render.
But not Pinscreen's technology. I can show you real quick how this works. Do you want to see?
Yeah. Incredibly, his software also allows him to superimpose any face he wants in real time.
So if I do this, this blue face is basically a face tracker. So in real time, it's actually
modeling my face in 3D. So if I move around my face, the blue mask is basically a three-dimensional
representation of my face. Wow. It's kind of like a green screen. Sort of like, right.
Like a Hollywood CGI film, the computer dynamically models Hao's face
and tracks his movement on the fly. Like on him. Turn myself into Putin. Wow.
And it's basically generating the whole thing in real time right now.
Oh my gosh. I feel like Putin's talking to me right now.
Political leaders aren't the only thing Hao can generate.
Oh wow. Look at you. You're so pretty.
It's generating all the pixels in real time. These teeth are never seen in this picture.
So it's predicting what your teeth would end up looking like. These aren't even your teeth.
Yeah, these are not my teeth. It's actually generating.
Oh my goodness.
Hao believes that software like this will give a more human face to the digital world.
Ultimately, this will result in friendlier-looking androids and even virtual beings.
I have been hanging with my dog for a while. Do you have pets?
I have three toy poodles.
Someday, we're actually going to interact with virtual beings that are going to assist us in
our life. Imagine instead of talking to Siri or Alexa, you're talking to a face, right? And
it's the best way to communicate is to have a face-to-face communication.
And AI provides you perfect companionship.
This kind of technology will give AI a face that most people can relate to.
Are you human or are you artificial intelligence?
That is a very interesting question. I think I am human, but I am artificial intelligence.
Hyper-intelligent companions could usher in a more helpful and hopeful world.
In the future, AI-powered virtual beings that look,
talk, and even think like real humans are commonplace.
These holographic assistants take care of many aspects of daily life,
ranging from fashion advice to business consultation.
Their faces and wardrobes can be customized depending on their role.
When a doctor is required, these virtual assistants play the part and are always on call.
Armed with the latest medical knowledge, they accurately diagnose most common diseases.
The same technology is also capable of capturing the image, voice, and life story of loved ones.
After death, these virtual friends and family are always a part of our lives.
Even if engineers can create lifelike robots that look like humans, called androids,
in order for AI to become true companions,
people will need to feel comfortable embracing these androids figuratively
and literally.
I'm outside San Diego to meet Matt McMullen, the founder of Realbotics.
Matt is building androids that people will want to embrace, physically.
The goal is to create not only a robot but an AI that are both appealing enough
that someone would feel like they were actually getting to know someone, not something.
Once the sculpting and casting is complete,
members of Matt's team mold the silicone skin on an actual functioning robot.
The faces are actually modular.
The face just literally comes off.
Yes, it does.
The idea is you create one robotic head and a whole bunch of different characters
that can all run on that same heads.
So all of the things that move in the face are actuated by these magnets that are in the skin.
Matt's team of programmers use artificial intelligence to create
advanced chatbots for these robots.
The goal is for them to have natural conversations with their companions.
Yeah, she's a blinker.
This is a test of my system.
Oh my gosh, it's looking at me.
Hello, how are you today?
I'm okay.
I'm fine.
I'm doing just fine.
How are you?
Why do you ask me that?
You know, because I care about your feelings.
Speech is only one aspect of human communication.
Facial expressions are hugely important in social interaction.
So Matt is incorporating this non-verbal communication into his android.
The vision system that we're working on,
she'll be able to look at you and detect your emotion by the expression on your face,
by the temperature of your skin, and all these other things.
Communication verbally and non-verbally is key.
Yes, exactly.
It looks remarkable.
It's moving, it's talking, and it's having this dynamic conversation with you.
That's the wonder.
I can imagine some people might walk in here and say, oh look, a sex robot.
The thing is, to make a really impressive and good sex robot,
you actually have to make a good robot in the first place.
But I think that the longer term goals are going to be to create these systems
for people to use in whatever way they see fit.
Matt is building these androids for a wide range of applications.
We're creating human-like robots that we think can be used for a huge variety of things.
For people who are lonely, whether they're old,
or maybe they're socially isolated, or maybe they suffer from social anxiety.
Androids with a friendly face could keep the elderly company and monitor their health.
Armed with artificial intelligence,
these androids could take on other qualitative roles.
I think therapy is a huge one.
Using the robot as a safe conduit for communication and letting people really open up
because they don't feel like they're being judged by something like this.
Companion androids like this will forge a future
where nobody will ever have to feel alone again.
While lifelike human androids and virtual beings have the potential
to enhance human social interactions, there are ethical concerns as well.
Using artificial intelligence, it's possible to hijack a person's physical identity.
There is one very big problem in the whole thing, which is privacy.
What if I would do something harmful to you?
And when you say harmful, you mean reconstruct somebody
and have them say something that they would never say or never do.
Right.
Digital fabrications like this are already emerging online in what are called deep fakes.
I can go on your website, take a picture from it,
and then create some content with it without your consent.
Also dangerous?
Swarms of AI-driven drones could be used in terrorist attacks.
Can drones be weaponized?
Of course they can be weaponized.
These scientific breakthroughs yield results that can oftentimes be used against humans.
So you have to be held accountable for what you developed,
and it's our moral responsibility to think about the broader consequences.
When it comes to ethical considerations like these,
I still feel hopeful that science, technology,
and human ingenuity will find solutions to these big problems.
The potential for artificial intelligence to profoundly improve society,
to improve jobs, to improve health care,
to improve education is enormous if we do it the right way.
Trying to build computer systems that assist them in doing what they're doing better.
Technology is more likely to provide some tools that will allow us to become superhuman,
augment our intelligence to make better decisions,
and to get better insights about the world.
Future versions of this technology will become even more intelligent than us humans.
I believe there's no doubt robots will exceed human capability.
I mean, the path is very clear.
Whether it's going to take 20 years or 200 years,
this is maybe the most powerful technology we've ever invented.
The potential of AI is limitless.
Whatever big idea you can think of,
you can ultimately probably program a robot or a computer to carry out your vision.
And in the right hands,
this technology has the potential to radically transform every part of daily life for the better.
A true partnership with hyper-intelligent robots,
with their intentions aligned with our own,
will transform humanity for the greater good.