Controller-free gaming for XBox 360

nature video
11 followers
2 Video Views·Sep 5, 2023
Project Natal gives XBox 360 players the ability to manipulate on-screen characters via natural body movements. In this interview for Scientific American, Microsoft reveals the science behind controller-free gaming. Read more here: http://www.scientificamerican.com/article.cfm?id=microsoft-project-natal
Project Natal is essentially Xbox 360's controller-free gaming,
where essentially all you need is life experience.
Just jump in, have fun, no controllers necessary.
We essentially are able to look at the scene
and fundamentally understand what is going on
and fundamentally hear what's going on.
And based on those two inputs,
we are able to come up with a whole new set of experiences
and a whole new way of telling stories
that has never been done before.
What excites me most about my job?
What doesn't excite me?
I mean, this is literally every child's dream
to be able to come into work every day
and fundamentally change an industry.
We are right now going through one of those fast
and furious transition periods.
If you are lucky enough to get to work in that transition period
and help the industry move from a world that was
to the world that will be,
it's a once-in-a-lifetime opportunity.
The world that was is this world where humans have to understand
the languages of machines.
The world that we are beginning to paint is a world
whereby essentially machines understand the languages of humans.
We cannot sit and program a reaction to a reaction.
We have to rethink the paradigm
and essentially teach the computer to perceive the world.
And that really translates into really teaching the sensor
to classify the world much like you do.
The first one is just going to people's living rooms,
go to dorm rooms, anywhere where people are likely to have the device.
We go in and we essentially film terabytes and terabytes
of users in their natural habitats, right,
in their natural environments.
And there's also two other ways of gathering data.
One is the mocap one, where we actually just go down to Hollywood land
and we sit in mocap studios and we essentially film people
much in the same way that you would Gollum in A Lord of the Rings kind of thing.
Now, if you think about living room as being the natural environment,
this is a synthetic environment, right,
where I essentially have actors sitting here.
I have a director.
And much like a movie, we have a set of shots they need to go through.
I'll show you one in the dark so that you can actually see the joints placed on them.
And this is the benefit of mocap.
At the end of the day, what you care about in a human is not the body parts.
It's the actual joints.
I'm doing a racing game.
I'm tracking your wrists or your elbows or your shoulders or all of the above.
What if I'm a woman and I'm wearing a skirt and I want to play soccer?
For you to play soccer, you're going to have to be able to see your kneecaps and your feet.
Now, if you're wearing a skirt, how do you do that?
Does that look like, from the perspective of the brain, someone wearing pants?
No.
At that point, you're going to have a lot of cloth over it,
and your knees are going to be essentially what you see through the cloth deformation.
I put my knee forward.
I'm deforming a piece of cloth.
And now the signal here is essentially very different than a guy wearing pants and doing this.
So in terms of training, not only do we train across body types,
we also train across different attire.
We also train across different hair characteristics, hats, not hats, beard and not beard.
The bulk of the time we spend is creating a platform that creates a brain that can classify and understand the real world.
If we can create a brain that can understand the real world, you can create any game you want.
Because you're guaranteed that anything you can come up with, you can actually do.
Which then boils down to, hey, all you need is life experience.
As long as you know how to do something in the real world,
that will replicate one-to-one to what you want to do in the game world, in the digital world.
We understand the historic moment in time.
We get it.
We're super excited about it.
The world a few years from now will be dramatically different.
Now Taoist journey is going to take us a long time to completely change an entire computer industry.
It starts in interactive entertainment.
It starts in Xbox, and that's our focus right now.
But we do see incredible applications from a productivity perspective, from a healthcare perspective,
from an enterprise perspective, you name it.
It's hard.
As soon as you change the paradigm, right?
Can you say I can fundamentally understand humans?
That's widely applicable anywhere, everywhere.
