Details On Natal's Motion Capture Technology 121
An anonymous reader writes "Following yesterday's announcement of a late 2010 launch date for Natal, more details are emerging on exactly how Natal works. Alex Kipman, the project's lead developer, explains that Natal uses only 10-15% of the Xbox's resources to calibrate to a new player inside 160 milliseconds, track one or two players simultaneously, and use rudimentary knowledge of body anatomy to estimate where hands or other body parts are even when they can't be seen by Natal — for instance when they are held behind the back."
How obfuscated protocol? (Score:3, Interesting)
The aspect of how Natal communicates with X360 is most interesting to me, and surely many others.
It seems like it could be mighty fun peripheral for robotics projects... (and who knows, perhaps MS could even promote it, seeing that they have their robotics suite? Certainly small number of Natals not used for gaming wouldn't harm MS financially in any significant way; but they might like good publicity)
Re:How obfuscated protocol? (Score:4, Interesting)
I believe there was talk of the hardware working on the PC, too, though the hardware is pretty much just a few cameras and other sensors, leaving it up to the libraries on the Xbox or PC to do the processing. Now, if Microsoft releases an SDK (which is in their best interest, if they want PC integration) then it shouldn't be an issue.
Re: (Score:2)
It's not exactly "pretty much just a few cameras and other sensors". Yes, there is RGB cam and microphone array, which will be often useful in their own way.
But most importantly, there's this:
http://en.wikipedia.org/wiki/Time-of-flight_camera [wikipedia.org]
And cheaply. Which might be huge for amateur robotics projects.
SDK for Windows might be the best we can hope for indeed, if wanting something usable. Not exactly optimal, you're basically forced to use some Windows netbook (instead of some ARM board) if your creation is
Re: (Score:2)
Link: [microsoft.com]
"What you have to do is write a program that runs on the AVR (using for instance WinAVR) and this program talks to Robotics Studio running on a PC."
Re: (Score:2)
That's not really applicable to Project Natal usage, don't you think? Using MS Robotics Studio to control something is a different thing than using fabulous (for the price) sensors of Natal by putting it on a robot of some sort. And wondering now how easy/hard it will be to access those sensors, what will be required.
awesome (Score:2, Flamebait)
I think Natal is amazing -- and you won't need to replace the batteries in your controller all the time. It's nice to see Microsoft do something besides suck for a change.
Re: (Score:1)
Name one motion capture based game that didn't suck?
If you search Nintendo Wii bad controls on Google you'll see what I mean - and its one of the better implementations of this technology. I doubt the next Nintendo console will have such a strong emphases on this.
Every single other motion control gimmick that has come out for any console (and each was heralded as the end of standard controllers) has failed in a big way.
Re: (Score:2)
EyeToy Kinetic.
Re: (Score:3, Insightful)
I agree that motion sensing won't kill controllers; but it will become an essential feature. Would you like to go back to using your computer with no mouse?
Re: (Score:1)
The funny part about that is.... nothing to do with the control scheme.
All the systems have some crappy shovel ware on them (The Wii simply has more because of its larger user base) The PS2 had more shovel ware then the GC and Xbox the PS1 had more then the N64 and the Dreamcast and no on and so forth.
When your the largest target you get a lot of shit thrown at you :(
I really feel the Wii missed out on some great games that should have been made with its control scheme in mind, Sooooo many of the Lucas Arts
Re: (Score:2)
Not JUST the larger user base, but the massively cheaper development. Last gen, the PS2 had far more marketshare, but wasn't particularly less expensive to develop a game for than the Gamecube or XBox. This gen, the consoles in 2nd and 3rd place also cost several times as m
Re: (Score:1)
Very true points, indeed it is cheaper to make a Wii game (and probably a bit easier too since it lacks all that multi core stuff and other weirdness) So yeah its a perfect storm of crap on the wii... Thank god at least Nintendo still makes some quality games (still enjoying NSMB on it, just need a few more coins....)
Yeah Starcraft would rule :( Oh well at least I can get part 2 for the PC and while on that subject please Blizzard do not make me pay $150 for Starcraft 2 or I will pirate it and simply not pl
Re: (Score:1)
From what i have seen of Natal, it is as sloppy as the Wiimote in terms of accuracy.
I love the idea of it recognizing you and logging you in by facial features etc... (not sure that made it into the final product) but... anyways it looks really inaccurate as a controller.
OBPennyArcade (Score:5, Funny)
I think Natal is amazing
I dunno.. I'm kinda scared of this: http://www.penny-arcade.com/comic/2009/6/5/ [penny-arcade.com]
Re: (Score:1)
Yeah, especially if you consider the following quote from the summary, "and use rudimentary knowledge of body anatomy to estimate where hands or other body parts are even when they can't be seen by Natal"
It just might come true after all...
Trecares
Re: (Score:2)
I think Natal is amazing -- and you won't need to replace the batteries in your controller all the time.
I don't have to replace the batteries in my 360 controller all the time anyway, because I plug it in frequently. I plug it into a netbook that is usually on since my 360 usually isn't if I'm not using the controller. It would be awfully nice to have a charging plug on the 2AA pack though, I may have to whip something out.
I just want to try it (Score:2)
I am just so unsure about it. It has alot of potential, but it could be a huge bomb. I think Latency in commands is going to be a big issue. But maybe its not as laggy as the initial demos seem.
If I could only somehow get into E3 without being someone really important or selling my body.
Re: (Score:3, Interesting)
What I want to see is room lighting requirements. (Score:5, Interesting)
Given how every other video game system camera fails to work correctly in typical home lighting environments, this is the thing I'm most curious about.
For instance in my home, when we want to use the Eye Toy to play something like Kinetic, we have to drag out a shop-light and a couple of reflectors to stage the room and assist the contrast detection. Otherwise, the accuracy is garbage. If this thing can't work in a dark room then this also pretty much kills playing games in a room with the lights out (which is how I generally play racing games and FPS games at night).
My expectations are less than low and I'm just waiting to hear about how ever single player to use the system will need to have a Live account (with your avatar adjusted to R/L body mass).
Re: (Score:2)
Oh yes, and I RTFA about them using IR.
Re: (Score:2)
So you know it's active, built around this idea: http://en.wikipedia.org/wiki/Time-of-flight_camera [wikipedia.org]
I suspect it will be fine.
Re: (Score:1)
I kinda doubt that playing FPS games with Natal will be very efficient even if you were to do it in a well lit room. Natal is for detecting movements of your hands (and other big body parts), not your thumbs.
Re: (Score:1)
I haven't seen any hard hitting racing games or FPS for the wii, but it seems to be doing very well.
I bet this will do very well with that same group of casual gamers. Most of them probably play games in social groups with the lights on :)
Re:What I want to see is room lighting requirement (Score:2)
It uses an infrared projector for the sensor I believe. So it's creating its own light.
You're in the Movies (Score:2)
We were given "You're in the Movies" for the 360 for Xmas. It samples the background scene before calling you up as 'actors'. This gives a FAR better detection/tracking performance than Eye-Toy ever did. You no longer have to light your living room like a professional photography studio to play a simple goddam casual game.
Re: (Score:2)
It's going to be shit. Real time optical tracking is so far off. This whole natal campaign sticks of bull. Seriously, optical tracking didn't even work properly on AVATAR ! They ended up having to manually animate the faces....i>
Just so we are clear, you are comparing capturing the facial expressions of actors using optical tracking to capturing body motion and gestures using a combination of optical and distance tracking?
I think you lost a few orders of magnitude in your comparison.
Re: (Score:2)
If Avatar can't do facial non-realtime in a perfect environment with the best tech. There is no way that Natal can do body realtime with cheap tech in a million different lighting environments. Obvious to anyone with a brain and not blinded by HYPE.
An Olympic athlete can't sprint 40 mph under the best of conditions and with the best trainers. There is no way an amateur can train to run a marathon without sponsors. Obvious to anyone with a brain.
The fact that the team that produced Avatar was unable to do
Anyone Know How Accurate Natal Is? (Score:3, Interesting)
My friend was lamenting how in Rainbow Six he could hug up to a corner or object and only expose a small part of his body when he poked out to shoot. The problem being that in Modern Warfare 2, this is not the case. You have to expose your whole body. He was hoping/speculating that with Natal, a first person shooter might be able to lock into a stationary mode (much like when one is shooting through a scope) and be able to move his head, shoulders, arms, etc as he pleases to peak (and/or) shoot around a corner when he's pinned down. He wants these games to get as close to real life as possible and I told him that it's a great idea and would increase the reality of pray and spray suppressive fire tactics
Re:Anyone Know How Accurate Natal Is? (Score:4, Funny)
I don't have any concrete numbers, but I did just pick up this foam number one a few days ago...
Re:Anyone Know How Accurate Natal Is? (Score:4, Informative)
Play with the spider woman (Score:5, Funny)
The article says that 31 limbs are tracked
So even many-limbed deities like Vishnu and various spider gods can play? That's so thoughtful of Microsoft to include the divine.
Multiple people (Score:5, Funny)
Headless (Score:2)
Re: (Score:2, Informative)
Vishnu didn't actually have more than the usual complement of 4 limbs; it's just Indian depictions of their deities and gods often use more than one pair of arms to symbolise different aspects of their being.
On the subject of Natal, I'm very skeptical of anything that has Peter Molyneux involved - even before the Milo videos turned out to be fraudulent i was very suspicious.
Re: (Score:2)
Re: (Score:2)
I guess you're not one of the cool kids (Score:2)
Re: (Score:2)
Maybe he'd be happier just playing paintball and/or AirSoft?
Re: (Score:2)
No joke.
He wants war games to get as close to real life as possible without any actual work, exertion, exercise, social interaction, or dirt.
Re: (Score:3, Funny)
Fixed that for you.
He wants war games to get as close to real life as possible without any actual work, exertion, exercise, social interaction, or dirt.
Re: (Score:2)
Re: (Score:2)
If your friend wants that level of realism then he shouldn't be playing COD. I'd suggest he tries the excellent ArmA games, for all their faults they're the best infantry simulators out there.
Re: (Score:2)
Re: (Score:1)
Natal Demo (Score:5, Informative)
The ramifications of a system like this are pretty important I think. Honestly, this is what I had hoped the Wii would become... Not to start a Wii controller vs. Natal debate or anything, but I feel like games using full body motion capture would be much more interesting to me. At the end of the day, when I tried to play Wii tennis using proper motions and what not, I felt like an idiot after looking over and seeing someone else accomplish the same thing by flicking their wrist.
That said, I am interested to see how finely grained the sensitivity will be tuned to Natal recognizing people and motions. IE: if person A is playing a single player game and person B walks up and makes a hand motion, will Natal disregard that motion, or will it do whatever person B did? If so, it could seriously dent the efficacy of games being played by one person... Part of the benefit of having a physical controller is that someone else has to take it from you when they want to play, haha!
Debate? I gots yer debate! (Score:2)
Not to start a Wii controller vs. Natal debate or anything
Yeah, but in a street fight, the Wii owners at least have weapons (they don't call it a nunchuck for nothing). The Natal gang will have to rely on their fists.
Re: (Score:3, Insightful)
Re: (Score:2)
Muramasa: The Demon Blade.
That is all.
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Maybe if I get some really strong hair gell and spike my hair I'll automatically become a super saiyan! :-)
The game won't even start up until Natal recognizes spiky yellow hair.
Re: (Score:2)
Re: (Score:1)
My problem with the Wi (Score:2)
was that I felt like it encouraged "cheating"
You didn't have to do it right, you just had to what the controller thought was right.
Now waldos without physical connection would be great.
Re: (Score:2)
Re: (Score:2)
Reminds me of my overweight friend who used to play Track & Field for the NES by laying on the floor and slapping at the Powerpad [wikipedia.org].
Re: (Score:1)
Re: (Score:3, Interesting)
I'm sorry, Molyneux is the LAST person I'd trust to give a balanced review of anything.
Especially Natal. [blorge.com]
You could also see for yourself [google.com]
You know what would be cool about that demo? (Score:2)
Re: (Score:2, Insightful)
I call shenanigans on this demo. once the footage of milo actually starts, you may notice that the avatar is looking out from the screen directly towards the girl at points.
Granted, she is rather easy on the eyes, but this tends to suggest the thing has been staged. Either that or the tv used has a 3d capability more advanced than anything in existence.
Re:Natal Demo (Score:5, Informative)
Er (Score:2)
Natal and the Milo demo especially are complete BS, but that's not really the indicative bit. I mean, you have a camera that is gathering position information, even if you just assume it's capturing what's directly in front of a set that should be enough to go off of for "character on screen looks at player". Natal is giving you information about player position and distance after all, a little calibration isn't much to assume.
No, the real BS is the content. There is absolutely nothing in the demo to ind
Re:Natal Demo (Score:5, Interesting)
Tech demo? More like total fabrication. Microsoft are merely insinuating they have perfected natural language processing, natural speech synthesis, perfect motion recognition and facial recognition amongst other things.
I'm sure Milo will turn up at some point but it will be a pale imitation of this. We'll realise that you can grunt and howl at Milo and get the same reactions.
Re: (Score:1)
I agree, the demo was definitely not live, there are several inconsitencies throughout the video where the software seems to read the users mind and reacts before a proper stimulus could have been detected. It was probably captured and cut together from several test runs and later reenacted. So this clip is more wishful thinking than an actual tech demo. Which is a pity, it looks promising. Hopefully it wasn't all fake...
Re: (Score:2)
More like total fabrication.
A fake. That's false advertising, financial fraud against millions of people because people make purchasing decisions based on videos like this.
So, who's going to jail?
---
DRM is the #1 cause of software failure today.
Re: (Score:3, Insightful)
Microsoft never made any such insinuation whatsoever. I don't blame you for having this mistaken understanding of the situation as there's so much FUD going around about the Milo demo created by fanboys of other consoles but it's still completely and utterly wrong so rather silly to keep repeating.
The Milo demo does one thing and does it well, it demonstrates how Natal can be used as a CONTROLLER to get PRE-SCRIPTED RESPONSES (whilst I hate caps, these points are fundamental to what Natal is and how it can
Re: (Score:2)
All that said, I've no idea why you would think they haven't perfected motion and facial recognition for use in games, these have been solved problems for a fair while now to a level where they're useful in many games and applications and as pointed above, the same go
Re: (Score:1)
Him: "Vagina!"
His team: "Flash and clear, roger!".
Sounds pretty cool (Score:5, Interesting)
Natal sounds cool, and might just be what i'm looking for for a reason to buy a 360 to compliment my PS3. I'm still concerned about the complete lack of buttons, and am wondering if MS is going to be releasing a controller you can strap to your arm or hold with 1 hand for when you need a button to say change weapons. Also the lag issue remains to be seen. This is one possible advantage Sony's motion MIGHT have.
But in the end it's all about the games. Both techs have the potential I think to be fun, just need to see the actual implementation. And I wonder what Nintendo has in their R&D to combat this (other then that Vitality Sensor) All very interesting, although I don't see this gameplay replacing old fashioned "sitting down with a controller in your hand" gaming, just another form of entertainment.
All of this talk makes me pretty excited for what comes after this current gen...
Sony also has better accuracy (Score:4, Informative)
Sony's motion tracking technology [venturebeat.com] also reportedly has sub-millimeter accuracy, which could be better for games (along with the fact that controllers could have real buttons which would give you finer action control while moving).
I'm still dubious if either system really ends up being better for games than the Wii though. It's fun to flail around for a while, but the Wii is nice in that for really long gaming sessions you don't have to exhaust yourself with full-body motion.
Re: (Score:2)
I agree with you here. Sony's decision to include controllers that feed more location data was a great idea. The tech doesn't seem to be far along enough to do it all via optical sensors quite yet, as evidenced by the accuracy of Sony's offering compared to Natal.
Does anyone know if that 4 cm accuracy box for Natal is their TARGET, or where the tech is currently sitting?
Re: (Score:2, Interesting)
Accuracy (Score:2, Insightful)
In my mind what will affect user experience the most is its accuracy, and latency. It uses a infrared camera, so I'm wondering if anyone knows the camera's specs, especially regarding resolution and latency. Without knowing these, it is difficult to evaluate what it is good for. The article quotes a several cm. accuracy, and 10 ms recognition time, but it not clear what the sensor's limits are, and how much latency there is in the processing.
Re: (Score:2)
Eye tracking typically uses infrared light to detect the subject's gaze, and eye trackers usually run at 60 hz. Some run even faster.
Re: (Score:2)
For a pure awesome use of eye tracking see Macross Plus (or Zero maybe). Target selection just by looking at it.
What we have here... (Score:2)
...is an interesting new way to communicate. I could think of MANY applications for this tech, especially once we get out of the "early" days and into the "refined" zone. Training for surgery, advanced sports virtualization, motion-capture for the amateur crowd, eye line tracking for race car/drag car drivers (for training and teaching purposes)...hell, even the lock on your front door (gang signs incoming!)
It is, of course, possible that Natal will be a flop...but this tech could have far ranging implica
Re: (Score:2)
The system locates body parts to within a 4-centimetre cube
The brain? Meh, it might not matter so much.
Re: (Score:2)
Note in my original post that I said the tech is still in the baby steps phase. How about once the system reaches accuracy to within half a millimeter? A thousandth of a millimeter? Less?
It obviously wouldn't be the only tool a training surgeon would use, but it would be a part of their toolbox.
Re: (Score:2)
Surgeons don't usually operate with open hands (psychic "surgeons" aside).
Natal could potentially track a bright orange scalpel with sub-millimeter accuracy, much like Sony's Eyetoy/wand system works (or they could use a PS3 instead).
Re: (Score:1)
Yes, I want the surgeon trained on Natal to work on my heart valves: The system locates body parts to within a 4-centimetre cube The brain? Meh, it might not matter so much.
To be fair, the interface looks a lot easier than the "state-of-the art" in virtual surgery. Just watch this promo video and tell me whether you'd be willing to go under the knife with this interface: http://www.simendo.eu/ [simendo.eu]
How do you pull the trigger? (Score:2)
I like the idea of your on screen char mimic your own movements. But there are hundreds of interactions that involve a specific command beyond run, jump, punch, etc. I must be missing something but how are those commands execured with Natal.
Re: (Score:2)
I could be completely off base here, but... ..holding your arms/hands and standing in a way that resembles how you hold a rifle is a possibility...although that approaches the "why bother" category with immeasurable speed...
Re: (Score:2)
Voice recognition just yell out "Pew Pew Pew" and the gun shoots
You hold a regular controller (Score:2)
Key point is, Natal will not replace the current controller, not for most existing game genres like shooters. We'll still play tradititional games with traditional controllers. For those games, Natal might be used for supplementry "leaning" while holding a controller, face recognition, voice control etc, though I hope devs don't overdo this (ref: Lair).
Sony's wands are targeted very differently to Natal; they're much more like an accurate Wiimote, and are more suited to existing Wii-type games. While simila
Re: (Score:2)
I like the idea of your on screen char mimic your own movements. But there are hundreds of interactions that involve a specific command beyond run, jump, punch, etc. I must be missing something but how are those commands execured with Natal.
Ahh, interpreting the internet for the lazy. I should really get paid for this.
To be more explicit, you could use hand Mudra [wikipedia.org]s. And now, for your outstanding laziness, I give you the one-fingered mudra of contempt.
Re: (Score:2)
Honestly, I think Microsoft is releasing Natal as a proof-of-concept for the technology so that they can improve it and sell it to other sectors (low-cost motion capture, sports training enhancement, supplementary surgical training, etc.)
Microsoft Visual Simulation (Score:2)
But Microsoft is nowhere near any of those fields, are they?
They are very much in this game:
Microsoft Visual Simulation Platform Licensed by Flight1 Tech [softpedia.com]
Microsoft Simulation Platform to Be Used in Warfighter Training [softpedia.com]
Nothing but Love for the Microsoft Simulation Platform From aviation and defense industry leaders [softpedia.com]
Hands (Score:2)
Yes-- behind the back. Let's go with that.
OH NOs! (Score:1)
Everyone panic, child pr0n! Xbox works for TSA!!
Amputees (Score:2, Interesting)
What is someone was missing from an elbow down? Would the system see that as someone with it pointing directly at the device?
Re: (Score:2)
What is someone was missing from an elbow down? Would the system see that as someone with it pointing directly at the device?
No, it would correctly detect it, but would make snickering noises, and automatically name that player's avatars "Lefty", and so forth.
You heard it here first: Natal is a jerk.
Las Vegas is in Arizona? I wish! (Score:1)
Microsoft's president of entertainment and devices, announced at the CES show in Las Vegas, Arizona, this week that Natal would go on sale in November.
natal latches on quickly (Score:1)
I think I dated Natal once.
Natal too expensive? (Score:2)
"The full Natal hardware/sensor combo always looked like an expensive proposition in a market where Microsoft really needs to turn a profit," said tech website Digital Foundry. "The notion of offloading the processing to the 360 CPU in the name of lower costs and easier upgradability makes sense.”
If this is true and they start cutting corners is it even going to work as good as in the demos? Sounds like it'll be pricey so they're gonna have to get it right upon launch.
Not to be a negative Nancy (Score:1)
The next step in NUI? (Score:1)
With the desire to move us away from GUI (and CLI for
It shouldn't be too big a leap to imagine doing away with the kludgy remote control for any home/leisure digital content environment (home server, media centre, iTunes, etc)
Re: (Score:1)
Hmm, I think the intended pronunciation is "nuh TAHL", not "NAY tuhl".
The latter one had never occurred to me, though it seems appropriate as well.