Forgot your password?
typodupeerror
Input Devices Microsoft Games

Details On Natal's Motion Capture Technology 121

Posted by CmdrTaco
from the do-a-little-dance dept.
An anonymous reader writes "Following yesterday's announcement of a late 2010 launch date for Natal, more details are emerging on exactly how Natal works. Alex Kipman, the project's lead developer, explains that Natal uses only 10-15% of the Xbox's resources to calibrate to a new player inside 160 milliseconds, track one or two players simultaneously, and use rudimentary knowledge of body anatomy to estimate where hands or other body parts are even when they can't be seen by Natal — for instance when they are held behind the back."
This discussion has been archived. No new comments can be posted.

Details On Natal's Motion Capture Technology

Comments Filter:
  • by sznupi (719324) on Thursday January 07, 2010 @02:04PM (#30685202) Homepage

    The aspect of how Natal communicates with X360 is most interesting to me, and surely many others.

    It seems like it could be mighty fun peripheral for robotics projects... (and who knows, perhaps MS could even promote it, seeing that they have their robotics suite? Certainly small number of Natals not used for gaming wouldn't harm MS financially in any significant way; but they might like good publicity)

    • by AndrewNeo (979708) on Thursday January 07, 2010 @02:29PM (#30685554) Homepage

      I believe there was talk of the hardware working on the PC, too, though the hardware is pretty much just a few cameras and other sensors, leaving it up to the libraries on the Xbox or PC to do the processing. Now, if Microsoft releases an SDK (which is in their best interest, if they want PC integration) then it shouldn't be an issue.

      • by sznupi (719324)

        It's not exactly "pretty much just a few cameras and other sensors". Yes, there is RGB cam and microphone array, which will be often useful in their own way.

        But most importantly, there's this:
        http://en.wikipedia.org/wiki/Time-of-flight_camera [wikipedia.org]
        And cheaply. Which might be huge for amateur robotics projects.

        SDK for Windows might be the best we can hope for indeed, if wanting something usable. Not exactly optimal, you're basically forced to use some Windows netbook (instead of some ARM board) if your creation is

        • You could still use an ARM on the machine itself, I think the first reply covers that nicely even though they are referring to a different architecture.

          Link: [microsoft.com]

          "What you have to do is write a program that runs on the AVR (using for instance WinAVR) and this program talks to Robotics Studio running on a PC."

          • by sznupi (719324)

            That's not really applicable to Project Natal usage, don't you think? Using MS Robotics Studio to control something is a different thing than using fabulous (for the price) sensors of Natal by putting it on a robot of some sort. And wondering now how easy/hard it will be to access those sensors, what will be required.

  • awesome (Score:2, Flamebait)

    by sneakyimp (1161443)

    I think Natal is amazing -- and you won't need to replace the batteries in your controller all the time. It's nice to see Microsoft do something besides suck for a change.

    • Name one motion capture based game that didn't suck?

      If you search Nintendo Wii bad controls on Google you'll see what I mean - and its one of the better implementations of this technology. I doubt the next Nintendo console will have such a strong emphases on this.

      Every single other motion control gimmick that has come out for any console (and each was heralded as the end of standard controllers) has failed in a big way.

      • EyeToy Kinetic.

      • Re: (Score:3, Insightful)

        by Toonol (1057698)
        There are a lot of Wii motion-based games that work well. The (numerous) ones that DO suck, would probably have been just AS terrible with a controller, because they were cheap cash-in titles.

        I agree that motion sensing won't kill controllers; but it will become an essential feature. Would you like to go back to using your computer with no mouse?
        • by JDeane (1402533)

          The funny part about that is.... nothing to do with the control scheme.

          All the systems have some crappy shovel ware on them (The Wii simply has more because of its larger user base) The PS2 had more shovel ware then the GC and Xbox the PS1 had more then the N64 and the Dreamcast and no on and so forth.

          When your the largest target you get a lot of shit thrown at you :(

          I really feel the Wii missed out on some great games that should have been made with its control scheme in mind, Sooooo many of the Lucas Arts

          • by Toonol (1057698)
            All the systems have some crappy shovel ware on them (The Wii simply has more because of its larger user base) The PS2 had more shovel ware then the GC and Xbox the PS1 had more then the N64 and the Dreamcast and no on and so forth.

            Not JUST the larger user base, but the massively cheaper development. Last gen, the PS2 had far more marketshare, but wasn't particularly less expensive to develop a game for than the Gamecube or XBox. This gen, the consoles in 2nd and 3rd place also cost several times as m
            • by JDeane (1402533)

              Very true points, indeed it is cheaper to make a Wii game (and probably a bit easier too since it lacks all that multi core stuff and other weirdness) So yeah its a perfect storm of crap on the wii... Thank god at least Nintendo still makes some quality games (still enjoying NSMB on it, just need a few more coins....)

              Yeah Starcraft would rule :( Oh well at least I can get part 2 for the PC and while on that subject please Blizzard do not make me pay $150 for Starcraft 2 or I will pirate it and simply not pl

    • From what i have seen of Natal, it is as sloppy as the Wiimote in terms of accuracy.

      I love the idea of it recognizing you and logging you in by facial features etc... (not sure that made it into the final product) but... anyways it looks really inaccurate as a controller.

    • by schon (31600) on Thursday January 07, 2010 @02:47PM (#30685798)

      I think Natal is amazing

      I dunno.. I'm kinda scared of this: http://www.penny-arcade.com/comic/2009/6/5/ [penny-arcade.com]

      • by Trecares (416205)

        Yeah, especially if you consider the following quote from the summary, "and use rudimentary knowledge of body anatomy to estimate where hands or other body parts are even when they can't be seen by Natal"

        It just might come true after all...

        Trecares

    • by drinkypoo (153816)

      I think Natal is amazing -- and you won't need to replace the batteries in your controller all the time.

      I don't have to replace the batteries in my 360 controller all the time anyway, because I plug it in frequently. I plug it into a netbook that is usually on since my 360 usually isn't if I'm not using the controller. It would be awfully nice to have a charging plug on the 2AA pack though, I may have to whip something out.

  • I am just so unsure about it. It has alot of potential, but it could be a huge bomb. I think Latency in commands is going to be a big issue. But maybe its not as laggy as the initial demos seem.

    If I could only somehow get into E3 without being someone really important or selling my body.

    • Re: (Score:3, Interesting)

      by flitty (981864)
      The article states the latency for the camera to recognize a new position is 10 miliseconds, which IIRC is close to what wireless controller lag is. The better complaint is that it is only accurate to about 1.5", which is fine for large gesture based gaming, but minor refined movements won't be picked up.
      • by gmezero (4448) on Thursday January 07, 2010 @02:37PM (#30685642) Homepage

        Given how every other video game system camera fails to work correctly in typical home lighting environments, this is the thing I'm most curious about.

        For instance in my home, when we want to use the Eye Toy to play something like Kinetic, we have to drag out a shop-light and a couple of reflectors to stage the room and assist the contrast detection. Otherwise, the accuracy is garbage. If this thing can't work in a dark room then this also pretty much kills playing games in a room with the lights out (which is how I generally play racing games and FPS games at night).

        My expectations are less than low and I'm just waiting to hear about how ever single player to use the system will need to have a Live account (with your avatar adjusted to R/L body mass).

  • by eldavojohn (898314) * <eldavojohn.gmail@com> on Thursday January 07, 2010 @02:08PM (#30685260) Journal
    Accuracy aberrations plague a lot the new free motion input devices. Does anyone know to what degree of accuracy this works? Down to the inch? Centimeter?

    My friend was lamenting how in Rainbow Six he could hug up to a corner or object and only expose a small part of his body when he poked out to shoot. The problem being that in Modern Warfare 2, this is not the case. You have to expose your whole body. He was hoping/speculating that with Natal, a first person shooter might be able to lock into a stationary mode (much like when one is shooting through a scope) and be able to move his head, shoulders, arms, etc as he pleases to peak (and/or) shoot around a corner when he's pinned down. He wants these games to get as close to real life as possible and I told him that it's a great idea and would increase the reality of pray and spray suppressive fire tactics ... but I was skeptical of the devices ability for this accuracy. So, anyone know any concrete numbers yet?
    • by TheRealMindChild (743925) on Thursday January 07, 2010 @02:10PM (#30685292) Homepage Journal
      So, anyone know any concrete numbers yet?

      I don't have any concrete numbers, but I did just pick up this foam number one a few days ago...
    • by flitty (981864) on Thursday January 07, 2010 @02:18PM (#30685408)
      The article says that 31 limbs are tracked to within a 4 cm box. So move your hand 1.5". It's that accurate (which isn't great).
    • Gee, I thought everyone here knew the accuracy specs of the unfinished and unreleased Natal. You mean *you* don't? [Nelson voice] HA HA!
    • He wants these games to get as close to real life as possible

      Maybe he'd be happier just playing paintball and/or AirSoft?

      • by gandhi_2 (1108023)

        No joke.

        He wants war games to get as close to real life as possible without any actual work, exertion, exercise, social interaction, or dirt.

        • Re: (Score:3, Funny)

          by MagicM (85041)

          Fixed that for you.

          He wants war games to get as close to real life as possible without any actual work, exertion, exercise, social interaction, or dirt.

    • Maybe you should not expose these small parts of your body if you are concerned that they are less than a centimeter.
    • by Nick Ives (317)

      If your friend wants that level of realism then he shouldn't be playing COD. I'd suggest he tries the excellent ArmA games, for all their faults they're the best infantry simulators out there.

    • by timeOday (582209)
      For games that require moving around the map like that, the bigger question is how you will walk around at all (much less positioning yourself precisely to take cover). You can't actually move away from the camera, nor can you physically interact with the 3d terrain. Researchers for training simulations have been working on it for years, and come up with many contraptions from treadmills to giant hampster balls, none of which are ideal, much less feasible in a livingroom. Who wants to place bets on wha
      • It might be an awkward combo, but one option would be to use standard gamepad controls, and then read relative shoulder/torso movement once you're in a position to lean.
  • Natal Demo (Score:5, Informative)

    by rehtonAesoohC (954490) on Thursday January 07, 2010 @02:12PM (#30685322) Journal
    Here [youtube.com] is a tech demo showing a Peter Molyneux demonstration of a game using Natal.

    The ramifications of a system like this are pretty important I think. Honestly, this is what I had hoped the Wii would become... Not to start a Wii controller vs. Natal debate or anything, but I feel like games using full body motion capture would be much more interesting to me. At the end of the day, when I tried to play Wii tennis using proper motions and what not, I felt like an idiot after looking over and seeing someone else accomplish the same thing by flicking their wrist.

    That said, I am interested to see how finely grained the sensitivity will be tuned to Natal recognizing people and motions. IE: if person A is playing a single player game and person B walks up and makes a hand motion, will Natal disregard that motion, or will it do whatever person B did? If so, it could seriously dent the efficacy of games being played by one person... Part of the benefit of having a physical controller is that someone else has to take it from you when they want to play, haha!
    • Not to start a Wii controller vs. Natal debate or anything

      Yeah, but in a street fight, the Wii owners at least have weapons (they don't call it a nunchuck for nothing). The Natal gang will have to rely on their fists.

      • Re: (Score:3, Insightful)

        by phigmeta (1714352)
        Considering the Wii gang is bunch of 7 year olds, girls who don't want to break their nails, and some gay guys who just finished a sweaty game of wii cheer 2..... I think we will be fine.
      • Maybe, but imagine how cool it will be playing a DBZ based game and doing the Kamehame wave attack! Maybe if I get some really strong hair gell and spike my hair I'll automatically become a super saiyan! :-)
        • Maybe if I get some really strong hair gell and spike my hair I'll automatically become a super saiyan! :-)

          The game won't even start up until Natal recognizes spiky yellow hair.

        • by s73v3r (963317)
          Yeah, but then you have to go through Fusion.
      • by d0rp (888607)
        Yes, but fists of fury!
    • was that I felt like it encouraged "cheating"

      You didn't have to do it right, you just had to what the controller thought was right.

      Now waldos without physical connection would be great.

      • See, that's exactly what bothered me about the Wii. Everyone was so excited about a game controller where you get to mimic real movement... that is until everyone figured out how you could really play... [penny-arcade.com]
        • Reminds me of my overweight friend who used to play Track & Field for the NES by laying on the floor and slapping at the Powerpad [wikipedia.org].

          • My uncle did the same exact thing. and he's rather thin as a rail. We all considered this cheating, he didnt think anything of it.
    • Re: (Score:3, Interesting)

      by Anonymous Coward

      I'm sorry, Molyneux is the LAST person I'd trust to give a balanced review of anything.

      Especially Natal. [blorge.com]

      You could also see for yourself [google.com]

    • If it would let you shove Milo off the bridge. (Guess that's a feature in the sequel to Bully.)
    • Re: (Score:2, Insightful)

      by shar303 (944843)

      I call shenanigans on this demo. once the footage of milo actually starts, you may notice that the avatar is looking out from the screen directly towards the girl at points.

      Granted, she is rather easy on the eyes, but this tends to suggest the thing has been staged. Either that or the tv used has a 3d capability more advanced than anything in existence.

      • Re:Natal Demo (Score:5, Informative)

        by sammyF70 (1154563) on Thursday January 07, 2010 @04:24PM (#30686988) Homepage Journal
        yeah ... the milo demo is a semi-fake. there was apparently someone behind a curtain controlling the character see : http://www.gametrailers.com/video/episode-60-invisible-walls/51228?type=wmv [gametrailers.com] (somewhere in the third quarter they talk about it)
      • by oGMo (379)

        Natal and the Milo demo especially are complete BS, but that's not really the indicative bit. I mean, you have a camera that is gathering position information, even if you just assume it's capturing what's directly in front of a set that should be enough to go off of for "character on screen looks at player". Natal is giving you information about player position and distance after all, a little calibration isn't much to assume.

        No, the real BS is the content. There is absolutely nothing in the demo to ind

    • Re:Natal Demo (Score:5, Interesting)

      by DrXym (126579) on Thursday January 07, 2010 @04:27PM (#30687030)
      Here is a tech demo showing a Peter Molyneux demonstration of a game using Natal.

      Tech demo? More like total fabrication. Microsoft are merely insinuating they have perfected natural language processing, natural speech synthesis, perfect motion recognition and facial recognition amongst other things.

      I'm sure Milo will turn up at some point but it will be a pale imitation of this. We'll realise that you can grunt and howl at Milo and get the same reactions.

      • by elFisico (877213)

        I agree, the demo was definitely not live, there are several inconsitencies throughout the video where the software seems to read the users mind and reacts before a proper stimulus could have been detected. It was probably captured and cut together from several test runs and later reenacted. So this clip is more wishful thinking than an actual tech demo. Which is a pity, it looks promising. Hopefully it wasn't all fake...

      • by bit01 (644603)

        More like total fabrication.

        A fake. That's false advertising, financial fraud against millions of people because people make purchasing decisions based on videos like this.

        So, who's going to jail?

        ---

        DRM is the #1 cause of software failure today.

      • Re: (Score:3, Insightful)

        by Xest (935314)

        Microsoft never made any such insinuation whatsoever. I don't blame you for having this mistaken understanding of the situation as there's so much FUD going around about the Milo demo created by fanboys of other consoles but it's still completely and utterly wrong so rather silly to keep repeating.

        The Milo demo does one thing and does it well, it demonstrates how Natal can be used as a CONTROLLER to get PRE-SCRIPTED RESPONSES (whilst I hate caps, these points are fundamental to what Natal is and how it can

        • by DrXym (126579)
          It was an obvious fake. I really don't see how you can possibly think otherwise from the footage or reactions of people who saw it behind closed doors. For example Shane Satterfield of Spike TV said it was "not legit" and "smoke and mirrors".

          All that said, I've no idea why you would think they haven't perfected motion and facial recognition for use in games, these have been solved problems for a fair while now to a level where they're useful in many games and applications and as pointed above, the same go

      • Reminds me of the first time me and my friend played Rainbow Six on the Xbox 1. I told him, "hey try out the voice commands!"

        Him: "Vagina!"
        His team: "Flash and clear, roger!".
  • Sounds pretty cool (Score:5, Interesting)

    by Reapman (740286) on Thursday January 07, 2010 @02:14PM (#30685344)

    Natal sounds cool, and might just be what i'm looking for for a reason to buy a 360 to compliment my PS3. I'm still concerned about the complete lack of buttons, and am wondering if MS is going to be releasing a controller you can strap to your arm or hold with 1 hand for when you need a button to say change weapons. Also the lag issue remains to be seen. This is one possible advantage Sony's motion MIGHT have.

    But in the end it's all about the games. Both techs have the potential I think to be fun, just need to see the actual implementation. And I wonder what Nintendo has in their R&D to combat this (other then that Vitality Sensor) All very interesting, although I don't see this gameplay replacing old fashioned "sitting down with a controller in your hand" gaming, just another form of entertainment.

    All of this talk makes me pretty excited for what comes after this current gen...

    • by SuperKendall (25149) on Thursday January 07, 2010 @02:39PM (#30685674)

      Sony's motion tracking technology [venturebeat.com] also reportedly has sub-millimeter accuracy, which could be better for games (along with the fact that controllers could have real buttons which would give you finer action control while moving).

      I'm still dubious if either system really ends up being better for games than the Wii though. It's fun to flail around for a while, but the Wii is nice in that for really long gaming sessions you don't have to exhaust yourself with full-body motion.

      • by Pojut (1027544)

        I agree with you here. Sony's decision to include controllers that feed more location data was a great idea. The tech doesn't seem to be far along enough to do it all via optical sensors quite yet, as evidenced by the accuracy of Sony's offering compared to Natal.

        Does anyone know if that 4 cm accuracy box for Natal is their TARGET, or where the tech is currently sitting?

    • Re: (Score:2, Interesting)

      by Dalambertian (963810)
      Despite what they keep showing in the press releases, I don't think the gesture control is what the big titles will be using it for. Remember Johnny Lee's wiimote hack at TED? http://blog.ted.com/2008/04/wii_remote_hack.php [ted.com] According to his blog, he's been working on Natal and seems very impressed with the specs. http://procrastineering.blogspot.com/2009/06/project-natal.html [blogspot.com] Anyway, I'm guessing it's new camera controls like his that would really sell this thing for the "serious gamers" and not necessarily
  • Accuracy (Score:2, Insightful)

    by chelberg (1712998)

    In my mind what will affect user experience the most is its accuracy, and latency. It uses a infrared camera, so I'm wondering if anyone knows the camera's specs, especially regarding resolution and latency. Without knowing these, it is difficult to evaluate what it is good for. The article quotes a several cm. accuracy, and 10 ms recognition time, but it not clear what the sensor's limits are, and how much latency there is in the processing.

    • by DeadCatX2 (950953)

      Eye tracking typically uses infrared light to detect the subject's gaze, and eye trackers usually run at 60 hz. Some run even faster.

      • by jgtg32a (1173373)

        For a pure awesome use of eye tracking see Macross Plus (or Zero maybe). Target selection just by looking at it.

  • ...is an interesting new way to communicate. I could think of MANY applications for this tech, especially once we get out of the "early" days and into the "refined" zone. Training for surgery, advanced sports virtualization, motion-capture for the amateur crowd, eye line tracking for race car/drag car drivers (for training and teaching purposes)...hell, even the lock on your front door (gang signs incoming!)

    It is, of course, possible that Natal will be a flop...but this tech could have far ranging implica

    • by icepick72 (834363)
      Yes, I want the surgeon trained on Natal to work on my heart valves:
      The system locates body parts to within a 4-centimetre cube
      The brain? Meh, it might not matter so much.
      • by Pojut (1027544)

        Note in my original post that I said the tech is still in the baby steps phase. How about once the system reaches accuracy to within half a millimeter? A thousandth of a millimeter? Less?

        It obviously wouldn't be the only tool a training surgeon would use, but it would be a part of their toolbox.

      • by Namarrgon (105036)

        Surgeons don't usually operate with open hands (psychic "surgeons" aside).

        Natal could potentially track a bright orange scalpel with sub-millimeter accuracy, much like Sony's Eyetoy/wand system works (or they could use a PS3 instead).

      • Yes, I want the surgeon trained on Natal to work on my heart valves: The system locates body parts to within a 4-centimetre cube The brain? Meh, it might not matter so much.

        To be fair, the interface looks a lot easier than the "state-of-the art" in virtual surgery. Just watch this promo video and tell me whether you'd be willing to go under the knife with this interface: http://www.simendo.eu/ [simendo.eu]

  • I like the idea of your on screen char mimic your own movements. But there are hundreds of interactions that involve a specific command beyond run, jump, punch, etc. I must be missing something but how are those commands execured with Natal.

    • by Pojut (1027544)

      I could be completely off base here, but... ..holding your arms/hands and standing in a way that resembles how you hold a rifle is a possibility...although that approaches the "why bother" category with immeasurable speed...

    • by jgtg32a (1173373)

      Voice recognition just yell out "Pew Pew Pew" and the gun shoots

    • Key point is, Natal will not replace the current controller, not for most existing game genres like shooters. We'll still play tradititional games with traditional controllers. For those games, Natal might be used for supplementry "leaning" while holding a controller, face recognition, voice control etc, though I hope devs don't overdo this (ref: Lair).

      Sony's wands are targeted very differently to Natal; they're much more like an accurate Wiimote, and are more suited to existing Wii-type games. While simila

    • by drinkypoo (153816)

      I like the idea of your on screen char mimic your own movements. But there are hundreds of interactions that involve a specific command beyond run, jump, punch, etc. I must be missing something but how are those commands execured with Natal.

      Ahh, interpreting the internet for the lazy. I should really get paid for this.

      Depending on the person's distance from the sensor, Project Natal [wikipedia.org] is capable of tracking models that can identify individual fingers.

      To be more explicit, you could use hand Mudra [wikipedia.org]s. And now, for your outstanding laziness, I give you the one-fingered mudra of contempt.

  • estimate where hands or other body parts are even when they can't be seen by Natal -- for instance when they are held behind the back

    Yes-- behind the back. Let's go with that.

  • So now my xbox is taking pictures of kids!

    Everyone panic, child pr0n! Xbox works for TSA!!
  • Amputees (Score:2, Interesting)

    by Anonymous Coward

    What is someone was missing from an elbow down? Would the system see that as someone with it pointing directly at the device?

    • by Chris Burke (6130)

      What is someone was missing from an elbow down? Would the system see that as someone with it pointing directly at the device?

      No, it would correctly detect it, but would make snickering noises, and automatically name that player's avatars "Lefty", and so forth.

      You heard it here first: Natal is a jerk.

  • Microsoft's president of entertainment and devices, announced at the CES show in Las Vegas, Arizona, this week that Natal would go on sale in November.

  • It needs only 160 milliseconds to latch on to the body shape of a new user stepping in front of it.

    I think I dated Natal once.
  • http://www.edge-online.com/news/rumour-microsoft-drops-internal-natal-chip [edge-online.com]

    "The full Natal hardware/sensor combo always looked like an expensive proposition in a market where Microsoft really needs to turn a profit," said tech website Digital Foundry. "The notion of offloading the processing to the 360 CPU in the name of lower costs and easier upgradability makes sense.”

    If this is true and they start cutting corners is it even going to work as good as in the demos? Sounds like it'll be pricey so they're gonna have to get it right upon launch.

  • But I've only seen Natal as a fancy webcam that's gotten too much press. I expect the games that will be made for it will be marginally more advanced than what you can get with the PS2 EyeToy. It can be a success, but I think we should temper our expectations of it.
  • Why restrict to gaming?
    With the desire to move us away from GUI (and CLI for /.ers) and towards "natural user interface" using gestures, it would seem like the next obvious step from the need - still - to interact directly with a surface (whether The Surface, or other multi-touch screen interface).
    It shouldn't be too big a leap to imagine doing away with the kludgy remote control for any home/leisure digital content environment (home server, media centre, iTunes, etc)

Swap read error. You lose your mind.

Working...