Forgot your password?
typodupeerror
Hardware Hacking Input Devices Microsoft Build Games

Microsoft Research Brings Kinect-Style Depth Perception to Ordinary Cameras 31

Posted by timothy
from the how-far-away-you-are dept.
mrspoonsi (2955715) writes "Microsoft has been working on ways to make any regular 2D camera capture depth, meaning it could do some of the same things a Kinect does. As you can see in the video below the team managed to pull this off and we might see this tech all around in the near future. What's really impressive is that this works with many types of cameras. The research team used a smartphone as well as a regular webcam and both managed to achieve some impressive results, the cameras have to be slightly modified but that's only to permit more IR light to hit the sensor." The video is impressive, but note that so are several of the other projects that Microsoft has created for this year's SIGGRAPH, in particular one that makes first-person sports-cam footage more watchable.
This discussion has been archived. No new comments can be posted.

Microsoft Research Brings Kinect-Style Depth Perception to Ordinary Cameras

Comments Filter:
  • This is pretty much how the leap-motion works. Nothing really new to see here, move along.

    • by Saint Gerbil (1155665) on Tuesday August 12, 2014 @09:02AM (#47654137)

      Leap motion uses two monochromatic IR cameras and three infrared LEDs.

      This claims to use one 2d Camera.

      Apple and Pears.

      • by Tx (96709) on Tuesday August 12, 2014 @09:12AM (#47654189) Journal

        It is apples and pears on one hand, however the fact that the camera needs a modification, however small, means that you will still be buying a special bit of hardware to make your gesture control work, so in that sense it is in the same boat as the Leap. Except of course that the piece of hardware in question should be a lot cheaper, and could easily be included in laptops/tablets/monitors at minimal extra cost, if it really works that well and the idea takes off.

        • by Anonymous Coward

          As fast as the price of camera hardware has been dropping, adding a 2nd camera doesn't seem like much of a price to pay to add a dimension to the resulting product....

    • by bloodhawk (813939)

      It is completely and utterly different from how leap motion works, they are not even vaguely similar. this is about using a single 2D camera not multiple cameras and LED's

      • by daid303 (843777)

        I mean, it works in the same way, as in, it does not really work for anything but a tech demo.

  • Why isn't this a split screen of without and with?

    • by plover (150551)

      Tell you what: you build and program such a system, and see if anyone on Slashdot crucifies you for your demo's UI. Oh, what's that? You've never built anything so cool in your life? Guess that won't happen then.

      • by Thanshin (1188877)

        So you have no opinion on anything you have no personal experience in?

        So that excludes conversation about politics (unless you've managed a country), religion (unless you're a god), the opposite sex of yours, the weather (again, unless god), ...

    • Why isn't what a split screen of without and with what?

  • I thought that the kinect, while nicer than the average cheapie camera in terms of optics and sensor, also used a fairly normal camera(well, one higher resolution visual band one for image and one IR one for depth) and that the real secret sauce was the IR laser device that projected the dot pattern on the environment for the camera to pick up and interpret. Am I remembering incorrectly?
    • by axedog (991609)
      You are indeed remembering incorrectly. Kinect has a colour camera, an IR camera and an IR Emitter. http://msdn.microsoft.com/en-u... [microsoft.com]
      • by axedog (991609)
        Sorry, I read your post a tad hurriedly. Kinect has the three components that you mentioned, but that make it something other than a "fairly normal camera".
      • by qpqp (1969898)
        I believe, the question was, whether there's been an advance in the depth-sensing algorithm in the sense that you don't need a specific IR pattern (i.e. a grid) like in the Kinect anymore, but that just a couple of IR emitters are enough.
    • by plover (150551) on Tuesday August 12, 2014 @09:27AM (#47654273) Homepage Journal

      You are correct. The IR laser and IR camera are used to measure depth, while the visual light camera only picks up the image.

      The cool thing about the Kinect's IR pair is that it senses depth in the same way a pair of eyes does, in that the delta between left and right eyes provides the depth info. But instead of using two eyes, it projects a grid from the location where one eye would be, and the camera in the other location measures the deltas of "where the dot is expected - where the dot is detected". The grid is slightly randomized so that straight edges can be detected. If you've ever stared into one of those Magic Eye random dot stereogram posters, you're doing pretty much the same thing the Kinect does.

      This system is very different. The Kinect has a deep field of view, but all the demos show this working in a very short range. I haven't yet read the paper, but I'm wondering if that's the point of the IR.

      • by OzPeter (195038) on Tuesday August 12, 2014 @09:35AM (#47654319)

        This system is very different. The Kinect has a deep field of view, but all the demos show this working in a very short range. I haven't yet read the paper, but I'm wondering if that's the point of the IR.

        From watching the video my understanding is that they illuminate the subject with a fixed IR source and map the drop off of the reflected IR in 2D space and then interpret that drop off as a depth map of the object they are looking at. Which looks surpassingly accurate for the sort of use cases they demonstrate. They also point out that this technique is not a general purpose 3D system.

      • by ab8ten (551673)
        That's how the Kinect 1 works. It projects structured light and then reconstructs the world based on deviations from the expected pattern. It's built from off-the-shelf parts. The Kinect 2 measures the time it takes for an emitted laser light to be reflected back to the sensor. It's much more accurate and reliable, but requires purpose-made sensors, thus increasing the cost. Here's a good article with technical descriptions of the two methods: http://www.gamasutra.com/blogs... [gamasutra.com]
    • Re: (Score:2, Informative)

      by Anonymous Coward

      I thought that the kinect, while nicer than the average cheapie camera in terms of optics and sensor, also used a fairly normal camera(well, one higher resolution visual band one for image and one IR one for depth) and that the real secret sauce was the IR laser device that projected the dot pattern on the environment for the camera to pick up and interpret. Am I remembering incorrectly?

      Yes and no.

      It's correct for Kinect 1. It uses a "structured light" approach (developed by PrimeSense), which projects a magic pattern and has a (regular) IR cam observing the distortion in the pattern.

      Kinect 2, on the other hand, uses real Time-of-Flight (or rather, it measures the phase difference between the modulated IR signal and the reflected IR light) imaging, very similar to laser distance meters, just 2D instead of a single point. (Versus Kinect 1, it provides a better resolution and less noise.)

  • by OzPeter (195038) on Tuesday August 12, 2014 @09:19AM (#47654221)

    At the very end of the video it describes how the system is tuned to skin albedo. The only problem with this is that various races around the world have different albedos - which does have a real world effect in photography when trying to expose correctly for skin. In the video they mentioned training the system on the user, but all users shown in the video were white - so I can't say how well it would work for non-whites. But in general I am impressed with what they have done.

    Back in 2009, in Better Off Ted [wikipedia.org] episode 4 "Racial Sensitivity", they developed a security system that had issues with skin albedo and not detecting (from memory) dark skinned people - which resulted in all sorts of hijinks for the African American employees

    • Re: (Score:2, Interesting)

      by Anonymous Coward

      That itself is probably based on the fact that there are biometric systems that do have issues with certain racial types. Iris scanners are one example, as the extra melatonin means there's too little contrast to pick up enough detail to be as reliable without tuning for that type of iris. That's been known since way before 2009.

  • They should rename HyperLapse to SmoothLapse, StableLapse or CleanLapse.
    • by Anonymous Coward
      ProLapse. (with apologies to RockStar)
  • It's hard to disagree. Microsoft is power!

If a listener nods his head when you're explaining your program, wake him up.

Working...