Microsoft Tech Can Deblur Images Automatically

An anonymous reader writes "At the annual SIGGRAPH show, Microsoft Research showed new technology that can remove the blur from images on your camera or phone using on-board sensors — the same sensors currently added to the iPhone 4. No more blurry low light photos!"

Re:lol yea sure

[Helios1182]

Microsoft Research puts out a lot of really interesting and successful research. They aren't the people programming the OS or office applications.

"No more blurry low light photos!"

[Average_Joe_Sixpack]

Social networking sites are about to get a whole lot more ugly

Re:lol yea sure

[bill_mcgonigle]

Single-window mode hasn't been released yet, but it's coming. This will make it usable for folks who aren't using fvwm with focus-follows-cursor.
 

Re:Interestingly simple concept

[Threni]

This isn't IS like Canon (for example) has on its lenses. This is making a note of the movement and removing it later (where later could mean just after the pic is taken) rather than using gyros or whatever to prevent the shaking from affecting the picture in the first place. Perhaps both systems could be used, but I'm not sure, given that I'm not sure if it makes sense to use a note of how a camera was moved when the picture was taken at the same time that some of the movement has been compensated for - you might end up adding it back in again.

Useful, but limited

[AliasMarlowe]

It won't help at all if the object is moving. In fact, this feature should be switched off if you're trying to photograph a moving object with the camera (common enough, and not just in sports). It would not be able to compensate for a mismatch between the object speed and your tracking movement, and would do entirely the wrong thing even if you tracked the moving object perfectly for the shot. In this case, there is no substitute for adequate light and/or a fast lens and/or a smooth accurate tracking movement.

As another comment, deconvolution requires a very accurate approximation of the true convolution kernel, which may be provided by the motion sensors. However, to reconstruct the image without artifacts, the true kernel must not approach zero in the Fourier domain below the Nyquist frequency of the intended reconstruction (which is limited by the antialias filter in front of the Bayer mask). In fact, if the kernel's Fourier transform has too small a magnitude at some frequency, the reconstruction at that frequency will be essentially noise, or will be zero if adequate regularization is used. If the motion blur is more than a few pixels, this will generally mean that the reconstructed image will have an abridged spectrum in the direction of blur, compared to directions in which no blur occurred. Of course, if your hand is so shaky and the exposure so long that blur occurs in all directions, then the spectrum of the reconstructed image will be more uniform. It is likely to be truncated compared to the spectrum of an image taken without motion blur.

The quality of the reconstructed image would also be limited by the effects of other convolutions in the optical pathway. For instance, if you're using a cheap superzoom lens, don't expect to get anywhere near the antialias filter's Nyquist frequency in the final image, as the lens will have buggered up the details nonlinearly across the image even before the motion blur is added. If you're using nice lenses (Canon "L" series or Pentax "*" series and suchlike), then this will not be an issue.

The method would seem to be useful in low-ish light photography of stationary objects. A sober photographer would beat a drunk photographer at this, but the technique would help both to some extent. A photographer using a tripod would do best, of course.

Re:Frankencamera.

[slashqwerty]

It's worth noting that page nine of the Frankencamera team's paper [stanford.edu] mentions the work of Joshi et al when it discusses deblurring pictures. Neel Joshi [microsoft.com] was the lead researcher from the article we are discussing.

Re:Here's a dumb question...

[ceoyoyo]

You've described blind deconvolution. It does work, but guided deconvolution, a version of which they're doing here, usually works better because you're providing more information. The search space is very large and you have to make assumptions anyway (just how does the computer assess the "sharpness" of an image?) so anything you can do to narrow it down usually improves your results.

Re:Even so...

[fyngyrz]

I suspect that if it's possible to get very many images of the subject then you can gather enough data to rebuild what would be a more accurate image of the subject.

Yes. We call that technique "stacking." And it can result in profound improvements. Here is [flickr.com] a before and after of stacking; at left, one normal shot from the camera at pushed ISO 12800 (ISO 3200 with an additional 2-stop digital push, in-camera), at right, the result of combining 36 of those shots and recovering the data through the noise. Kind of amazing, isn't it? That was done with nothing but a Canon EOS 50D camera and Canon's 200mm f/2.8 lens, no telescope, no tracking rig of any kind. I wrote software to rotate and translate each frame to get them to overlay despite the motion between frames, and then take a 48-bit accurate average of the resulting stacks of pixels; results as you see.

The Ring Nebula is magnitude 9, which is no minor feat to resolve with a middle of the line DSLR... but stacking is the big hammer when it comes to this kind of work. You want to talk low light... magnitude 9 is low, all right. And if that's not enough to impress you, there's a magnitude 15(!!!) object resolved in that same picture -- see the notes. Doesn't get much more "low light" than that.