Hot Comments
- Re:But what created the law of gravity? (5 points, Insightful) by neiltrodden on Thursday September 02, @08:58AM attached to Hawking Picks Physics Over God For Big Bang
- Is India trying to *stab* its economy? (5 points, Insightful) by FooAtWFU on Wednesday September 01, @10:35AM attached to India Now Wants Access To Google and Skype
- Re:Pfah. (5 points, Insightful) by mini me on Wednesday September 01, @01:06PM attached to Yale Researchers Prove That ACID Is Scalable
- The Principles of the Internet according to AT& (5 points, Insightful) by brennanw on Wednesday September 01, @09:57AM attached to AT&T Says Net Rules Must Allow 'Paid Prioritization'
- Re:Goodbye India (5 points, Funny) by Predius on Wednesday September 01, @10:52AM attached to India Now Wants Access To Google and Skype
14721952
story
Comments: 204 + - Hardware: Microsoft Tech Can Deblur Images Automatically on Saturday July 31, @05:50PM
background: url(//a.fsdn.com/sd/topics/topicinputdevices.gif); width:70px; height:68px;
inputdev
background: url(//a.fsdn.com/sd/topics/topiciphone.png); width:51px; height:100px;
iphone
background: url(//a.fsdn.com/sd/topics/topiccellphone.gif); width:75px; height:74px;
cellphones
background: url(//a.fsdn.com/sd/topics/topicgraphics3.gif); width:69px; height:62px;
graphics
background: url(//a.fsdn.com/sd/topics/topicms.gif); width:75px; height:55px;
microsoft
An anonymous reader writes "At the annual SIGGRAPH show, Microsoft Research showed new technology that can remove the blur from images on your camera or phone using on-board sensors — the same sensors currently added to the iPhone 4. No more blurry low light photos!"
Related Stories
Submission: Microsoft Can Deblur Images Automatically by Anonymous Coward
I like work; it fascinates me; I can sit and look at it for hours.
All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2010 Geeknet, Inc.
Enhance (Score:5, Funny)
Re:Enhance (Score:5, Funny)
Parent
Re:Enhance (Score:5, Interesting)
Parent
Re:Enhance (Score:5, Funny)
Actually, that's a Alpha version. The production version is demonstrated here:
http://www.youtube.com/watch?v=KUFkb0d1kbU [youtube.com]
Parent
Re: (Score:2, Funny)
Ugh (Score:2)
That's the first thing that came to mind. Luckily, it's a hardware attachment so we can still tell people to fuck off when they come to us with blurry photos.
Unless they have the attachment.
Windows 7 (Score:3, Funny)
I bet it can remove the blur from the titlebar for screenshots of a Windows 7 app. Now we can all see what those developers are viewing behind that window!
Useful, but limited (Score:5, Informative)
As another comment, deconvolution requires a very accurate approximation of the true convolution kernel, which may be provided by the motion sensors. However, to reconstruct the image without artifacts, the true kernel must not approach zero in the Fourier domain below the Nyquist frequency of the intended reconstruction (which is limited by the antialias filter in front of the Bayer mask). In fact, if the kernel's Fourier transform has too small a magnitude at some frequency, the reconstruction at that frequency will be essentially noise, or will be zero if adequate regularization is used. If the motion blur is more than a few pixels, this will generally mean that the reconstructed image will have an abridged spectrum in the direction of blur, compared to directions in which no blur occurred. Of course, if your hand is so shaky and the exposure so long that blur occurs in all directions, then the spectrum of the reconstructed image will be more uniform. It is likely to be truncated compared to the spectrum of an image taken without motion blur.
The quality of the reconstructed image would also be limited by the effects of other convolutions in the optical pathway. For instance, if you're using a cheap superzoom lens, don't expect to get anywhere near the antialias filter's Nyquist frequency in the final image, as the lens will have buggered up the details nonlinearly across the image even before the motion blur is added. If you're using nice lenses (Canon "L" series or Pentax "*" series and suchlike), then this will not be an issue.
The method would seem to be useful in low-ish light photography of stationary objects. A sober photographer would beat a drunk photographer at this, but the technique would help both to some extent. A photographer using a tripod would do best, of course.
Parent
I think you oughta look at the examples. (Score:3, Insightful)
There are some full-size samples of the results of the technique [microsoft.com], where you can compare the original image with the result of their technique, and the results of two older techniques. Their technique show some very obvious problems:
Re:Useful, but limited (Score:4, Interesting)
Do you mention FTs just for reference, or are you implying that they are typically used in deconvolutions? In my experience, signals with any amount of noise are much better handled with iterative algorithms.
Yes. Fourier methods are unlikely to be used in practice on images. However, it's instructive to look at the process in a transform space to understand the extent to which information is irrecoverably lost in the optical path. The consequences can be explained using any suitable integral transform, but engineers are most familiar with Fourier and wavelet methods.
Suppose the Fourier transform of the "perfect" image is J, and the Fourier transform of the exact blurring kernel is K, then the transform of the blurred image is L=JK. If the kernel is known exactly, and has adequate magnitude through the frequency range of interest, the image can be recovered simply by using an inverse of the kernel J=L/K. Due to the physics of photon detection, the measured image will also contain some amount of noise N, so even in this ideal case, the recovered image will be corrupted by an amount of noise J'=(L+N)/K. Realistically, the kernel is not known exactly, and may have small magnitude at some frequencies so that its inverse is unreliable. A related consequence is that the blurred image at those bands will be primarily noise. Use of pseudo-inverses is also unreliable, since they are discontinuous near a spectral zero with very high sensitivity to perturbation near that zero. So-called Wiener deconvolution attempts to circumvent this by diagonally biasing the transform of the estimated kernel before inverting, but with generally unsatisfactory results.
Information which has been destroyed cannot be recovered, by any method. Any attempt to do so would merely amplify the noise in the measured blurry image at those bands. Iterative methods (typically a variant of Richardon-Lucy for images) try to minimize the amplification of noise in various ways, all imperfect but preferable to a direct Fourier method. Most iterative methods will, left to themselves, converge on the same asymptote as the Fourier method. However, iterative deconvolution methods always employ a regularization step in each iteration, whose primary purpose is to attenuate adjustments in bands where the kernel is uncertain. They also generally use a small number of iterations, since the first iterations are less affected by noise than later iterations. The end result is that information which was destroyed is not spuriously recreated from noise (in principle at least).
If you're interested, I recommend: P. Jansson (ed.), Deconvolution of Images and Spectra, 2nd ed., Academic Press, 1996. Alas, it appears to be out of print (and my copy is not for sale).
Parent
Interestingly simple concept (Score:4, Insightful)
Re: (Score:2)
This method (like all motion compensating algorithms) can correct for some motion blur but it will add other defects to the image. So while it might "save" a picture already captured it's better to take a new photo.
Re: (Score:2)
The concept is in fact so simple it has already been done. This is probably just a new enhanced(!) algorithm. I have a several years old digital camera which has the ability to compensate for moving or shaking the camera, this basic feature is just off by default on my camera, though some more idiot-proff cameras have it on by default.
Re: (Score:2)
I imagine that if this tech does make it into higher-end cameras (namely SLRs), the accelerometer data will in fact be saved as extra data in the RAW file. In fact due to the nature of RAW files, I think it would have to be done that way. Naturally if you're shooting jpegs (phones, P&S, foolish SLR users), then you just take what you get and that's it. It will probably just become another part of RAW "development" for higher-end shooters.
Ultimately, the concept isn't very different than the image stabil
non-inertial frame (Score:3, Insightful)
My only tiny little concern is that you might introduce artifacts into your photos - which makes me wonder if it wouldn't be better to store a raw image and the data from these sensors independently? I wonder if there is a scenario where you might be moving but the object you're taking a picture of is stationary relative to your movement.
I suspect in the majority of cases, this would improve photos. As to your query, my first thought of a problematic environment would be trying to take a photo of a friend sitting next to you--in a moving roller coaster as it hurls around a bend. You and your friend are [mostly] stationary WRT each other, but you (and the camera) are all undergoing acceleration, which the camera dutifully attepts to remove from the photo. Certainly a comparatively rare event compared to the majority of photo-ops.
Put it to good use (Score:2)
Frankencamera. (Score:5, Interesting)
Step back! This is a job for Frankencamera [stanford.edu]. Run it on your Nokia N900 [nokia.com] today.
OTOH having that Arduino board and a mess of wires attached to your camera does score you a lot more geek cred than photographing using an plain old mobile phone.
/greger
Arduino board and a mess of wires... (Score:2, Insightful)
Re:Frankencamera. (Score:5, Informative)
Parent
Re: (Score:2)
Or on the other other hand you could just learn how to use your camera and not get such a shit load of blur in the first place.
That's what I always tell people. The raw light they spontaneously emit from knowing what they're doing will go and light up that dim scene, thus letting them use a faster shutter. It's really quite simple.
I thought this only worked when taking pictures over your shoulder and your buttocks were pointed at the subject?
"No more blurry low light photos!" (Score:4, Informative)
Social networking sites are about to get a whole lot more ugly
comparison to other methods? (Score:4, Interesting)
Okay. (Score:3, Insightful)
Great, you can improve your motion blur removing algorithm by recording the motion which created the blur.
Although technically, the blur in the image itself already recorded the motion, with better precision and without calibration issues. So this is more of supplementary data. The before and after images leave out the whole "you can already do this without the extra sensor data" aspect.
And really, you'll get far better results if you just use an adequately short exposure time and some mechanical stabilization. Brace your shooting arm. If you want to get fancy, use something like Canon IS lenses.
Yeah, this is nifty, especially for smartphone based cameras which may already have built-in sensors to do this. But neither is it exactly revolutionary. You'll get better photos out of learning some basic photography than you will out of fancy sensors and analysis software.
Re: (Score:2)
Re:Okay. (Score:5, Insightful)
This isn't for people who want to learn photography and take good pictures, it's for people who are shooting their friends in a bar at night to post on Your Face in a Tube and laugh about for a week before being forgotten -- it's merely intended to allow point-and-click shooting work more reliably in poor conditions on cheap equipment with inattentive and untrained operators.
Parent
Re: (Score:3, Interesting)
Agreed. I feel the same way about auto-focus.
Re:Okay. (Score:4, Insightful)
Parent
Also could well help pros (Score:2)
There are real limits to the human body. Anyone who says "I can hold a camera perfectly steady," is lying. We are not perfect platforms. So image stabilization can help a lot. Long range photography, in particular of fast moving objects like in sports, got a big boost when optical image stabilization came out. The length that you could zoom and still get a good shot increased. Wasn't that the photographers were bad, it was that they were at the human limits. The optical stabilizers enhanced that are upped t
MicroSoft is impressive at SIGGRAPH (Score:5, Interesting)
This reminds me of Xerox PARC - great R & D output, poor commercialization of these results. People wonder if their lab was a toy-of-Bill or a tax write-off.
Now they just need to attach this to Ballmer's (Score:5, Funny)
Now they just need to attach this to Ballmer's head to deblur the company vision a little.
Information théory (Score:3, Interesting)
Information théory tell us that once some info has been lost, it can't be recovered. If the picture has been somehow "damaged" by some motion blur, the original picture can't be reconstructed.
On the image, we'll have much more than the motion blur from the camera's movement : ...
- noise added from sensor electronic noise
- blur from target movement
- distortion coming from lens defect (mostly for low end cameras)
- distortion/blur from bad focus (autofocus in not perfect)
The operation that will reduce the camera's motion blur will probably increase the effect from all other defects. You reduce one kind of image destruction and increase the impact of the other one.
Re:Information théory (Score:5, Insightful)
But they are adding information to the system with the additional hardware attachment with all the gyroscopes and so-on. This information can be used to improve the photo, correcting some of the damage. So information wasn't "lost"'; it was just reacquired from a different source, as it were.
It looks like camera shake blur would be reduced, but target motion blur would remain intact.
Of course, if you do a 90-second exposure of the sun, it's likely going to be all-white no matter how much shake-correction occurs. But this solution wasn't meant to fix that problem.
Parent
Re:lol yea sure (Score:5, Informative)
Microsoft Research puts out a lot of really interesting and successful research. They aren't the people programming the OS or office applications.
Parent
Re:lol yea sure (Score:5, Insightful)
Parent
Even so... (Score:5, Interesting)
Clearly (pun intended) the results have a ways to go yet. Look at the coca-cola image, at the 'a' on the end of the cola... that thing is hosed by the blur, and they're unable to recover it because there's no intermediate contrasting color. Same thing for the spokes on the car rims.
This problem can't be completely solved post-picture. Only large-scale elements with nothing else around them will yield pixel-sharp solutions.
The optimum way to correct blur is to apply active or passive (e.g. tripod) stabilization to the lens prior to the shot; active technology is already pretty decent (photographers tend to measure things in stops; it's intuitive to them... when they say an active stabilizer "gives you" four stops, for instance with Canon, what they mean is that you can shoot four stops slower with the shutter and you won't get blur from camera movement.) Doesn't solve subject movement at all, but then, nothing really does other than cranking down the exposure time.
So... considering lens stabilization has been in-camera for years, and this requires more hardware, but gives you less... I'm going to go out on a limb and say it isn't of interest to camera folks. Maybe in some esoteric role... a spacecraft or something else with a tight power budget where stabilization can't be done for some reason (certainly measurement takes less power than actual stabilization)... but DSLRs and point-and-shoots... no.
Parent
Re: (Score:3, Interesting)
I'm going to go out on a limb and say it isn't of interest to camera folks. Maybe in some esoteric role... a spacecraft or something else with a tight power budget where stabilization can't be done for some reason (certainly measurement takes less power than actual stabilization)... but DSLRs and point-and-shoots... no.
Well, sort of, I disagree somewhat. For starters, take camera phones. What do they need to do this? I'm too lazy to read the paper, but seems like accelerometer data. How many phones come wit
Re: (Score:3, Informative)
Yes. We call that technique "stacking." And it can result in profound improvements. Here is [flickr.com] a before and after of stacking; at left, one normal shot from the camera at pushed ISO 12800 (ISO 3200 with an additional 2-stop digital push, in-camera), at right, the result of combining 36 of those shots and recovering the data through the noise.
Re:lol yea sure (Score:5, Funny)
Probably only half-working coming from microsoft
It could be worse... the GIMP developers could have built it, in which case it would be a mostly working implementation of half the features of some existing software. However, nobody would realize this since only the developers would be able to comprehend the UI.
Parent
Re:lol yea sure (Score:4, Insightful)
Have you used GIMP in past 5 years?
Parent
Re: (Score:3, Informative)
Single-window mode hasn't been released yet, but it's coming. This will make it usable for folks who aren't using fvwm with focus-follows-cursor.
Re: (Score:3, Insightful)
Those people should use a better setup.
Surprisingly enough, different people have different needs.
The lack of auto mouse focus default really makes windows desktop suck, plus the lack of workspaces.
I'm too much of a spaz to use focus-follows-mouse. Every time I try it I wind up bumping the mouse and typing into the wrong window. If I were a hardcode pre-trunk GIMP user I'd definitely have a session set up that way, though. Fortunately, the GIMP developers have come around to an option that works with mos
Re: (Score:3, Insightful)
I only run into the occasional problem with GIMP. They really have come a long way.
I switched from Photoshop to GIMP years ago. Photoshop kept crashing on my machine, and GIMP didn't. Then I found there were more things I could do with GIMP, so I stayed. Once in a while I try out Photoshop again, but I stay with GIMP. A few times, Photoshop folks have run into problems, so I tell them to just send me their file, and I fix it in GIMP and send it back. :)
But he
Bitching about gimp (Score:5, Insightful)
You know, you -- and 99% of the others bitching about the Gimp -- you're utterly full of shit. I write commercial image processing / editing / animation / generation software for a living, I'm expert - you can read that as "terrifyingly exert" - with Photoshop, Gimp and a whole raft of others... and Gimp is an easy to use powerhouse.
Now I will grant you exactly ONE thing, and that is, you need to sit down and learn to use it. That should take a few hours if you're familiar with something (anything) else; maybe a week hunting down tutorials, or a day hanging with a qualified mentor, if editing bitmaps is all new to you.
If it takes you longer than that, you're either stupid or lazy.
There's *nothing* significantly wrong with the Gimp. It has its limits, like everything does (Photoshop has some really annoying limits too), but for the vast majority of image processing and touch-up needs, it's very nice.
Oh, mommie, my crop function is in a different menu... Some people just need a good smack in the head.
If you really knew what you were doing, you'd have, and use, a whole suite of these programs, because for the big ones, there are areas where they excel, and that's the time to put them into play. If you can't learn to use them because the keystrokes are different, or there is a different paradigm... it isn't the program that sucks. It's you.
Also, if you actually knew how to use them, you wouldn't be bitching about them.
Parent
Re: (Score:3, Insightful)
That should take a few hours if you're familiar with something (anything) else; maybe a week hunting down tutorials, or a day hanging with a qualified mentor, if editing bitmaps is all new to you.
If it takes you longer than that, you're either stupid or lazy.
I think i am stupid. I am an occasional user of editing software for my home needs. I do manage to do some things with (a downloaded) photoshop, but I stay new with it HOW to do things. With GIMP I often fail to do it in a reasonable timeframe. Recent
Re: (Score:3, Funny)
It could be worse... the GIMP developers could have built it, in which case it would be a mostly working implementation of half the features of some existing software. However, nobody would realize this since only the developers would be able to comprehend the UI.
If you don't like the GUI, there's always the Lisp interface. If another OSS project gets named after a disability, I'm sure the gimp devs will incorporate it somehow.
Re: (Score:3, Insightful)
Re: (Score:2)
Yes and no. There are limitations to how quickly and accurately the physical IS systems can work. Overall they're fantastic and well worth the premium if you're a serious shooter, but this could provide a much cheaper alternative that could be nearly as effective. Also, provided you have sufficiently accurate accelerometer data, you could reprocess the RAWs as deblurring algorithms improve for better results (check out the difference in noise reduction in the latest version of Adobe Camera RAW). This could
Re: (Score:2)
While it's a nice idea, isn't this just a poor man's image stabilization? Even cheap compacts come with some form of IS these days, and high end SLR lenses certainly do.
I think that the key point here is the 6DOF measurement of the camera movement. I will admit to not knowing about IS, but I would guess that it doesn't handle 6 DOF.
Re: (Score:3, Insightful)
Re: (Score:3, Informative)
You've described blind deconvolution. It does work, but guided deconvolution, a version of which they're doing here, usually works better because you're providing more information. The search space is very large and you have to make assumptions anyway (just how does the computer assess the "sharpness" of an image?) so anything you can do to narrow it down usually improves your results.