Robotic Camera Extension Takes Gigapixel Photos

schliz writes "Scientists at Carnegie Mellon University have developed a device that lets a standard digital camera take pictures with a resolution of 1-gigapixel (1,000-megapixels). The Gigapan is a robotic arm that takes multiple pictures of the same scene and blends them into a single image. The resulting picture can be expanded to show incredible detail."

Not so novel

[Anonymous Coward]

Seth Teller at MIT EE was doing this 8 years ago. Check out his Cityproject.

Re:

[2.7182]

How true. Another example of people in computer vision with no ideas who are stealing from people who don't sell themselves well enough. What a sad subject computer vision has become. See Adam Kropps paper with Seth Teller here on spherical mosaics. [mit.edu]

Re:

[The End Of Days]

Yeah, it's a real shame that people don't take the time to find out every single idea anyone has ever had so they don't duplicate things. What a sad world.

Re:Not so novel

[2.7182]

Well its a good idea to check before they submit for a patent, since there is prior art.

Also, the MIT work is well known to anyone in this area. It's not that hard to google some of the keywords and get the MIT page. The CMU people either knew and ignored it, or they simply didn't do what most of the scientists at their institution usually do, which is read the standard conference papers in computer vision, and browse the web (just a little!!). It's not as if the MIT work was published in some obscure place.

Re:

[yfarren]

How does this get modded insightful when it is nothing but a troll? Seriously Mods, the article doesn't talk anywhere about a patent. This is just some guy trolling trying to start some argument about something wholly unrelated to the use/interest of using a regular camera to get a higher resolution photo. No-one but the poster mentioned patents.

That's a troll. Look at it, see its warts and throw it back under the bridge it came from.

Re:

[yfarren]

Anyone know if there is a way to check if the first guy who modded GP Insightful was the same guy who modded (my own) parent "Troll"?

Re:

[davolfman]

I thought the original was scans from air reconnaissance film, not stitched digital. Or am I thinking of yet another giant picture project.

Re:Not so novel

[Clover_Kicker]

You know what they say, a month in the lab can often save an afternoon in the library.

Re:

[iamhassi]

"it's a real shame that people don't take the time to find out every single idea anyone has ever had so they don't duplicate things"

I know! It's not like you can just do a "search" of the "internet" for words like "1 gigapixel" and get results! [google.com].

say... that would be useful, if there was a website where you could search the entire internet with just a few key words... someone should invent that! I'd use it everyday!



but seriously this isn't news worthy at all. It was 5 years ago when 6 megapixel

Re:

[Pemdas]

Strangely, people seem to be focused on the "OMG it's something that people know how to do already" and missed out on the price point. Yeah, given $15,000, I could develop something that does this by putting together panotools and a commercially available robot arm; that's not the point. The point is, it costs under $300, and that's pretty cool.

Ironically, the people claiming these researchers haven't done their homework seem to have not bothered to look into what claims to novelty are being made, somet

Re:

[GiMP]

I believe that Steve Mann [wikipedia.org] of wearable computing fame was the first to create an algorithm for photo stitching [wearcam.org].

We Did It in 1990

[Doc Ruby]

I worked for a SF area startup in 1990 that produced and sold cameras for "digital prepress" [accessmylibrary.com] (later called "desktop publishing", and now just "publishing" ;) that had the highest resolution around, to compete with drum scanners [wikipedia.org] that were then the expensive industry standard equipment.

We took a 512x512 Hitachi video sensor with a 2x2 C-M/Y-K mask repeated over it, for initial 1Kx1Kx40bit images that we derived from DSP on the intensity of the color-masked pixels. Then we physically stepped the sensor through 8x8 subpixel shifts, subsampling each pixel 64x. We ran the resulting 320MB raw composite files through a bank of multiple 25MFLOPS DSPs (interconnected and logic-accelerated by a fat FPGA) to produce 4Kx4Kx36bit 72MB files. In 1990 that was an awesome achievement.

We poured dramatic engineering work into that platform, which replaced a $150K drum scanner with a $30K PC (on DOS or Win3.0, or plus optional $5K Mac with its GUI including Photoshop 1.0). We had to deal with DSP for micropositioning the video sensor quickly (using feedback data from a laser/interferometer), with new color spaces (I was part of the JPEG org that produced the image format), with custom interconnects at blazing bandwidth, with parallel multiprocessing at then-supercomputer speeds written in C on DOS, and even with the physics of the light variably distorted by turbulence in the air between the camera and scanned slides, heated by the hot lights necessary for exposures fast enough to allow 64 frames and rescan before the sensor wiggled.

All for a 16Mpxl camera that's now beaten by big sensors on handheld consumer devices for under $2K (in 2008, not 1990, dollars). But I can proudly say that we beat them by almost 20 years.

Re:

[ettlz]

I must ask... did you test it on Lenna?

Re:

[Doc Ruby]

Not really, though we did have a Lenna slide kicking around. We primarily used a Kodak test slide of color bars/wheels and greyscale gradients, and one image of a European/American looking blonde on Kodak slide and one image of a young Japanese looking woman on Fuji slide. We had different colorspaces for US/Europe and Japan, because Fuji film had a larger green dynamic range supposedly because Japanese people have more acute green-band vision (though I've never independently verified that).

Once the camera

Re:

[rastoboy29]

That's really cool.  Can I ask you a question?  Not knowing anything about that industry--what was it your machine took pictures _of_?

Re:

[Doc Ruby]

Mostly we scanned photos printed from film, or directly scanned the film slides themselves (two different kinds of film, different scanning params). The scans were used to import the photos into magazines and newspapers. At the time (1990) there were no hirez portable digital cameras for direct digital photography of suitable quality for publishing. They used these $150K+ drum scanners, which were whirling cylinders with the photos taped on, scanned in successive lines with an A/D sensor head (like a printe

Re:

[sjs132]

There is also a program that you can do this with... You take a series of pics, and adjust the focal point as it puts them all together to get super fine detail out of a "normal" camer. Lots of folks use it for pictures of their mineral and rock specimens... The name of the program escapes me at the moment, you may find more infor at www.mindat.org in the forums and/or info on the pictures if you look up a mineral and check out some of the cool pics.

I like the folks that do this, it makes for some supe

Another giga-pixel project

[edesio]

Take a look at http://www.gigapxl.org/ [gigapxl.org]. They take a single photo with more then 1 gigapixel. They are now aiming at 4 gigapixels.

ALE

[pipatron]

Also check out Anti-Lameness Engine, http://auricle.dyndns.org/ALE/ [dyndns.org] which does exactly the same thing, but you have to provide your own arm.

Any superresolution software for average Joe?

[grimJester]

Is there any superresolution software good enough that I could, for example, take twenty blurry pics with my phone and merge them to a single sharp one?

Re:Any superresolution software for average Joe?

[Sitnalta]

Yep. Open Adobe Photoshop and go to File -> Automate -> Photomerge. Then simply point it to the folder containing your photo array.

Re:Easter Eggs and bunnies

[Technician]

Follow the link in the article to the photo. scroll down to the other photos. Look at the mad hatter's photo. At first it doesn't look like much, but it has been photoshopped to include lots of hidden stuff. If you have trouble, zoom in in the sidewalk cracks to get a start. There are at least 2 bunnies in each sidewalk joint. Have fun.

I found the egg in the basket with bunnies painted on it. I still need to find the purple bunny.

Re:

[sumdumass]

I can find all of them but the camouflaged bunny. The closest I can come is the camo colored fabric in the bottom right hand corner saying something about 1at 20 emails would be entered into a a drawing for an easter gift. I'm sure it too late for that but I can't find the camo bunny. BTW, I can tell you where the purple bunny is.

Re:

[loraksus]

Or, instead of paying $700 for the latest version of photoshop, use Autostich [cs.ubc.ca]
It's free and not half bad (ILM even uses it)

Re:

[Phat_Tony]

Autostitch looks interesting for stitching, but unless you used your phone cam with forethought of stitching a panorama, and shot your subject with a dozen overlapping closeups carefully arranged to cover your intended field for panorama stitching, it's not going to help here. I take it the grandparent poster has a bunch of pictures of essentially the same composition, that are all blurry. Photoshop will auto-align these for you, but adjusting a bunch of aligned blurry pictures to increase the apparent reso

Re:

[commanderfoxtrot]

If they are close enough, yes.

Look up image stacking and also there's a tool called "ale".. but I can't find the link right now.

Re:

[darenw]

http://auricle.dyndns.org/ALE/ [dyndns.org]

Re:

[darenw]

there's the "drizzling" technique in astronomy. i don't have my notes on it handy, but google it and you will find.

Re:

[Fishbulb]

Yes, Hugin [sourceforge.net].

Of other interest is the PanoTools Wiki [panotools.org].

Note however, that you can't make cake from crap. 'Garbage in, garbage out' as the saying goes. The whole concept of a camera on your phone, to me, is like having a television on your fridge.

Re:

[Ceriel Nosforit]

It's called image stacking, and is used a lot in astronomy. There exists freeware, or open source, which will do it for you.

Re:

[4D6963]

Is there any superresolution software good enough that I could, for example, take twenty blurry pics with my phone and merge them to a single sharp one?

I don't know what pics made by your cell phone look like but I'd say they probably don't have a lot of aliasing going on, which, if I'm not mistaken is necessary to apply super resolution techniques. The article or even CMU's page on the topic is very light on details, but it seems to be more about panorama than true super-resolution techniques, that is recovering aliased high frequency components by comparing many aliased images.

So in that sense, sure, you can make panoramas out of your blurry cell phone

Re:

[SpinyManiac]

PhotoAcute [photoacute.com]. It supports Windows, Mac and Linux via Wine. It does noise reduction, resolution increase, moving object removal and HDR. Very bad HDR in my experience... ;)

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:

[tomhudson]

You need to load the AgentDeckard module into your BladeRunnerCam.

Wow

[Sitnalta]

So, basically it can do the exact same thing as Photoshop, except with the added expense and complications of a robotic arm. Way to go, Carnegie Mellon.

Re:

[ColdWetDog]

So, basically it can do the exact same thing as Photoshop, except with the added expense and complications of a robotic arm. Way to go, Carnegie Mellon.

Which is rather ironic since the Photomerge routine in Photoshop CS3 is quite adept at taking multiple hand shot images and stitching them together. Traditionally, photographers have used leveling tripods and paid careful attention to exposures. While this can lead to better results than a hand shot and stitch, the latter is awfully good. The intelligenc

Re:

[FrenchSilk]

Well, as I said in my earlier post, it is next to impossible to do this without a robot if you are using a long lens to take a large array of photographs that are closely spaced. And as to viewing gigapixel images, there are numerous ways to do it. You can use zoomify, as in this image: http://www.donfrenchphotography.com/Zoomify/HalfDome2D.htm [donfrenchphotography.com] where you can zoom in close enough to see people standing on top of Half Dome. Or go to the gigapan site (http:\\www.gigapan.org) to see many examples of gigapi

Re:

[solitas]

You don't want to move the camera horizontally & vertically - more like PIVOT it around the focal point of the lens so you don't get any perspective offsets (i.e. what your two, or more, eyes do to get binocular/multiocular vision).

$279, applications for beta testing open

[grimJester]

Here [cmu.edu]

Open until October 2007, that is... D'oh!

[grimJester]

Nevermind...

Except that it ended in Oct 2007

[lullabud]

The beta sign-up ended October 19th, 2007. At 6 months almost to the day this story is hardly news.

1000 megapixel?

[duguk]

From TFS:

take pictures with a resolution of 1-gigapixel (1,000-megapixels)

Where's the other 24 megapixels? ;)

caveat: yes, i know. don't start. it was a joke. don't link to wikipedia to explain, besides; xkcd explained it better. [xkcd.com]

Re:

[symbolic]

Cisco has them.

Link?

[Patentmat]

To some sample pictures?

Re:

[Jeff321]

The article links to a sample located at http://www.gigapan.org/ [gigapan.org]

Re:

[penguin king]

h einThat is actually a fairly awesome photo if you look at it. Given the scope of the photo, the fact that there is enough resolution to zoom in far enough to read the time on the clock (on that little tower in the upper corner) and also make out the individual Roman numerals on its face is fairly cool! In fact you can also see in the individual windows... hrmm privacy debate anyone :p

Some Links

[Rufus211]

This is a pretty cool project, and I actually saw it when I was at CMU a bit ago (and was wondering what the hell it was).

There's a CMU press release [cmu.edu] about it.
The site with all the pictures is http://www.gigapan.org/ [gigapan.org]
You can see the hardware here [charmedlabs.com].

The only problem with this, and any other multi-picture stitching, is that you get obvious stitching problems when there is any movement in the scene, like the trolley in the middle of this scene [gigapan.org].

Movement indeed

[Fortran IV]

For some real vehicle distortion, remember Scanner Photography? [awardspace.com] Michael Golembewski built high-res cameras out of flatbed scanners; the model he described on this site took a 115-megapixel image--with each exposure.

Re:

[Skapare]

How about some real images instead of some plug-in program? The web already supports image files, so there is no excuse for forcing flash on people. Once Firefox developers figure out how to display video, then we can finally get rid of flash.

Higher Resolution != Higher Quality

[Manip]

I think this just proves that higher resolution doesn't result in a higher quality photo.

If you look at the entire photo it doesn't look any better than a regular photo even if it contains much more information.

For years now there has been a push to larger and larger resolution photos with people often mistaking this with "quality."

All a higher resolution really allows you to do is zoom in more after a certain point. Which is awesome from a photo editing point of view, but for most people unimportant.

What you really want to be focusing on is the lens quality, zoom quality (lol Digital Zoom), noise, and other characteristics of the camera (e.g. ISO rating).

So it is great that they spent lot's of time doing this but it isn't all that interesting to average Joes or even serious photographers. We all really want better quality pictures, not bigger ones.

Re:

[religious freak]

Ain't that the truth? I learned this the hard way when I gave up my wonderful Canon (which was just too old) for a crappy Casio. Went with the higher megapixels without truly educating myself first.

Re:Higher Resolution != Higher Quality

[RichardKaufmann]

You're confusing three different aspects of quality:

1. Resolution (the number of pixels in am image, here increased by stitching overlapping images)
2. Dynamic range, color fidelity, noise (the quality of a particular pixel). This can be somewhat ameliorated by HDR photography or just averaging identical shots (all with no moving subjects and a sturdy tripod). Google Photomatix for details.
3. Whether the shot is interesting, well composed, in focus, without motion blur, etc. Panorama photography is most interesting for its artistic potential; more pixels is just a delightful side effect.

#1 and #2 can be addressed by money and a willingness to prostrate yourself to the camera gods. #3 requires talent!

And to put a final nail in the megapixel coffin: check out http://www.luminous-landscape.com/essays/Equivalent-Lenses.shtml [luminous-landscape.com] (particularly Nathan Nyhrvold's comments) for a discussion of how sensor size and f-stop place an upper bound on resolution irrespective of sensor density. Physics can be a pain sometimes!

Re:

[Skapare]

And also high dynamic range [panoramio.com].

Re:

[v1]

When you get to the highest resolution the image is terrible. It looks like a muddy smoothing of a blocky jpeg. You can't say this is a gigapixel image much more than you can say a 640x480 tiff expanded to 400,000x250,000 pixels (in its blocky glory) is a gigapixel image.

I was expecting to see good quality all the way down to the highest zoom. something like google-earth quality for the most part. They don't let you just keep zooming in past the point where the resolution has hit the wall like this does

Re:

[loraksus]

At a certain point, you're going to run into a point where air turbulence, haze, lens quality, etc affect your image and nothing can avoid that.
What should be done is to limit the zoom to that point (even if the image can technically zoom in further)

Take for example, this pic (it's big - and I sort of hate my webhost, so feel free)
http://vehiclehitech.com/pictures/!%20Photography/2008-02-24-KelownaSkylinePanorama-attempt1.jpg [vehiclehitech.com]
I shot it with a $100 lens, so at 100% it doesn't look too great (the jpeg artifact

Re:

[v1]

My point though is not complaining about the actual quality, but to the advertised quality. That image you linked to is a 46 megapixel image. At your recommended 35% scaling to look clear, it's a 5.7m megapixel image. Take that scale image and export it and then open that and zoom it back up to 46 megapixels and smooth it, and it looks about as good as the original 46 megapixel image. That illustrates the point that you can make a higher megapixel image based on a lower megapixel source, but that doesn'

Re:

[loraksus]

The thing is... higher resolution - and by that I mean a larger number of pixels AND a lens that can actually give you enough detail to utilize it will improve pictures.
Too many lenses nowadays are crap - details are blurred and muddy even before you zoom in. It's difficult to get a decently sharp picture, even if you shoot in RAW and do a fair bit of post.

This detail is (imho) important in landscapes, and having this detail is an important part in getting pictures to "pop". Portraits too, but obviously thi

Re:

[SoupIsGoodFood_42]

No affordable lens around has the resolution I want for landscapes. Photo stitching like this is the only way I can get the images I want.

misuse of word resolution

[cinnamon colbert]

to my understanding, resolution refers to a mapping of the object on to the image - resolution is a ration of 1 cm in object/x cm in image
it has nothing to do with pixels per image, although you can have more of the object , at the same resolution, with more pixels

Re:

[vandoravp]

Sort of. Resolution is the fineness of information that can be resolved given a fixed size. Keep in mind that number of pixels is not exactly image size, which can be described in cm or inches depending on the context. For example, a 10in by 10in image at 300dpi will have a much higher density of information when compared to a 10in by 10in image at 72dpi. So, these gigapixel images are potentially much higher resolution, depending on the physical dimensions it is presented at. Compared to most graphics

Big Deal!

[Evildonald]

I've been doing this for years with a film camera and then sticky-taping all the photos together. Then when i want to "zoom in" I just move my head closer to the picture.

Another entry into the market is always welcome

[AsmordeanX]

That's not really all that new. Motorized panorama heads have been around for a long time. People have even built them from Lego Technics.

As an avid pano/gigapixel photographer myself I'm interested in any new entry into the excessively priced head market. I'm using a Kadian Quickpan Pro that cost me $400 a few years ago. An automated system would be very nice but the cost is usually horrific. I've even had a head custom built at one point.

As for the use, I like to take big pictures. I have a 6ft x 3ft print hanging on my wall. The print is 400dpi taken from a 43000x22000 (just shy of 1GP). People see the picture and say it looks nice then walk a little closer, and closer, and closer. Pretty soon they are standing 4" away and excitedly reading the serial number on the front of a train car that is only 2" across on the print.

Re:

[v1]

what kind of a printer does it take to print a high res monster like that?

Though most real people are unlikely to afford such a camera, is there any way to borrow/lease/rent one?

Re:

[Skapare]

what kind of a printer does it take to print a high res monster like that?

This kind [epson.co.uk].

Re:

[AsmordeanX]

For the camera I use a Canon 300D (recently upgraded to a 450D) and a 300mm lens. I've seen people do mosaics with a 600L lens to create a virtual 50mm lens that would crash most PCs even trying to load, let alone actually stitch.

I love PTGui

Re:

[Skapare]

Had Canon finally added auto-bracketing in the 450D?

Re:

[SoupIsGoodFood_42]

It's not fully automatic -- you still need to fire the shutter 3 times, but otherwise, yes. At least on the 400D.

Re:

[Skapare]

I want to minimize any vibration in taking the shots. Once the mirror is up, what I want is for it to just do all the bracketed shots. I could use a shutter cable or whatever. This will be on a tripod. I just want those shots to all be framed as close alike as possible.

Re:

[niceone]

On my 350d if you set it to continuous shooting and AEB it takes all three shots when you press the shutter button once. But it does put the mirror down between shots, and setting the Mirror Lockup custom function doesn't seem to work well with AEB - there's no way to avoid the mirror going back down between shots.

Also ...

[Skapare]

... you might want to read more here [wikipedia.org].

Re:

[rew]

How much did it cost to make that print? Where did you have that done?

That's great for still life

[davidwr]

It doesn't work well for action shots though. Well, I guess it could if you eliminated the "single camera" requirement.

Besides, when we get to 10,000 dpi by about 12 bits per color, we will be as sharp as film. A 150 megapixel, normal-sized, 36-bit camera is probably 4-6 years away in the sub-$500 consumer market, sooner in the professional market.

Of course, for normal consumer 4x6 prints with no cropping, you don't need nearly that level of detail.

I can't wait!

[rakzor]

For the 1000-megapixel porn industry.

This and tourist remover...

[Anonymous Coward]

Combine this with tourist remover http://www.snapmania.com/info/en/trm/ (take several pictures of the same scene, and only use the bits which are stationary).

That would make large pics without the motion distortion.

Posted a similar story once...

[thrill12]

here... [slashdot.org] - what's new ?

You can read the magazine covers...

[dogdick]

thats pretty amazing...
There is also a woman blowing a guy in one of the windows.

Redundancy?

[PingPongBoy]

Incredible detail or incredible redundancy?

When I think of detail, I think of zoom. Multiple pictures can help define some fuzzy areas, and assuming the subject doesn't change, correct for atmospheric distortion. However, as the naked eye staring at a distant object can't quite make out what is being seen, a whole bunch of fuzzy snapshots aren't going to give any big confidence improvement.

Aperture synthesis [wikipedia.org] involves simultaneous processing of light to zoom by adding light that is in the same phase [wikipedia.org]. A digit

Autostitch does it automatically

[electrostatic]

Autostitch [cs.ubc.ca] "is the world's first fully automatic 2D image stitcher." The order in which you take the photos in not important, just that you cover everything and that there is plenty of overlap. You don't have to worry about keeping the camera horizontal -- it will rotate individual shots as needed. And you can ZOOM in on certain shots for more detail. I've used it to merge 154 shots into one panorama. Free.

Autostitch for higher res inside consumer cameras?

[ErkDemon]

Wow, that looks fun!

So I guess, if you wanted to use multiple images to boost the resolution of a given scene past the sensor resolution of your camera, you'd take a stack of near-identical photos, use an editing package to blow them all up by a factor of, say, five or ten, so that all the photos show big square single-colour blocks at high magnification that correspond to single pixels in the original images, and then when the software has found the "best-fit" alignment and rotations for the different im

Gigapixel picture != Gigapixel camera

[SamP2]

True, you can put together a lot of composite pictures to achieve an arbitrary size and resolution. You can legitimately call the result a gigapixel picture (if it reaches the resolution requirement, of course). But that shouldn't be confused with a gigapixel camera, and in and by itself is not actually that impressive.

For example, if you take the entire photoset of Google Earth, you'd probably get a few peta-pixels worth of data. Ultimately, it all boils down to how much of that data is needed at any given time. You might need a low-detail, large-area image (e.g. view of Earth as a whole), or a high-detail, small-area image of your backyard. In either case, you wouldn't need more than at most a few dozen megapixels at any given time. It's unlikely anyone ever needs more than that size, whether they are studying galaxies or atoms, because the more detail you need, the less physical area you need covered, and vice versa.

Similar to Microscope Technology

[Sir Holo]

There are microscopes that play similar tricks. We have a couple the will compose images from 3x3 mosaics which are taken automatically by the camera.

Cameras that automatically do sub-pixel shifts between frames (for resolution) and that do frame-shifts (for large images) are commonly available in the marketplace.

Some others will instead bracket focus and automagically composite an image with a huge apparent depth-of-focus.

So its

[Kamineko]

A little motory gadget thingy. In the meantime, get yourself some Autostitch [cs.ubc.ca].

Where not to use your Gigapan

[jgs]

Tangentially related, here's [andycarvin.com] a blog entry by an NPR staffer who was harassed and threatened with arrest for using a Gigapan in Union Station.

NPR reporters almost arrested for using Gigapan

[GrahamIX]

Meanwhile, some NPR reporters using the new gigapan camera were almost arrested for taking pictures with it at Union Station.

www.andycarvin.com [andycarvin.com]

Another symptom of the knee jerk reaction against anything and everything unfamiliar in the War on Terror.