Slashdot Log In
How Google's High Speed Book Scanner De-Warps Pages
Posted by
ScuttleMonkey
on Fri May 15, 2009 03:17 PM
from the onto-dewarping-brains-next dept.
from the onto-dewarping-brains-next dept.
Hugh Pickens writes "Patent 7,508,978, awarded to Google, shows how the company has already managed to scan more than 7 million books. Google's system uses two cameras and infrared light to automatically correct for the curvature of pages in a book. By constructing a 3D model of each page and then 'de-warping' it afterward, Google can present flat-looking pages online without having to slice books up or mash them onto a flatbed scanner. Stephen Shankland writes that the 'sophistication of the technology illustrates that would-be competitors who want to feature their own digitized libraries won't have a trivial time catching up to Google.' First, a book is placed on a flat surface, while above it, an infrared projector displays a special mazelike pattern onto the pages. Next, two infrared cameras photograph the infrared pattern from different perspectives. 'The images can be stereoscopically combined, using known stereoscopic techniques, to obtain a three-dimensional mapping of the pattern,' according to the patent. 'The pattern falls on the surface of (the) book, causing the three-dimensional mapping of the pattern to correspond to the three-dimensional surface of the page of the book.'"
Related Stories
[+]
Idle: Prisons To Get Bottom Scanners 3 comments
In an attempt to stop prisoners smuggling mobile phones into jail, Britain plans on introducing bottom scanners. Prisoners will have to sit on the scanners (chairs), called Body Orifice Security Scanners, which bleep if their subjects have a phone hidden inside them. The £6,500 chairs are going in 102 jails across Britain, and can also be used to detect drugs and weapons. The chairs are very reasonably priced when you think of the savings on latex gloves, lube, and anti-bacterial soap they provide.
[+]
Technology: Cheap Scanners Can "Fingerprint" Paper 88 comments
carusoj writes "Researchers at Princeton University and University College London say they can identify unique information, essentially like a fingerprint, from any blank sheet of paper using any reasonably good scanner. The technique could be used to crack down on counterfeiting or even keep track of confidential documents. The researchers' paper on the finding is set to be presented at an IEEE security conference in Oakland, Calif., in May."
Update: 03/10 22:43 GMT by T : J. Alex Halderman, Associate Professor of Electrical Engineering and Computer Science at the University of Michigan and one of the authors of the study, writes with more: "My group has just put up a site about the work and a copy of the full paper, and we will probably add a video later tonight."
[+]
Technology: Google's Book Scanning Technology Revealed 100 comments
blee37 writes "Last March we discussed Google's patent for a rapid book scanning system. This article describes and provides pictures of how the system works in practice. Google is secretive, but the system's inner workings were apparently divulged by University of Tokyo researchers who wrote a research article on essentially identical technology. There are also videos of robotic page flippers and information about how Google wants to use music to help humans flip pages."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Patent!!??!! (Score:5, Funny)
Re:Patent!!??!! (Score:4, Funny)
So why didnt you do or patent it before?
Parent
Re:Patent!!??!! (Score:5, Informative)
I believe the pattern barcode scanners use is simply trying to look for the barcode in several different directions, but I could be wrong.
I also believe there's either rudimentary correction for common types of distortion (i.e. on cylindrical objects) or just wide enough tolerances to allow it to work anyways.
Parent
Re:Patent!!??!! (Score:4, Informative)
Parent
Re:Patent!!??!! (Score:5, Informative)
You jest, but this technique *has* been around for years. I remember when digital cameras first became available there was a product that could perform a 3D scan by projecting a pattern onto the object and using an offset picture. I think the pattern came on a slide - that's how long ago it was! Here's a whole wikipedia page about the scanning technique: http://en.wikipedia.org/wiki/Structured_Light_3D_Scanner [wikipedia.org]
This picture is especially good: http://en.wikipedia.org/wiki/File:6-seat.jpg [wikipedia.org]
Anyway after reading the patent abstract, it isn't about the 3D scanning at all, it appears to be about an algorithm to find the fold once you've already got the point cloud. I would have thought that was fairly trivial. A possible approach would be to take the radon transform of the height map and find the smallest value that's roughly in the middle.
Parent
Re: (Score:3, Funny)
Whoa, "radon transform"? Hold on a second, wiz-kid. Does that use poisonous gas or something? It's certainly not mathematics, because that means stuff like "three times four".
Re:Patent!!??!! (Score:4, Informative)
It certainly is mathematics and it's not that hard to understand either. basically it is the mathematical equivilent of what a hard field tomograph does.
Consider a function of two values and consider those values to be 2D coordinates. Consider also that the function is zero outside of a defined area.
Now consider that there are an infiniate number infinitely long number of straight lines passing through that area and each can be defined by two parameters, an angle and an offset from the orgin in the direction perpendicular to the line.
Along each of those lines an integral can be calculated. those integrals form the radon transform of the function (with each integral being identified by the two parameters).
Not really that complicated, the trickiest bit is probablly deciding how best to approximate the line integrals from your limited number of data points.
Parent
Re: (Score:3, Insightful)
I almost feel bad. I know what a radon transform is and I've taken a class on inverse problems.
My point was just that the common view of what is mathematics is rather anemic and quick to give engineering credit to relatively simple ideas. I suspect that the patent office has similar fallacious thinking.
So... (Score:5, Interesting)
They've been making "anti-copy paper" designed to defeat optical scanning for years now, surely something similar in the IR band could be effected...
Re: (Score:3, Insightful)
Maybe those books are less important to commit to a digital scan ;-)
Re:So... (Score:5, Insightful)
Failing that there are alternative methods that might work as well.
Parent
Re:So... (Score:4, Interesting)
With respect to the foolishness over "copy protection" it is interesting to consider the possible application of the old line "the worse, the better." [wikipedia.org] The idea is that, in order for a bad situation to change, it must get worse, so that the cost of tolerating it becomes unbearably high. As long as DRM and anti-copy paper, and macrovision and all the others cause relatively limited customer displeasure and support calls, there will be little incentive to change, and things will remain as they are. If you can drive the content guys to ever more intrusive measures, things might actually get bad enough to spur a blowback.
Parent
Patent? Prior Art? (Score:3, Insightful)
Wasn't this a Sci-Fi movie staple back in the 80s? They used it for body and object scanning, not books...but still.
Re: (Score:3, Funny)
The New Bell Labs? (Score:5, Interesting)
I've read many comments over the years about the old Bell Labs and how a huge amount of pioneering research came out of them over the course of their existance, i.e. before they got axed.
It would seem that Google Labs is performing somewhat the same function, albeit more oriented towards software rather than physical research.
Re: (Score:3, Interesting)
Doesn't Google have something called the 20% policy or something like that? Where Google engineers devote 20% of their time to non-Google projects?
Not exactly basic research, but not necessarily commercial applications.
The closure of Bell Labs is
Mostest importanly... (Score:4, Interesting)
...who's flipping the pages?
Re: (Score:3, Funny)
I heard from some guy, somewhere, that on weekends the Oompa Loompas do it.
What are the chances... (Score:4, Interesting)
...that Google licenses this to scanner manufacturers and we see this at a consumer level at some point in the future? I know I'd pay good money for a book scanner that doesn't need to have a 'book edge' (which you already have to pay through the nose for)...
Why is this a big deal? (Score:5, Insightful)
I don't see why this is such a showstopper for other book scanning projects. Right off the top of my head I can think of three methods of dewarping book scans that have nothing do to with Google's methods. While Google's method is definitely quite interesting and seems like a great solution, it is by no means whatsoever the only way of accomplishing this.
Re: (Score:3, Insightful)
No one said its a big deal, its simply a 'neat' way to accomplish the goal. As geeks we are generally interested in these neat ideas.
No one said Google was evil for patenting it.
No one said Google now has a monopoly on book scanning.
No one really said anything other than 'this is how they do it' and we all said 'neat'.
cool, but not patent-worthy (Score:4, Insightful)
This is useful and interesting, but doesn't seem particularly novel.
Projecting a known pattern onto a surface or using multiple cameras to determine the shape of a surface have been around for quite a while, so adding it to an OCR system doesn't seem like a big deal.
But can they remove finger-scans and hand-scans? (Score:3, Interesting)
De-warping sounds useful, but there are problems that it probably won't solve --
Like the operator who scans a book page with his/her fingers or hand stuck between the page and the scanner-glass. For example, the dreaded 'New York Hand' or its fingers can be seen occupying the place of part of the text or figures on many pages of books scanned for Google-Books from the New York Public Library. On some pages, the impression of the fingers is clear enough to show the rings worn by the Hand that was doing the scanning. :(
It will take more than a de-warping patent to solve that one .....
-wb-
Wood chipper? (Score:3, Funny)
This is way better than my idea, which was to throw the book into a wood chipper, scan the results, and then algorithmically reassemble them...
Re:IMPORTANT QUESTIONS (Score:4, Funny)
The same way as your face.
Parent
Re:Playing Catch-up (Score:5, Insightful)
Parent
Re:Playing Catch-up (Score:5, Informative)
Notably, for instance, there has been a fair bit of interest, for some years, in using digital cameras in concert with projectors, either for automatic keystone/distortion correction, for projectors that aren't perfectly aligned with the projection surface, or for automatic coordination of multiple projectors illuminating the same surface, without laborious manual tiling adjustment. This is, in essence, an equivalent problem(inferring a surface's geometry based on pictures of a known image projected upon it).
The IEEE has held "Projector-Camera systems" workshops since 2003 [procams.org], and somebody was obviously working on it before that. I'm not saying that Google's patent falls into asshole troll territory or anything; but the notion of doing surface geometry inference based on known image projection isn't nearly as novel as it might seem.
Parent
Re: (Score:3, Insightful)
This may be a projector thing, but they are doing something of physical manipulation. It would be pretty much appropriate to be patented. The whole thing is physically transformative. Meanwhile, if someone made their own version using something different, it too, would be patentable/improvement patent, which is how the patent system is supposed to work.
To be clear, I'm saying the system as a whole should be patentable (infrared), but not the software used to decode it.
That reminds me (off topic) (Score:3)
Totally off topic here but I'll risk it.
It really bothers me that neither Rock Band nor Guitar Hero can auto-calibrate the audio lag using the microphone. There's absolutely no reason I can see that they can't "listen" for the calibration beeps with the mic to get a perfect reading.
Re:Playing Catch-up (Score:5, Funny)
This is actually what I envisioned for a book scanner, years ago.
But unlike Google, I...
1) Never built it.
2) Am not facing lawsuits from overzealous sue-happy publishers.
Seems like a good defensive patent to have.
Parent
Re:Playing Catch-up (Score:5, Interesting)
This trick has been used for 20 years in astronomy. You shine a really powerful laser of known metrics into the sky and measure the atmospheric distortion suffered by the beam.
Then you take those numbers and calculate what it would take to even out the beam, and you feed THAT set of numbers to a telescope with adaptive optics which will then correct for the atmospheric distortion. Bingo, suddenly your telescope is able to take sharp images without having the air screw it up.
The technique is very effective and results in ground-based telescopes that rival anything the Hubble can do. Plus they are easier to fix.
I want to say this is called Guidestar but I am not sure.
Anyway the similarity to Google's process is simply that you shine a light or image of known value on something unknown and look at how the image now deviates from what you expect. A little math and suddenly you know exactly the shape of the unknown object. Brilliant.
Parent
Re:Playing Catch-up (Score:5, Informative)
It's simply called adaptive optics (AO). In AO, a guidestar is a natural isolated point-like star that is close to your science object (what you are trying to look at). If a laser is used to excite the sodium layer to create an artificial reference, it's called a "laser guidestar".
Anyway, this "trick" is completely different from adaptive optics in both the mathematics and implementation.
Parent
Re:Playing Catch-up (Score:5, Interesting)
Word.
I was involved in evaluating rare books back around the turn of the century.
I can personally attest that representatives of online book search companies were attempting to buy up one of a kind pieces for destructive scanning.
There was one dealer in possession of a somewhat flawed, but well examined Shakespeare folio that had to put the kabosh on a reputation making deal because he found out the buyer was going to slice the piece out of its binding for scanning.
I turned down a much smaller offer on a much less significant, but still very cool, two hundred year old angler's guide (with hand colored plates and original binding) for the same reason.
Quality scans without destruction can only help raise the profile of rare books and the value they offer society - not simply for their content, but as tangible examples of the evolution of the art of communication.
Parent
Re: (Score:3, Funny)
If you were a rare book expert during the turn of the century, why isn't your slashdot ID smaller?
;
Re: (Score:3, Informative)
Really? Structured light to find 3D geometry is old hat ... the optical and signal processing part of book scanning seem pretty easy, making the mechanical part for page flipping robust seems a lot harder to me.
Re:Obvious question... (Score:4, Funny)
That's cool and all that, but who (or what) flips the pages?
Interns.
Parent
You laugh, but look at this (Score:5, Interesting)
That's modded funny, but take a look at this. [google.com]
Maybe they use automated page turning machines for normal books, and turn pages by hand for older/more fragile works?
Parent
Re:You laugh, but look at this (Score:5, Funny)
Now THAT'S a page turner.
Ba dum dum. Thanks, I'll be here all week! Try the veal, and don't forget to tip your waitress!
Parent
Re: (Score:3, Informative)
Re: (Score:3, Interesting)
This is another one.
http://www.treventus.com/index_en.html [treventus.com]
http://www.youtube.com/watch?v=hlOQuuLYavY [youtube.com]
Re:Unnecessary? (Score:4, Interesting)
Pages lie different from the front to the back of the book, and books are bound differently. So you can't use a generic model and expect it to be accurate in most cases.
I actually think this is really cool because it seems to account for any scenario, including folded pages, I would assume. Although, I suppose that in extreme bends it might not be perfect, but certainly they just need to ensure that pages are adequately flat. It automates the entire process.
I wonder if they've built an automated page-turning mechanism; I would assume they have. Just drop in a book and let the machine go to town on it.
Parent
Re:Why? (Score:4, Insightful)
Ok, is it just me, but wouldn't it be easier to just cut the spine off the book instead of developing a whole new way of scanning it?
With 7 million books, the manpower and time saved for them to cut the spine off would be worth it.
Also, they can resell the books if needed or give them charity after they are done.
Kind of would be a waste of a paper to tear that many books apart.
Parent
Re: (Score:3, Insightful)
Re:Why? (Score:5, Informative)
Parent
Re: (Score:3, Interesting)
Only if Google refused to license it. Google isn't Microsoft or Intel; I doubt they'd go that route.
In fact, since Google has paid for the innovation of this tech, including the R&D for it, patenting it and then allowing companies to license it reduces the barrier since companies that couldn't have paid for the research now have the technique available to them.
Re:Butt what about... (Score:4, Funny)
Is this what the graphics department is talking about bump mapping?
Karma burn.
Parent
Re: (Score:3, Funny)
Re:As a writer, I did not give my permission to co (Score:4, Interesting)
Cough, you don't ahve to. I can copy your book all gad damn day long and have not violated your rights or the copyright code.
The moment I try to distribute them, then it's a copyright violation.
It's called copyright, because the only reason one would copy it was to distribute it.
Backup really wasn't an issue then like it is now.
Parent
Re:As a writer, I did not give my permission to co (Score:4, Informative)
Be sure to check out the exclusive rights in copyrighted works [cornell.edu] before making blanket assertions on what is and is not legal under copyright law. The exclusive rights granted by copyright include both reproduction and distribution. There are lots of exceptions to these exclusive rights, but an interpretation that completely eviscerates the exclusive right to reproduce a work is not supported by the Copyright Act.
Parent
Re:Isn't that all known? (Score:4, Interesting)
Building 3d computer models by stereoscopic analysis of project light patterns is at least twenty years old. In fact it mentions in the summary that it they use an established technique.
As for your second comment... that's kind of my point. Since the technique is not new, the equipment is not new, what did google do that was new? Perhaps there is some actual invention in the process somewhere; but I don't have enough faith in the patent process to unquestioningly ASSUME that there is.
Parent