Google Two Years Into Overhaul of the Google File System 217
El Reg writes "As its ten-year-old file system — GFS — struggles to keep up with Gmail, YouTube, and other apps it was never designed to support, Google is brewing a replacement. According to the company, it's two years into a GFS sequel designed specifically for customer-facing apps that require ultra low latency."
Re:Curiously (Score:2, Interesting)
"..but dividing a gigantic database into pieces that are 64 time smaller doesn't make intuitive sense..."
It does if it was 64x too big to begin with. Live and learn.
Re:hmm (Score:5, Interesting)
They use the Linux platform to the absolute max, leveraging all the blood and sweat Linux developers poured into its development over the past 15 years, and yet, not contributing back any of their most significant enhancements.
i see your point, but its not like google isnt giving signifigantly in return. most people would be hard pressed to deny that Googles search engine was a game changer in the interweb. at its release it was leaps and bounds better tahn just about anything out there, and is still the gold standard for finding information. hell they gave us the verb "to google" we got a pretty decent browser out of it, gmail, google docs, google maps, and a whole bunch of other stuff they've generated. not to mention a forthcoming OS. at this point i can already hear critics screaming about Googles profits driving these services, and you know what, maybe they are, but i havent paid Google a dime, and most likely, neither have you. i dont care if they make money, theres nothing wrong with it, and i'm even happier that they make money without involving me whatsoever. in many ways i would think Google would be a champion to the FOSS community. so they want to keep a filesystem proprietary, frankly thats not so bad, competition is good but competitors arent usually. Google is a good counter balance to Microsoft and other would-be owners of the interwebs. are they "good" as in saintly? no, but they never claimed to be, they claimed "dont be evil" i'd say they're pretty far from that.
Re:Google is IT done right... (Score:3, Interesting)
Because they're not allowed to share their ideas with IT, and vice versa. I can't list the number of times developers have published brain-scrambled vomit as part of their projects, because it didn't interest them and no one with experience was around to explain the inevitable problems. The maintenance model for subversion where you have to completely rebuild the repository to completely delete an accidentally stored DVD image is a classic example.
Conversely, I've expressed extreme doubts about projects that turned out to be effective and workable because my knowledge of file system behavior or hardware limitations was 3 weeks behind the times. I even spiked a project once for such reasons, although when the developer and I spoke without the confused manager in the way it became clear that the hardware _could_ support his needs.
Re:hmm (Score:5, Interesting)
You clearly weren't a daily Google user 10 years ago.
The moment I realized Google was completely superior to the others was when I was able to paste an obscure compile error for an equally obscure CPU architecture into Google and immediately get the answer back... the kind of utterly random error that a few years previous would have potentially taken hours to debug...
If Google hadn't come along when it did, someone else would have stepped up. Maybe Altavista, or Yahoo
And you were modded Insightful - sigh... So you are saying they decided "oh, well Google is pretty good at this - let's NOT STEP UP." Yeah, that's what companies do in that situation. Or maybe they do try, and fail (nothing wrong with trying and failing... but that's the REALITY of the situation).
Re:Google is IT done right... (Score:5, Interesting)
Not really, it's IT done by not letting anyone over 30 or with any experience into the room. Every single issue they had to learn and fix mentioned in the article is quite literally standard textbook stuff in distributed systems, and has been for over 40 years. The failure model, the huge chunk sized, the single master problems... etc. Nobody who had taken even one decent class would have ever considered the original design viable.
They really should just stick to buying their tech pre-made like everything else Google is known for - acquisitions [wikipedia.org]. Other companies are willing to hire experienced people. You know, those old lazy bastards that only work 40 hours a week because they have families, cost way too much to provide health insurance to, but get things done 5x as fast because they have done it before :)
Better back your files up (Score:3, Interesting)
I have not doubt that the new file system will be great, but after reading this summary, the first thought I had was that I should back-up all that Gmail before they cut it over. I've been putting it off for far too long, but I'll just have to burn a couple of days of attention to do it.