SKA Telescope Set To Generate More Data Than Current Net 73
angry tapir writes "The forthcoming $2.1 billion Square Kilometre Array (SKA) radio telescope could generate more data per day than the entire internet when it comes online in 2020, according to the director of the International Centre for Radio Astronomy Research (ICRAR), Professor Peter Quinn. SKA — which Australia with New Zealand and South Africa are competing to host, and which will help the search for Earth-like planets, alien life forms, dark matter and black holes — will be 10,000 times more powerful than any telescope currently used. Slashdot has previously discussed the proposal to use 'Skynet' — a grid-computing-based solution for processing and storage."
Sensationalistically inaccurate article... (Score:4, Informative)
The project is expected to deliver up to an exabyte a day of raw data, compressed to some 10 petabytes of data in images for storage.
So, 10 petabytes of data - who cares about the raw source. I work for a video streaming company and we have several petabytes of H.264 video. If that were to be uncompressed into 30 FPS 1080p raw data, it would be 50-100x that, so already approaching a couple hundred petabytes. And think of all of the JPEGs out there - why don't we just uncompress all of those for the comparison as well?
A (likely conservative) back of the hand calculation by Google estimated at least 5 exabytes accessible on the Internet (so even the wrong estimate is wrong). I'd imagine a huge percentage of that is compressed video, audio, and images. So, basically 5 exabytes vs 10 petabytes - it's off by 3 orders of magnitude.
Re:Sensationalistically inaccurate article... (Score:2, Informative)
Indeed ... while it's an impressive number, we already have experiments that generate more "raw data" per day than that: "CERN experiments generating one petabyte of data every second" http://www.v3.co.uk/v3-uk/news/2081263/cern-experiments-generating-petabyte That's 84EB per day. But "all" of it is crap, and they eventually store only about 25PB per year.