Stored Data to Exceed 1.8 Zettabytes by 2011

jcatcw writes "By 2011, there will be 1.8 zettabytes of electronic data stored in 20 quadrillion files, packets or other containers because of, among other things, the massive growth rate of social networks, and digital equipment such as cameras, cell phones and televisions, according to a new study by IDC. Data is growing by a factor of 10 every five years. According to John Gantz, IDC's lead analyst, "at some point in the life of every file, or bit or packet, 85% of that information somewhere goes through a corporate computer, website, network or asset," meaning any given corporation becomes responsible for protecting large amounts of data that it and its customers may not have created. The study, which coincided with the launch of a " digital footprint" calculator, also found that as the world changes over to digital televisions, analog sets and obsolete set-top boxes and DVDs "will be heaped on the waste piles, which will double by 2011.""

Riiight

[InvisblePinkUnicorn]

"as the world changes over to digital televisions, analog sets and obsolete set-top boxes and DVDs"

That's what I plan on doing. I'm going to throw out all my DVDs and buy the Blu-Ray equivalent.

Or maybe I'll just keep the DVDs (and the player) and buy whatever cable adapters I need to get them working on these newfangled devices.

Well yes...

[theM_xl]

85% of that information somewhere goes through a corporate computer, website, network or asset

That's all? I mean, a good deal will be created by corporations in the first place, all the major bits of internet infrastructure belong to one corporation (for-profit or not) or another, the post office is a corporation... 85% seems low, actually.

Re:Riiight

[Tony Hoyle]

Get a decent TV. There's a massive difference between DVD and Bluray.

DRM? Who cares. I'm not planning on copying 20gb+ disks.

Re:Well yes...

[Dan East]

I don't know about that. Imagine all of the digital pictures taken that never travel outside the home user's computer, memory card or CDs. Even more important, consider the amount of digital video data generated by home users with their camcorders. A single 60 minute Mini-DV tape is in the neighborhood of 15 GB. That's one single tape, and my family alone has dozens of them just from a single year. Even if those videos are uploaded to the internet, they must first be converted to some other format that has a vastly lower bitrate. So the original gigabytes of data still never touches corporate infrastructure - only the small, crappy quality encodings that end up on YouTube.
They might also be counting swap files and hibernate files. In the case of hibernate files, a computer with 2 GB RAM generates 2 GB of data every time it hibernates.

Data figures are misleading

[Bombula]

The interesting thing here is the part about data being relayed through third parties and the issues involved. As for the data figures themselves, those are pretty misleading because data does not equal useful information. There is far less useful information in an MS Word file than 100Kb or whatever, for example, so these zetabyte figures bandied about aren't terribly meaningful other than to draw attention to the infrastructure needed to support digital data relaying. To see my point, turn things upside down: there is vastly more data stored on an LP record or celluloid film than on a CD or digital photograph. But is that data useful information? Only a few audiophiles and filmophiles would argue that there is.

Yes, there is a lot of data in the world. But is there really that much more information out there? A zillion copies of the same song just means more data, not more information.

Re:Which definition of a zetabyte?

[sapphire wyvern]

At the risk of being modded down, isn't that distinction the whole point of the IEC's "zebibyte" proposal?

Anyway, most measurements of mass storage (bandwidth quotas, hard disk capacity etc) seem to measured in actual megabytes (MB), gigabytes (GB) etc, as opposed to binary megabytes (MiB), binary gigabytes (GiB) and so on. Binary byte prefixes only seem to be used for RAM and flash these days, presumably because of the convenient manufacturing realities involved - and I really wish that manufacturers of those products would get with the program and label their products with unambiguous units.

So I assume the estimate means 10^15 bytes.

Re:Which definition of a zetabyte?

[TheRaven64]

In theory, yes. In practice, the whole Zebibyte thing is complete nonsense. Everyone other than hard drive manufacturers has been using the SI prefixes to refer to power of two quantities when referring to binary data for 40 years. Attempting to redefine them retroactively just causes confusion. If I see something that says KB, and don't know when it was written, I have no idea if it pre or post-dates the KiB nonsense and so I have no idea if it refers to 1024 or 1000 bytes.

Re:Which definition of a zetabyte?

[xaxa]

So you're better off if someone does use the proper prefix then. Without it, KB could mean either. With it, at least you know what kiB means, so you're definitely right some of the time.

Re:Riiight

[Aenoxi]

Please mod parent up. If I had a nickel for every person who spouted that same upscaled DVD tripe, then, then, then I'd have enough to buy a Blu Ray disk ;)

There is a world of difference between 1080p and DVD quality - but you'll never see it if your TV can't natively display 1080p (or at least 720) or you use a composite video interconnect rather than HDMI/DVI or component (yes, I know, but you'd be surprised how many people still do...)

Whilst I can imagine that a true 1080p picture might look similar to upscaled DVD on a small screen (which necessarily has very small dot pitch), the difference becomes clear as you scale up the screen beyond 30 inches or so (and bleeding obvious once you get beyond 42"). Interpolation and post-processing can only get you so far. Notwithstanding CSI, even high-end upscaling cannot create genuine detail that didn't exist in the original image - and the more post-processing you do, the more artifacts you are going to see.

I've been running a Pioneer BR player via HDMI to a 1080p 60" plasma for 6 months and whilst upscaled DVD is nice, it can't hold a candle to the 1080 BR picture. Double blind test anyone on a similar system and there's no way you'd get anything but a 100% success rate of identifying HD BR vs upscaled DVD.

Re:Riiight

[meringuoid]

DRM? Who cares. I'm not planning on copying 20gb+ disks.

I would have said that about DVDs not so long ago. Disk space and bandwidth become cheaper with time.

And besides copying, a DRM crack allows me to play discs on the operating system of my choice, to extract small parts of the feature for purposes of review, criticism or parody, and to bypass any annoying previews, trailers, propaganda, threats, or other junk that the studio may have seen fit to prepend to the show.

Re:Which definition of a zetabyte?

[Waffle Iron]

Everyone other than hard drive manufacturers has been using the SI prefixes to refer to power of two quantities when referring to binary data for 40 years. Attempting to redefine them retroactively just causes confusion.

No, the confusion is cause by using a pseudo-binary based number system in a world where almost everything else is decimal.

Quick question: You have a 2000 MiB video file and a 2470 MiB video file. Will they both fit on a 4.37 GiB DVD? Now you need your calculator.

It's much easier to figure out if a 2097 MB and a 2590 MB file fit on a 4.7 GB disk. You can do that in your head.

I've been burned numerous times by programs ambiguously reporting sizes in KiB and MiB causing me to run out of space on something that I'm trying to fill. All storage sizes should always be reported in decimal numbers. If RAM manufacturers want to keep using powers of two due to the implementation detail of how their chips are constructed, they should *always* use KiB, MiB and GiB.

Re:Riiight

[Hatta]

I don't even care enough about high fidelity imagery to wear my glasses day to day. The resolution of a normal TV is plenty for me.

High fidelity audio however is an entirely different story.

4 Gigs.

[Warll]

So I used their digital foot print calc, it told me mine was 4.5 gigabytes. A little on the low side I'd say, I have 1.1 TB of HDD sitting right next to me.

Re:Which definition of a zetabyte?

[Waffle Iron]

For everything else, that is, using a computer, it's back to binary.

It is not. RAM is the only quantity in computers commonly measured in binary. Hard drives have always been in decimal. Floppies have always been in an even more stupid system where "MB" == 1000*1024. Clock speeds have always been decimal.

Going farther, measuring IO or network performance, to cite two trivial examples, or understanding any of those subjects in general, you're binary to binary.

You appear to have been bambooozled yourself by the confusion caused by this issue. I/O speed of buses is always decimal because it derives from MHz and GHz, which are decimal. Network bandwidth is more often measured in decimal megabits, not binary.

You seem to think that just because one user app, Windows Explorer, confusingly shows binary based quantities, then everything else in the computer is or ought to be measured that way as well. You're incorrect.

I don't see why learning powers of two, and then extending that (for the "power users") to base 16, is unreasonable.

If you were advocating that people learn and work in pure hexadecimal, you might have a point. However, these units aren't a consistent radix. They're a strange mishmash of binary and decimal based on the accident that 2**10 is somewhere close to 10**3. They have completely different math for each of KiB, MiB, GiB, etc. You're telling people that they need to work with four or more distinct new number systems, and be prepared to convert between any and all of them, depending on approximately how much data they're working with. That's just stupid.