How Long Do You Want Digital Media To Last? 398
spamfiltertest writes "CNET asks 'Would you like your digital-storage media to last 20 years, 25 years, 30 years, 35 years or 40 years?' If you're an organization or government agency, the U.S. government and an optical-disc industry group would like you to answer that question in a quick survey. I would think that we would like our data to last forever, but maybe it's just me."
I'll take the survey in a bit, but... (Score:5, Insightful)
If you're an organization or government agency, the U.S. government and an optical-disc industry group would like you to answer that question in a quick survey.
I work in the records department of a two year tech college. We use document imaging hardware and software to store student files on WORM optical media permanently and then we destroy the physical paper files over time.
We expect that our digital media will far outlast what we have on other permanent storage mediums, such as microfiche, which go back to 1972. If the "antiquated" microfiche can hold up that long why not our records stored on the digital media?
We realize that no storage method is 100% foolproof (i.e. you can misfile microfiche, lose physical files, misplace pages, etc) but we have put a lot of faith into the setup we currently have. If time has a negative effect on both the originals and backups we could find ourselves reverting to tried and true methods used in years past.
It's mildly humorous to me that long term data integrity (i.e. "forever") is never mentioned when companies present you with all the benefits of a digital setup. The benefits of the system are great (such as easy access to student information at various sites without any reproduction necessary, security features, etc) but will our microfiche outlast our digital media? I may never know but currently, based on recent discussions about the degradation of digital media over time, it appears that it may.
I feel sorry for the poor bastards that would have to go back to storing and reproducing everything to and from microfiche if and when we find out that digital media might not have the necessary longevity we require.
Make it.. (Score:5, Insightful)
forever, but... (Score:5, Insightful)
sum.zero
does it matter? (Score:5, Insightful)
As long as needed. (Score:2, Insightful)
Data != Media (Score:5, Insightful)
If you have media that you know won't last over 30 years, just copy it onto new media at the 20-25 year point. In most cases, that's not that big of a deal. Besides, by the time that 20-25 year mark rolls around, it's very likely that you'd want to convert to a faster "online" media anyway, like holographic storage.
"Forever" would be nice, but... (Score:5, Insightful)
So if you want stuff to last forever, each generation of people needs to convert the old stuff into a new format. But if you are only doing this once a generation, it's not that big of a deal. You could even make it a family tradition, the passing of the old to the new. Assuming of course that you actually care about keeping something 'forever'.
Re:I'll take the survey in a bit, but... (Score:5, Insightful)
The trend is not for newer storage methods to outlast older ones by any stretch of the imagination.
Stone inscriptions, stored reasonably well, will last quite a long time. Books printed with appropriate inks and stored well will also last ages. Comparing to those, "antiquated" media like microfiche will be useless much earlier.
From what I recall, we use newer media forms not because they last longer but because they're more convenient. You can store information much more densely on a DVD than you can on microfiche, which is in turn a more dense storage form than paper, which was a big improvement over marble and clay tablets.
If you really want longevity you should take your microfiche and cut the words into sheets of gold.
Re:Why depend on physical media (Score:5, Insightful)
The whole point of storing data on WORM media is to prove that the data remained unaltered during storage.
You want to be able to have an audit trail that shows any modifications (timestamps included) to the records. You also want to make sure that images that were stored were unaltered ("photoshopped"). You want to make sure that an exact copy of the information was stored and remains exact for the life of the media.
If it's not stored on write once media then that can't be guaranteed.
Re:does it matter? (Score:5, Insightful)
I think we'll see CD media be readable by the consumer for at least ten or fifteen years. The consumer will probably be able to get a CD/DVD reader if he so desires for ten or fifteen years after that -- after all CD and DVD are popular formats, unlike 8 track which was never very successful.
After that, I'm sure there will be companies that will be able to read your old optical media into the quantum dust specks or whatever they'll be using in fifty years. If your CD-Rs last that long -- which they probably won't unless you are very careful about storing them.
Re:Make it.. (Score:4, Insightful)
if you want you could probably etch your data on a block of gold, but what would that cost?
Re:Make it.. (Score:3, Insightful)
Re:Longer than the copyright protection (Score:4, Insightful)
I want my digital pictures to last forever (Score:3, Insightful)
Now I have a nice digital camera(Canon Digital Rebel) that was expensive, but I got it for a good reason. I am about to get married and do the whole family thing. I hope someday that a great-grand kids over maybe even a further down the line will be able to look at all the pictures I will take and maybe understand a little better where they came from, what the world was like, and how pretty there great grandma was:)
Re:I'll take the survey in a bit, but... (Score:4, Insightful)
The difference is that with books, all you will need to read them in 40 years is your mental ability to read, your natural vision, and maybe a set of bifocals.
In 40 years try to find a way to read your DVD full of MSAccess95 DBs, Word 95 docs, etc. Heck I still have a shoebox full of cassette tapes that (at least used to) have Commodore VIC20 software on them. I've got an 8" floppy-disk that we use as a frisby in the office. None of those have anywhere near the longevity of a book, due to technilogical change, totally seperate from any media degredation. From TFA:
(One should consider the issues of digital obsolescence and migration - Is 100 years (or 'forever') really practical for typical long-term digital storage strategies? While you may need to preserve data for a particular length of time, is it really necessary to preserve that data on any particular technology or can it be migrated to newer technologies?).
The difference between good policy and reality (Score:5, Insightful)
Professional archivists tend to recommend that data be turned over onto new media every 5 years regardless of how well it's weathering the years.
But the truth is that, paradoxically, the most critical data tends to be the least likely to be refreshed, because access to it is typically quite limited.
Our own department of defense doesn't know where it stashed all of it's nuclear materials over the years. Why? because they recorded it on a magnetic tape, put the tape in a vault, and had someone stand in front of the vault with a gun for 40 years, and now the tape has turned to goo, and in other cases the tape seems readable but there is no technology available to read it.
We should always strive for and recommend rigorous archival policies, but we should also strive for media that can possibly withstand the ages should some knucklehead put it in a concrete box or just forget about it completely for a few decades instead.
Re:forever, but... (Score:3, Insightful)
Provided:
Then with a long enough key, it is possible to prove that the data is secure for, oooh... a couple of billion years.
Note that these points apply to most forms of encryption in common use today. Every bit you add to they key length doubles the number of potentially-correct keys, so 128-bit encryption is twice as hard to break as 127-bit encryption.
Losing that set of keys effectively means your data is gone for good. Whether or not quantum computing would solve the issue I really don't know.
Re:I'll take the survey in a bit, but... (Score:2, Insightful)
Or Bochs or similar. I admit its a little silly, but as long as what you're working on is completely open, i.e. fully understood specs down to the hardware, its completely virtualizable, and you can just package your data up along with all the open source software that reads and runs it, and know that it will be available later
I will be fully amazed if the chain of necessary virtualizations to recreate an x86 computing environment, implemented in an open source way, capable of running linux ever breaks.
Obviously this is unprovable at this point, but I think its a safe bet, and I've taken it - its not just conjecture - I've got massive amounts of data, and I use optical media as coasters. Mine lives on multiple raid5 arrays, monitored closely, but online.
Re:I have 1.2MB floppies a good as day one and.. (Score:3, Insightful)
Re:I'll take the survey in a bit, but... (Score:3, Insightful)
Unlikely. CDs have been around for almost 25 years, and all proposed next-gen drives (Blue-Ray and HD-DVD) will have CD and DVD compatibility.
Moreover, people have large collections of DVD movies, which alone will ensure the availability of DVD drives for years to come.
VHS is 29 years old, and you can still buy players. Heck, even Laserdisc and Betamax players are pretty easy to find if you look on eBay.
Re:Digital media is easily migrated... (Score:4, Insightful)
The files I wrote in the late '70s are still around today, and completely readable and editable.
Whatever editor I used (I used edlin for a very short time, and then WordPerfect and MS Word - as well as several no-name apps - on DOS machines - later I found vi and emacs under Unix - and have dabbled in OpenOffice and Abiword on my Linux systems) I made sure it had the option of saving the files as plain text - or I quickly stopped using it.
Nowadays I am using XML for anything significant (that I think I may want to publish - on the web or in print) - and plain text for everything else (and XML is really plain text from a software standpoint).
I don't have any software incompatibilities because I don't use proprietary formats to begin with.
Everyone doesn't think like that, however I am trying to educate as many as possible. Some yahoo sent me a Visio drawing the other day; I sent him a message saying, "save it as jpeg or png so I can read it". He did, and I was happy (not to mention I could easily incorporate his drawing in my own documentation/notes or translate it to some other format if needed).
This happens all the time, someone sends me a Microsoft Project file, or some other format that I do not have software for. I force them to change it to an open format - and after awhile they learn (at least to send me data that fits my open model - or that I can translate to something open). When their tools are dead and their files are useless, I will still be able to reference information that happened in the past.
So, your argument about software is only valid if you use proprietary file formats (only readable by one software application). I do not - and so your argument is not valid for me.
No one should use proprietary file formats for this reason - proprietary file formats hinder the migration of data from one technology to another (hardware or software).
Re:What about Decentralized Replication? (Score:3, Insightful)
1) The data over time becomes corrupted. This can be from ordinary memory copy errors (a stray cosmic ray turns a 1 into a 0 or the other way around), or when you send a packet over the network somehow the checksum works out even with corrupted data (it does happen quite often... especially over many generations of data). It happens, so get used to it, and over thousands of years it will be a huge issue. I've found that bit rot over even 10-15 years is incredibly huge for most magnetic media, and optical media, while slightly better than magnetic media, still has some serious problems over time. Electronic memory (RAM) is even worse.
2) P2P data stores are based on popularity. Data that is frequently requested will always be available. The problem is with the data that may only have occasional usefulness, but when it is needed it is very valuable. This is BTW a problem of the ages as well, as even dead-tree librarians also struggle with this same issue, where you have to discard genuine garbage from time to time, and have to decide if it truly is garbage or something that has long term value. The difference with a dead-tree library and a P2P system is that this cycle is 5 to 10 years for a dead tree library but only on the order of days or hours for a P2P repository, and stuff gets discarded much more quickly.
3) Trusted sources of data are hard to identify. This is an issue even larger for P2P systems. The point of a decentralized P2P system is that taking down any one node won't kill the network or even lose the data (hopefully). The problem here is that with all nodes being (supposedly) equal you can't tell real data from forged and/or modified data (avenues for censorship of all kinds and forms). Just because you have 10 copies from 10 sources that says one piece of data is a certain way doesn't mean that the one lone server that says differently is wrong. What is the criteria to show which data packet should be ignored? Again this is a dead-tree library issue as well, but there you have publisher reputations and "original manuscripts" to compare against that are not available in a P2P environment.
While a neat idea, there is quite a bit more work to be done addressing these and other problems with P2P networks. There are valid uses for the technology, and some of these issues are being dealt with in various degrees, but you can't ignore the fundimental problems with the technology and information storage issues in general.