How To Verify CD-R Data Retention Over Time? 303
Peter (Professor) Fo writes "I've recently had two CD-Rs reported to me as faulty which are just 3 years old. This is worrying — I suspect the failure rate for this batch could be 10%. When researching CD longevity there is old and unreliable information; pious 'how to cosset your discs so they last 100 years' blurb; and endless discussions of what sort of dye to use, don't use cheap media, burn slower (or don't), but not much by way of hard facts besides there's a lot of data loss going on.
Does anyone know of a generic utility (win or *nix would suit me) that can map sector readability/error rates of CDs? I'd like to measure decay over time in my environment with my media and my other variables; and I expect others would too."
CDCheck (Score:2, Interesting)
http://www.kvipu.com/CDCheck/
Create a CRC file for the CD. Saved me more than once.
Re:It's ok... (Score:5, Interesting)
RAID5 for CDs? Is there anything where I can burn 3 CDs with a 'set' of data. When I want to restore my data I just put in each disk sequentially and then it does some RAID5 magic and spits out my data?
Be a cool project, IMHO.
Tape (Score:3, Interesting)
Backups? Use tape.
Optical media is inherently shitty.
If you want to get the best out of it:
Buy good media.
Burn at a slower speed.
Verify the data after burning it.
Store it well. A hard case, and a cool, dry location away from the sun are all you really need.
If you want to test the quality of a disc, go ahead and use any of the tools recommended here.
If you want to harden your discs, go ahead and use any of the CRC tools recommended here.
But really, you shouldn't be using optical media as anything other than a cheap delivery medium. If you need to send stuff to people and you need them to have a copy of it indefinitely, tell them to make a damned copy of it, or give them 2 copies, or keep an ISO and send them a copy when theirs fails.
Re:par2 (Score:5, Interesting)
If it's really important, I always verify the integrity no matter what
I do that even if it's not important. I have a script which creates an md5 checksum file for a directory tree and adds it to the directory, and I always run it before burning a CD or DVD. Once burned, I verify the checksum on two different computers.
There have been a few times that the computer that burned the disc successfully verified a new disc, but a different one didn't. When that has happened, I trashed that disc and made a new one.
Sometimes I wonder if a lot of the reports of "deteriorating discs" are actually cases where someone burned a coaster in the first place, and just never happened to try to read or verify the data until years later.
Re:Professional Advice (Score:2, Interesting)
SMART over time (Score:3, Interesting)
The request asks no such thing, he simply wants to measure decay rate.
The method of dd+diff can tell only whether the disc has decayed or has not decayed: 1 bit of information. Something that can read C1/C2 error rates, like the program Wanker mentioned [slashdot.org], gives much more information that can be used to give a better idea of how much decay has happened before it becomes unreadable. Plotting this over time gives (ta-da) the rate of decay.
Re:I highly recommend using Archival Grade Media (Score:5, Interesting)
Re:I highly recommend using Archival Grade Media (Score:2, Interesting)
Re:I highly recommend using Archival Grade Media (Score:4, Interesting)
Exactly. I have 60 archival grade CD-R
s from over 10 years ago that are still readable Yes I have tried them, it is a part of our backup integrity testing every year.
Storage and handling is also very important with them as well.
DVD-RAM too (Score:5, Interesting)
I use dvd-ram [wikipedia.org] to archive important files. Designed for archival type storage, the slower media has a 30 year designed life, the faster media has something like 5 year. Add in the builtin ecc and cheap cost, it is a good way to save my source code and photos.
Re:It's ok... (Score:1, Interesting)
RAID5 for CDs? Is there anything where I can burn 3 CDs with a 'set' of data. When I want to restore my data I just put in each disk sequentially and then it does some RAID5 magic and spits out my data?
Be a cool project, IMHO.
3 CDRW drives + direct writing and RAID options turned on in kernel + mdadm to setup the desired RAID level
Just off the top of my head, but it might work...
--
k
Re:I highly recommend using Archival Grade Media (Score:2, Interesting)
I have always thought CD media to be the BEST way to backup/archive data, because it doesnt have magnetic retention that can be lost over time. Out of any media, what media is the best way to archive data? And for the dumb Question:
How do CD's actually go bad? They are physically carved
Re:I highly recommend using Archival Grade Media (Score:3, Interesting)
Re:dvdisaster (Score:3, Interesting)
The storage footprint of data tends to be inversely proportional to its importance.
People lose sight of this fact, and because they don't see a good way to backup the 500 gigabytes of data they have, they somehow fail to backup the 20 pages of documents that they need to protect their assets or limit their liabilities.
Don't underestimate the value of a document printed in archival ink, stored in several fireproof locations. Definitely don't underestimate the archival value of, and potential longevity of, hand-written journals.
Re:It's ok... (Score:4, Interesting)
This has worked well for me. I had a situation where a RAID 5 lost two drives and my backup had some corruption in a 100 gig video project. Pulled out my 3 year old DVD's and recovered the data fine (and yes there were one or two of chunks that had problems but recovered from the redundancy).
Just my method and 2 cents.
--
So who is hotter? Ali or Ali's Sister?
Re:I highly recommend using Archival Grade Media (Score:2, Interesting)
I have boxes of TRS-80 diskettes, Model I 40 track disks, from as far back as 1978, that have been stored in the most careless manner in very nasty conditions (were even, literally, in a barn for a few years) and they can all still be read. It's unfortunate that none of the information on them is important, which superstition tells me may have something to do with their success rate. Yesterday I burned an Ubuntu 8.10 CD onto a store-brand CDR, and it failed in the second machine I used it in. To be fair, it has a radial scratch that got on it somewhere, and, I burn these things intending to throw them away :-)
CD, DVD Data (Score:3, Interesting)
Thus unreadability means gross damage, dye decay, scratches or thick dirt. Physical scratches/dirt can often be recovered with very fine metal polish and wash. DVDs are _much_ more iffy. A few readers let you read all the 2352 mode 2 data bits of a data CD but most dont and you cannot detect 14 2 8 bit correction which is the first indication of the dye degrading. If you store the CDs in the dark, in scratch resistant envelopes in a strong box you have a good chance of 20+ years. I wrote the one of the first Philips writer drivers and have CDs that old.
Otherwise you are down to replication and data washing, but dont throw the washed CDs out!
Re:I highly recommend using Archival Grade Media (Score:1, Interesting)
Wish I could mod you to +6, sir. As data retention and migration has factored in my career (and hobby) for many years, it's great to see such concise accuracy.
I will vouch that CDs are more fragile than DVDs. I have lost entire Kodak Gold CDs from a single flake off the label, near the disc center (the TOC). Meanwhile last month I experimented with an xacto knife and a DVD-R movie. I could not make a cut deep enough on the label side to disrupt the playback.
I have tried most 24 karat gold media available on the market, and gold media is the only thing that I will consider archival. However not all are created equal, and the Kodak gold media you can buy in 2008 is not the same reknowned manufacturing as in 2000. The advantage now is that DVD sizes are widely available, but still notably expensive for a hobbyist. MAM is still the cream of the crop.