USB Flash Drives for Backup/Long-Term Storage? 81
cyberdigm is curious about this issue: "I am writing two books and have just recently gone through the trauma of having my hard drive flake out (physical damage to several sectors). Fortunately, while the OS instance was trashed, the file system is still intact, so I have been able to recover my files.Given that, I am now much more aware of the needed to regularly back up my files. I'd be interested in any opinions about the suitability of USB flash drives to help me solve this problem. The idea would be to store copies of all my files on a USB drive and back them up every day. I like that USB drives are generally fairly cheap. My concern is the long-term wisdom of this approach. Are there (practical) rewrite limits for USB flash drives? Is there a chance that the data would degrade on the drive over time? Other alternatives I am considering include external/USB hard drives. Of course, an overarching concern is that I'd rather not spend a lot of money."
Practical Concerns (Score:5, Interesting)
#1: Rewrite limits. Currently, flash memory has about a 1 million rewrite limit theoretically. In practice, though, I had a CompactFlash card (no real different technology from the USB drives other than interface to system) fail after about a year of daily backups.
#2: Time-to-destruction- I once left pictures of my honeymoon for nearly 6 months in my digital camera, also using flash memory. After 6 months, the files had a 50% corruption rate. So I wouldn't consider this a very long term storage solution- at least not without refresh.
Asside from those concerns, it's a very cool idea- especially if you kept the backup software on the key and increased your potential by using say, 7 keys (one for each day of the week) and kept the backups off site.
Re:Practical Concerns (Score:3, Insightful)
Also talked to them about an iPod - they have a business so they could have depreciated as a backup device :)
Re:Practical Concerns (Score:2)
Good call.
Re:Practical Concerns (Score:2)
From long experience... (Score:2, Informative)
The only practical way to maintain integrity, given that data was stored offline (in fireproof safes) on mag tapes was to use a grandfather/father/son system of backups, with special tapes reserved for end-of-month or end-of-year processes.
I know we've come a long way since the days of batch-processing,
Re:From long experience... (Score:2, Insightful)
Yes it does. Merriam-Webster agrees with my usage:
One entry found for pedantic.
Main Entry: pedantic
Pronunciation: pi-'dan-tik
Function: adjective
1 : of, relating to, or being a pedant
2 : narrowly, stodgily, and often ostentatiously learned
3 : UNIMAGINATIVE, PEDESTRIAN
Re:Practical Concerns (Score:2)
An iPod is a terrible backup device. It has even more points of failure than a regular hard drive in an enclosure. If the proprietary cable that connects it to the computer breaks, you are dead in the water until you acquire another one.
Furthermore, a common solution to iPod troubleshooting techniques is to run the restore through the iPod Updater utility. This kills all data on it. The iPod sh
Re:Practical Concerns (Score:2)
Re:Practical Concerns (Score:4, Interesting)
He also won't, for the life of him, trust hard drives, zip disks, CD-Rs, dedicated network storagem or anything else to store his resume, which he updates and tweaks nightly. Not really in need of a job, being an international energy lawyer (i.e., oil man, and in *this* administration of all times), it's more of a hobby.
Luckily, there's something about the size and heft of the disk for him that makes it oh-so-magical, so I got him a DynaMO [dynamoo.com] drive, which is a magneto-optical drive. I won't go into details (someone feel free to provide), but because of the way the media is written to the disk (not to mention the casing), they can take a beating, and much more than flash or other 'sensitive' media where scratches, low heat, or simply Murphy's Law can kill your data.
Pricey (~$200-250), but not considering you're writing books. Use some of your advance money and invest.
Re:Practical Concerns (Score:5, Informative)
magneto optical discs get anywhere from 128mb to 5.2gb that i've seen, and they come in three varieties: minidisc, which is primarily for audio, but a few data ones are being sold, the old version holds ~1/5 of a cd, so ~130mb, the newer version (uses multiple layers) holds 1gb, and i don't know if they have a data version or not. 3.5" mo discs come in 128mb-1.3gb that i've seen. slightly older drives accept 640mb discs while the new ones take 1.3gb discs. this value may have increased since i last looked for a mo drive. 5.25" mo discs come in sizes up to 5.2gb so far as i've seen; this value might be bigger now.
Re:Practical Concerns (Score:2)
Here's how I do it. Certain critical small configuration files always get backed up to floppy. They also get backed up to multiple drives on multiple machines along with all of the larger files. I don't trust burners or flash except for sneaker net ops.
Re:Practical Concerns (Score:1)
He also won't, for the life of him, trust hard drives, zip disks, CD-Rs, dedica...
two words: print it
I don't trust any storage media 100%. If something is really important to me, I print a hard copy. It's not that I don't burn backups to disk, or even ftp stuff to other machines, (I do) but nothing beats the trust factor I have for paper and ink.
My filing cabinet hasn't had a single crash or virus in all the years I've owned it...
Re:Practical Concerns (Score:2)
I believe your story, I don't think flash drives are suitable for long term storage...
However... to add my own $0.02... about 18 months ago I lost a 64MB USB key... couldn't find it for the life of me... it had several documents that I really missed and was mad about losing...
I decided to sell my car... found it jammed against the power-seat rail... the external case was cracked from it being ground against th
Software RAID (Score:2, Informative)
Re:Software RAID (Score:3, Interesting)
That's all fine and good... Until you have multiple drives in the array fail. I used to think it was such a low possibility that it just wasn't worth worrying about.
I've lost both drives in a mirror set within 15 minutes of each other three times now.
Re:Software RAID (Score:1, Insightful)
Dude, get a fan. That is normally the problem. Also, dont forget to send the drives back to the manufacturer... the will let you know what happened. ( normally a 3 year warranty). As a bonus you will get new drives.
btw- When did you get out of the steel buisness?
-- john galt
Re:Software RAID (Score:2)
The problem wasn't heat, it was flaky Maxtor drives from the same lot. Two of the replacements I got from Maxtor also failed in the same manner.
Well, you know... I kinda got down in the dumps for a while, with a friend screwin
Re:Software RAID (Score:3, Informative)
Uhm, I think the parent poster did in fact cover that. Let me paste it back for you:
Software RAID on a bunch of different hard drives (preferably SCSI, but you can also use IDE/ATAPI/UDMA/USB) and automatic off-site (e.g. a remote ftp or scp) backup cron job should do the job.
See bold text if you still don't get it . . .
Re:Software RAID (Score:2)
Are you sure it's not a flaky power supply killing them off? It's an often neglected cause of hard drive failures.
Re:Software RAID (Score:2)
Yep, I'm sure. I explained it completely in another response in this thread, but I got bit by the fact that 1) the drives were from Maxtor; and 2) they were all from the same manufacturing lot.
Re:Software RAID (Score:1)
Re:Software RAID (Score:2)
Yep, and that's what nailed me at that particular point in time. I had purchased a box of 24 drives from the store to use, not even thinking about batch-related failures.
Plus, come to find out, Maxtor seems to be a little looser on their "good" qualifications for the platters, so that was another negative.
It's one of those "live and learn" situations. I do try to keep with the same manufacturer so that the performance specs are the same, but I carefully check the lot numbers before making purchases like t
Re:Software RAID (Score:2)
relatively cheap or reliable. (Score:3, Insightful)
why do people not realize that costs for the physical item are only part of a price tag?
NAND based...... :-( (Score:3, Informative)
If you decide to use these flash drives as a backup medium, you should definitely use some sort of encoding that allows for bit-corrections. Possibly some sort of Forward Error Correction. Or use a RAID parity/striping method.
Re:NAND based...... :-( (Score:1)
Couple of easier, low-tech solutions (Score:4, Insightful)
Second, since we *are* just talking about text, it might be worth your while to use email. I've got a community network email account that I do this with - email them important info, they store up to 15 megs worth of data, where it will pretty much sit forever.
Third, little more expensive: last I checked, a 512MB USB disk drive costs about $70 CAD; you can buy a brand new low GB HD for about that much. Just run two HDs on your system, sync the data every night, and there you go. HDs don't tend to flake out as often as you think, and this way if one goes, you've still got an onsite copy. Then just buy another $70 HD and keep going.
Re:Couple of easier, low-tech solutions (Score:2)
Re:Couple of easier, low-tech solutions (Score:3, Insightful)
Forever? (Score:2)
email them important info, they store up to 15 megs worth of data, where it will pretty much sit forever.
Have you tested this version of foreverness?
Re:Forever? (Score:2)
Those fuckers will destroy your entire inbox faster than you can say 'a 32 day vacation touring Eastern Europe.'
Not that I actually lost anything important, mind you
well (Score:1)
1) I've had my hotmail account for many years, and since its my primary accoutn I couldnt switch if I wanted to, I need it - too many people have that address. The only downside I have with hotmail is they do
Re:Couple of easier, low-tech solutions (Score:1)
However I do know that most PC power supplies are despicable as of late, with a suicide rate far exceeding that of HDDs (except for certain IBM models :)). And if your power supply goes, you have a pretty good chance that your data
then you're a fool (Score:2)
you're an anonymous idiot (Score:2)
this is not the same as backup, you fool. Disks come and go: your array remains. What's stored ON THAT ARRAY is your problem, and you maintain it despite the stupidity of your users via OFFSITE BACKUPS. THE END.
Re:Couple of easier, low-tech solutions (Score:1)
So I guess that would be an auto-script that copies files to the key drive every time it's plugged in, and also mails a copy of those files to Gmail. So then your "backup activity" would consist of plugging in your key
Re:Couple of easier, low-tech solutions (Score:1)
GmailFS (Score:3, Informative)
Win32 Version [viksoe.dk]
*nix version [jones.name]
Re:GmailFS (Score:2)
Just send to your gmail address a text attachment containing each night's changes to the work and there it is - a running back up of your book, accessible from anywhere.
You might also look into encrypting it. Isn't Google going to open the email up to its search engine after some period of time?
Re:GmailFS (Score:2)
What!? They wouldn't do that. Not because they're "not evil" or anything, but because 1) random people's e-mails will dilute search quality and 2) they will get sued out the window if they try to make these e-mails public, regardless of how you can interpret the user agreement.
Re:GmailFS (Score:2)
Re:GmailFS (Score:2)
Use multiple approaches (Score:5, Insightful)
The USB drive will be fine as one approach--use it daily--but don't leave it at that. It's a convenient medium, but you want it to be able to fail without hurting you much. If you combine it with other media, you can enjoy the convenience without exposing yourself to whatever risks there may be.
Then get yourself Web hosting from some reasonably good quality host and FTP your files to your own website. If you're not very technical, the Web host can tell you how. You don't need to learn how to build a website to FTP files. The more important the data is, and the harder to recreate, the more you need to have it on the website. If you can just get it up there, they will do your backup for you. This assumes that you don't have tens of gigabytes of personal data, which could be a mistaken assumption if you are talking about photography, for example, instead of writing.
From time to time, burn a CD-R of your files (or multiple DVDs if you have gigabytes of important personal stuff), make multiple copies, and stash them in different locations: with your parents, in a self-storage locker, or whatever. That's too much bother to do very often (if you include the offsite stash), but it's a good thing to do occasionally. Just to be sure, check and see if the CDs/DVDs you burn can be read in your parent's, friends, or kids' computers.
Re:Use multiple approaches (Score:2)
Multiple Redundancy (Score:5, Insightful)
While a USB key is certainly portable and convenient, it may also create another problem - easy theft of your work. How easy is it to lose a USB key, or have it stolen. And what happens if the finder claims your work as their own? If you did use a USB key, I would definately not keep it with you, but store it in a safe and secure location.
In that vein of thought, since you are working on a book, do you keep hard copy backups? I know it would be a pain to OCR all those pages back in if you lost everything, but it would be better than starting from scratch. If kept in a fire retardant safe, they would fare much better than digital media would.
Re:Multiple Redundancy (Score:1)
This is a very good point, but I don't think the parent post takes it far enough.
If your livelihood depends on your data, back it up in multiple locations. A USB key is convenient for moving your data around, but it's not designed for backup. First, backup your working directory daily to one or two backup directories on the same drive; this is convenient if you accidently munge your working directory.
Second, backup your working dire
My solution (Score:1)
External harddrives [maxtor.com]
(sorry if the link is bogus)
Network backup (Score:2)
The best option for you would be network backup. Seriously. Either pay a little for a little space on a remote server somewhere, or squeeze a folder out of your friendly publisher network technician / techy friend / college kid / local bum.
USB drives are bigger than you think (Score:2)
Loss (Score:1)
My only worry has been actually losing the drive itself. Then again, I'm doing interm
Gmail (Score:2)
Gmail (Score:2)
Email yourself a daily backup.
Reasonably secure, readily accessible storage, plus fodder for a future book: "How I wrote the Great American Novel - Stages and iterations."
RAID 5 (Score:5, Funny)
Re:RAID 5 (Score:2)
Re:RAID 5 (Score:2)
It worked as you'd expect, the array was as slow as the floppy, the floppy never stopped moving, but it did work.
md is fun when you're bored. I also set up a 4 device RAID 5 on a single drive with 4 partitions to win a bet with a friend who said "you can't do raid 5 on one drive".
Re:RAID 5 (Score:2)
gmail (Score:1, Redundant)
1 gig of space, just send yourself an email with the latest version every day when you're done. You can clean up after yourself if you do run out of space.
(email me at ralf at muhlberger.com if you need/want an invite)
Free (Score:2)
1) Web storage (encrypted to your heart's content)
2) Floppies/cds (yes, they arent great for long term, but if you make semi-frequient backups, that should be no problem.)
3) Hard copies. Just take the files down to kinkos/office depot/whatever and print the entire work off. (re-entry might be hard, but non-recoverable data corruption would be harder, also sometimes editing by hand is a nice alte
Multiple backups are the way to go. (Score:1)
The Email option is a good standby and can be relatively cheap (some one on this thread offered you a GMAIL invite, there's 1GB of online storage for nothing), also Yahoo offers a 2
What's wrong with CD-R? (Score:1)
My opinion based on work in flash storage industry (Score:4, Informative)
What happens? (Score:4, Informative)
When your house burns down, or floods, and your drives are underwater, including the backup drives?
Here is how I would probably do this.
First, check out RCS. You have one file, you want to keep a record of revisions.
Second, depending on how much I'd want to spend, I'd either back up to CDR once a week (keeping the old backup "off-site", say, on in my vehicle parked on the street, at work, etc) or else buy an online storage space for a few dollars a month.
A quick google search shows 50 MB for $3/month [ibackup.com], which is a lot of plain text. If you are using some funky word processor format and/or images, half a gig is available for $10/month. It even supports rsync!
Durability of USB Flash Drives (Score:2, Interesting)
Re:Durability of USB Flash Drives (Score:1)
Re:Durability of USB Flash Drives (Score:1)
I'm just impressed that a /.'er actually does laundry!
Reliability (Score:2)
My (expensive) USB flash stick died a few weeks ago, after two years of usage. I could still read it, but it refused to be written. Gladly, my vendor gave me a new USB stick for free, but this is definetly not the media I want to store my important files on.
My backup is an external firewire case with a large ATA harddisk, combined with a script that writes all modifications in the main RAID-1 filesystem to the backup harddisk. Tape drives are too expensive, and tapes wear out rapidly. Burning complete back
Venti (Score:2)
One of the plan9 file servers uses block hashing for it's file store. Thus if you save the same file twice it doesn't use any extra disk space (modulo housekeeping meta data)
thus you could backup 100 windows machines and you would consume the sum of their overlapping data, repeated data not repeatedly stored
in this way plan9's full daily snapshot backup system consumes something approaching the minimum disk space required, one can step through one's file system a day at a time
this would be a useful way t
Failed USB Key (Score:2, Interesting)
This was with one of the Sandisk Mini Cruzer 256MB drives. I replaced it with another Cruzer (newer model), and after another three months it's mostly reliable, however I've had it become "unplugged" on its own a few times recently.