Best Home Backup Strategy Now? 611
jollyreaper writes "Technology moves quickly and what was conventional wisdom last year can be folly this year. But the one thing that's remained constant is hard drives are far too large to backup via conventional means. Tape is expensive and can be unreliable, though it certainly has its proponents. DVDs are just too small. There are prosumer devices like the Drobo, but it's still just a giant box of hard drives, basically RAID. And as we've all had drilled into our heads, 'RAID is not backup.' When last this topic came up on Slashdot, the consensus was that hard drives were the best way to backup hard drives. Backup your internal HDD to an external one, and if your data is really important, have two externals and swap one off-site once a week. Is there any better advice these days?"
External and Online (Score:5, Informative)
Switching off-site backups every week is an unnecessary hassle. Back up to an external hard drive and an online backup service. Anything more than that is overkill unless you have really important data.
Network Backups (Score:1, Informative)
You should try Mozy - free for 2GB, $5/month for unlimited. I've been using them for a year and its seamless.
https://mozy.com/?code=WAQ9DM
BD-R DL discs aren't too bad (Score:4, Informative)
Almost 50GB per disc and brand name blanks aren't too expensive if you know where to look [smartimports.net]. (Hey Newegg: surely y'all could save us some nuisance if you'd import a shipping container or two of blanks direct from Japan...) Nero Linux supports Blu-ray drives. RAID1 for primary storage with BD-R DL backup, with the backups ideally stored securely off-site should be sufficiently paranoid for most home users though Blu-ray is too new to have real-world long-term integrity statistics.
Remote backup to a rented dedicated server is also a possibility though not terribly practical in America due to certain monopoly carriers (<cough>AT&T</cough>) being too cheap to build FTTH, at least until they run out of duct tape and bailing wire to keep their WWII-era copper plant patched together, and even then.
Differential + hard drive - online (Score:3, Informative)
I like my layered approach.. (Score:5, Informative)
I decided that I have three main "categories of data":
- easily replaceable: This is stuff that is fairly easy to replace.. for instance I have ripped a huge portion of my DVD collection (for my own use). If I lost this data, it would not be a tragedy .. just a pain in the ass.
- hard to replace: This is stuff that does exist "out there".. but would not be easy to replace. This includes old TV shows that you can't buy or if you can are very hard to find.
- irreplaceable: Self explanatory.. this is my documents, code, photos, etc that could not be replaced if lost
I keep everything besides OS files on a file server. Raid 6 (two parity stripes).. this is the first layer..
to me this is adequate to protect "easily replaceable" stuff (which in my case constitutes a huge chunk of file space).
I backup everything in the "hard to replace" and "irreplaceable" categories to a seperate (removable but stays in the system) hard disk (so far 1TB has been enough to hold all this data). I make a
secondary backup to a second removable drive and store this "off site". This secondary backup does not get updated very often.. which is the trade off I guess... but it provides a "last hope" if something
crazy ever happened.. like my house burning down.
Oh.. and backups are encrypted!
Windows Home Server + Jungle Disk (Score:4, Informative)
Re:Network Backups (Score:2, Informative)
Mozy is good (Score:3, Informative)
Read The Fucking Summary (Score:1, Informative)
But the one thing that's remained constant is hard drives are far too large to backup via conventional means.
Maybe it's unconventional to use, I dunno, another hard drive?
From said fucking summary:
"When last this topic came up on Slashdot, the consensus was that hard drives were the best way to backup hard drives. Backup your internal HDD to an external one, and if your data is really important, have two externals and swap one off-site once a week. Is there any better advice these days?"
Re:say what? (Score:5, Informative)
I think the OP's post arose from a misunderstanding of what "RAID is not backup" means.
The adage isn't an admonition not to use hard drives as a means of backing up data. Rather, it is concerned with the fact that any change to your data is committed to each duplicate volume in a RAID, so if you delete an important file, for example, it's just as gone as if you weren't running a RAID.
That's completely different from mirroring your drive onto an external hard drive and putting it on a shelf somewhere. If you delete a file on your live system, you can restore from that backup.
cost (Score:5, Informative)
Once upon a time, the computer you wanted always cost (at least) $5,000.
This trend ended in the late 80's. All of a sudden, package system prices started trending seriously downwards, because due to Moore's law, computer speed started outrunning almost everything you'd want to run on it. Not true for certain specific apps, including graphics and games, but for office use it was perfectly fine.
I remember buying a 200 MB hard drive for $500 and thinking about what a great price it was.
Up until recently hard drives were one of the more expensive components left in a computer package. Now? Most are under $100. That's lower than tape backups used to be at their lowest prices. It's true, right now the best way to back up your hard drive is a second hard drive.
IMO the big question now is where that second hard drive will be. You can stick it in your computer and mirror your main drive in real time easily enough, but that means a virus or software issue will ruin both drives simultaneously. Better to sync them once a week? Perhaps.
Of course, this won't help you if there's a house fire. The fireproof hard drives are still darned expensive. Internet-based remote backup is great, if your broadband can handle it.
Re:Mozy is good (Score:1, Informative)
hmmm. i use mozy on my macbook, but no shell, that i am aware of. it's automatic and free up to 2 GB.
Re:I use... (Score:1, Informative)
Amazon S3 would cost me over $300/month. I don't know where you got 0.17 pricing. but that does not include sending or retreiving the data. Browsing also has costs...
http://aws.amazon.com/s3/#pricing
Re:External and Online (Score:3, Informative)
That's fine if your ISP doesn't have draconian caps. I have over 2TB of stuff (legal, mind you, lets not get a redundant "You must be pirating" theme going). Mostly photos and video content. My ISP caps at 100GB per month. Online backup is not a viable option except for my most important stuff. I use the offsite backup drive method, however I don't have two sets that I swap, I just have one offsite backup that I bring home from work ever other week.
Some of those online backup services offer the option to send in harddrives or tapes to make the start. If you stick with offsite backups, you can leave one big basis backup at work, and only swap the incremental backups. Then two simple 2.5" usb drives are big enough to handle that.
as always, it depends (Score:2, Informative)
Let's define backups.. again... (Score:3, Informative)
Backups are:
- off line (viruses, power surge, sabotage...)
- off site (fires, theft...)
- tested (i've got horrors stories of people that THOUGHT they had backups...)
- multiple (... and of backups that turn bad at the worst possible moment)
So backing up data is a hassle, and can be expensive depending on what you need: a DVD, a BD, an HD... But pretty much the only foolproof solution anyway is to burn your data onto a media you then send away to your parents' or other trusted 3rd party. Once a month is the very minimum.
If you're using HDs, you may want to re-use them after a while, but don't forget to keep some very old ones, for when you realize ages after the facts that one of your files got corrupted.
Levels of importance (Score:5, Informative)
There are three kinds of data:
1. If you lose this data you will go to jail.
2. If you lose this data, your business will be impacted.
3. If you lose this data, you will have less options for entertainment.
#1 tends to be a megabyte or less.
#2 tends to be a few hundred megabytes of documents.
#3 tends to be terabytes.
My company has a PDF of every document that we've touched in the past decade (federal law requires this retention), and our entire business continuity backup fits easily on four LTO-4 tapes, plus a very less-than-full tape that we rotate for offsite storage weekly. We've explored every backup system out there and this is by far the most cost-effective for us.
I don't understand why the OP claims "tape is unreliable", as I have not heard of a single instance of in-service failure of an LTO-4. As for it being expensive, it is, but before we went to tape we were using Firewire800 external drives, much more expensive than tape cartridges, and not as reliable as some people have been led to believe.
USB and FW external drives almost never fail as long as they are powered on. They fail in storage, which seemed pretty weird to me, since they should be able to sit on a warehouse shelf indefinitely. My low-sample unscientific data from experience says otherwise.
Since everybody is going from LTO-2/3 to LTO-4, you should be able to get LTO-3 transports pretty cheaply.
But my first advice is to identify the data in categories #1 and #2, where you might realize that it's a good practice in any case, to store the important stuff with its own priority. This is the hard part. Identifying what's actually important. If you don't do this, no matter what backup system you end up using, you're going to be burying the important stuff in the noise, introducing risk.
The OP also mentioned Drobo. I have a Drobo and I love it, but I must warn you that it's pretty slow, even with really fast drives. Don't expect to be able to copy a terabyte to it in less than 40 or 50 hours, even with firewire 800. This is the problem that drove us to tape, which is much faster than any filesystem we can feed it from.
Back up the Data Files to the Cloud (Score:5, Informative)
Ghost Virtual Machine [g.ho.st] gives 15gigs of Amazon.com data storage and right now if you use the promotion code of "launch" you get 10Gigs more as a bonus for 25Gigs. If you want to give me a referral my id is orion_blastar there, and each person you referred grants you 5Gigs more in a bonus.
Google Docs [google.com] also has document storage but does not give as much as G.ho.st does. The Ghost Virtual Machine can access your Google Docs drive as well.
Here is a review of the top 5 online cloud storage sites [readwriteweb.com] so you can take your pick.
MyBloop [mybloop.com] offers unlimited free storage, but I am not 100% sure of that or their privacy policy.
Lifehacker talks about using your Yahoo Mail account for unlimited storage [lifehacker.com] and also that Google's GMail almost offers the same service as well.
Re:SSD (Score:5, Informative)
This is a bad idea. Other than the ludicrous cost of the SSD, flash drives tend to fail all at once. bam! and all your data is gone. This is also why i do not use a USB key for backups.
On my system everything is dumped on a 2TB mirrored system (simple 2 x 2TB HDDs running debian software encryption + RAID lvm) and periodically backed up to blu ray DLs in duplicate. At $10/disk from japan (see ebay) two verbatims back up 50GB in duplicate for $20.
Typically it takes 2-3 months to generate that amount which means its cost effective. DVDRs (Taiyo Yudens) fill the gap if there is not enough data to justify a bunch of blu rays.
Re:Windows Home Server + Jungle Disk (Score:1, Informative)
Hit by lightning (Score:3, Informative)
Re:SSD (Score:3, Informative)
DVDs are slightly better than CDs. A CD has the physical pits pressed on the top of the disk, then a thin silver backing is placed on top. When that backing gets damaged, goodbye data. A DVD, on the other hand, has the silver reflective part sandwiched in the middle of the disk. Therefore it is more resistant to physical damage.
Safest home backup solution (Score:2, Informative)
Use standard commodity parts for easy replacement/reconstruction.
ie. Consumer grade writable disc's and external hard drives.
Backup Home data to DVD/BR/CD regularly (Monthly)
Backup Bulk data with 3 Hard disks!!!!
1) Use one Live HD
2) Use one to backup Live
3) Use ANOTHER to backup Live again
The Reason?
If your backup is large, You will probably need to erase the target drive.
At this point you are VERY vulnerable.
With your only backup erased,
You had better hope your live drive doesn't fail while backing up again.
Or you may accidentally delete something from the live drive WHILE backing up.
With a 3rd hard drive, At least you can recover from your last backup if your current backup somehow nukes itself.
Dangers....
1) Don't have a live drive and think your safe with another live drive synced, You really want to keep hard drive backups IN
STORAGE. If it's live it can and probably will die. It's just a matter of time or circumstances. This goes for raid too.
2) This is home storage, So 6/12 month hard disk backups should root out any bit rot or hard disk seizures.
Frequent duplication is NOT the answer (Score:3, Informative)
For example you deleted a file last week that you need now. A supplier has sent this month's data with last month's filenames and overwritten last month's data. Your database has 'acquired' corruptions and now you need to go back to find a working or clean version. A duplication strategy just means you have two copies of the bad stuff!
Here is what I do
The history chain has extending time gaps between copies eg 0.9,1,3,7,15,30,90 days. So for a daily backup of a file that changes every day the two most recent copies are always shifted down the chain and every 3 days the second is bumped down to 3rd position which in turn might bump 3 to 4 if 3 is more than a week newer than 4. This is ever so easy to program - I even did it in a DOS batch file.
Let's review what happens if the computer goes bang! - Reload from USB hard drive and flash. Alternatively if data gets corrupted - Trawl through the history on the HD.
Re:SSD (Score:3, Informative)
an 8GB drive (which costs, ooh, about £5 - hardly a "ludicrous" cost)
A 2TB hard drive costs $230. A 16GB flash drive costs $42. That is 200 times the per GB cost. If you are trying to store large volumes of data (which is what was being referred to) SSDs ARE ludicrously expensive. 200 times as expensive as hard drives. Given that an external 2.5" drive DOES fit in my pocket, and costs FAR less per GB, there is little justification for the thumb drive.
Your math only works if you have two full terabytes to fill up. If you only have 16 gigs of stuff, the cost per gig isn't that useful of measurement.
Believe it or not, I agree with your assessment, at lesat in the context of my own needs. I'm just not convinced on the 'one size fits all' argument.
Re:Differential + hard drive - online (Score:3, Informative)
I use Mozy, it's a couple bucks a month, the initial upload took a week or so, but it was all backgrounded and I never even noticed (yes, you can turn your computer off, etc.).
That's the point that most people miss - it's not going to download all your data every time it does a backup, just the changed files. And that's a much lower number than many people believe, and uses much less bandwidth than most anybody would expect, especially when you factor in compression!
Even on busy servers with over 250,000 file transfer operations per day, I see backups done remotely at 1 Mb taking just a few hours using rsync - I can't see Mozy being any hassle at all.
Re:Mozy is good, but they don't encrypt filenames (Score:2, Informative)
Plug: In 2006 I founded https://spideroak.com/ [spideroak.com] specifically to provide a zero-knowledge approach to online backups. We don't know anything about your data, including your file and foldernames. On the servers we just see sequentially numbered data blocks. It's written in Python and C and we've always supported Linux and OS X (and Windows if that's what you're into.) SpiderOak keeps historical versions of your files and deleted files forever (or until you decide to remove them) and will sync folders for you across several computers. Some reviews are http://www.linuxplanet.com/linuxplanet/reviews/6644/1/ [linuxplanet.com] and http://www.maclife.com/article/reviews/online_storage_battle_which_cloud_backup_service_reigns_supreme [maclife.com]
Re:Do we have to bring this up over and over again (Score:2, Informative)
So can anybody say something to rsync-like features in other backup software? Surely there should be loads of commercial ones that do this differential copying thing?
Just googled for it and at least found DeltaCopy, a Windows wrapper around rsync incorporating a scheduler. If that's done properly, it should be alright for the average Windows user.
Re:SSD (Score:3, Informative)
That's a fairly naive cost assessment. The lifespan of that SSD is much shorter than the 2.5" drive. I'd suspect that it is at least half as long, making it in fact 400 times as expensive, and (most importantly) requiring more frequent backups.
Assuming what you say is true (due to wear levelling and the utter lack of moving parts, I'm not inclined to agree), the issue I brought up is still there. It's only more expensive if you reach the threshold.
Everybody has different things to consider. Cost-per-gig is not a one-size-fits-all answer.
Re:External and Online (Score:2, Informative)
I have a pair of external FW800 drives in a software RAID-1 configuration. Every month I bring one of them home from the office, remirror, break the mirror again and take it back to the office.
If I have a total system loss (fire, theft, etc.) I'm worst case 1 month out of date.
My very critical data that I can't tolerate a 1 month loss of, I use online backup (http://www.tarsnap.com)
Re:SSD (Score:4, Informative)
Bullshit.
I've got a pretty large collection of old games. One of the things I've noticed about them is that, even for factory-pressed discs, you have to deal with the following factors:
- Unsealing. If the disc wasn't sealed properly, it will begin to oxidize. Several of mine have done this. It moves from the edge inwards.
- Plastic clouding. Even if it's stored in a cool, dry place and kept out of sunlight, clouding can and will occur. The cheaper the disc was produced, the sooner this happens. Many of my older titles (Civ 2 among them) suffer from precisely this problem and that's one reason I began taking them all and making ISO backups just in case as well.
- Physical usage. If it's something you use often, wear and tear occurs. Games have access patterns that are clearly different from music CD's. For one, music CD's tend to spin at the standard 1X rate (unless you're seeking), which doesn't cause the disc to deform much (as opposed to full-speed data access... see the high-speed footage from the Mythbusters ep on this if you want). Two, data CD's access a lot more erratically - and the more "back and forth" you have, the more likely you are to scratch the disc from ordinary use. If you put an older CD (not physically designed for higher-speed drives) into a modern higher-speed drive, you can make the problem worse.
Of course, sure, you could abuse them. I saw a Rock Band disc once that looked like an LP - someone had put it in at a convention 15 feet away from the DDR setup, and the room had been bouncing so violently that the disc read head had literally put grooves into the disc. But I doubt that's what happened to the gpp's Civ 2 disc.
Re:SSD (Score:3, Informative)
Your statements are logically consistent, but I respectfully agree with Mr. Coward for the following reasons.
1) Main features of backups non-exhaustively include the ability to restore files and to restore functionality. You might be able to restore functionality by ripping from a commercial disc, but the resulting byte stream will almost certainly be different from the original. Even with respect to restoring functionality, ripping from a commercial disc will almost certainly take longer than copying an encoded file with the same human level content. And that does not include important meta-data such as ratings, access times, album organization/collections, album covers, etc.
2) Clearly the intent of this entire story is to discuss best practices with respect to backing up user generated data, of which one special case is audio-visual content, of which one special case is content which is available commercially as pressed discs.
In short, you have presented one relatively expensive solution for one special case of a special case, which as you have acknowledged, is not generalizable as you've described it to the problem being discussed in the story. Pressed discs as a backup solution fit neither the spirit nor the letter of the problem described.
About the only way to discuss pressed discs with relevance to a home backup method would be to point out the costs of getting a disc pressed for really irreplaceable data (currently around $10^3 to $10^4 per disc), or the relative merits of the various optical disc formats with respect to the probable availability of devices to read them at various points in the future (signals from a CD-* can be read with a flatbed scanner, but blue laser devices which can record at a higher density are more likely to be relevant for data storage in two decades than red laser devices).
Re:Differential + hard drive - online (Score:1, Informative)
Mozy is fast because it relies on the idea that most people are backing up the same files. eg all your DLLs that come with Windows. There is nothing to upload, it just remembers that you have that file.
But it won't be fast if you are backing up photos, images, Adobe Illustrator files, original documents, conference proceedings, baby videos, and other unique files that noone else would have. If you generate a lot of documents, then its gonna take a while to upload.
Re:External and Online (Score:3, Informative)
Re:SSD (Score:3, Informative)
Well here I go again:
1) You need to have a backup strategy first:. The strategy will be elaborated according to your needs and the amount of risk that you are willing to take.
2) Incremental backups should be part of your strategy.
3) You should backup on a second computer to prevent against a single point of failure (e.g. you controller). It is best that this computer is physically hosted at a different site than your production computer to prevent against disasters. If not, you should at least take full images once in a while that you store somewhere else. Again, this all depends on the amount of risk that you are willing to take, no backup solution is 100% proof. A fire at my bank, a fire at my prod site and a fire at my backup site occurring at about the same time would leave me totally screwed.
http://slashdot.org/comments.pl?sid=1292323&cid=28588177&art_pos=40 [slashdot.org]
http://slashdot.org/comments.pl?sid=1292323&cid=28588079&art_pos=41 [slashdot.org]
http://slashdot.org/comments.pl?sid=1277921&cid=28430691&art_pos=76 [slashdot.org]
http://slashdot.org/comments.pl?sid=1277921&cid=28429713&art_pos=78 [slashdot.org]
Re:SSD (Score:3, Informative)
From common usage (rather than one pedant), it's a verb, a noun, and an adjective.
Verb: You should backup your data
Noun: All my data is on a backup
Adjective: You need a backup disk