Ask Slashdot: Asynchronous RAID-1 Free Software Backup For Laptops? 227
First time accepted submitter ormembar writes "I have a laptop with a 1 TB hard disk. I use rsync to perform my backups (hopefully quite regularly) on an external 1 TB hard disk. But, with such a large hard disk, it takes quite some time to perform backups because rsync scans the whole disk for updates (15 minutes in average). Does it exist somewhere a kind of asynchronous RAID-1 free software that would record in a journal all the changes that I perform on the disk and replay this journal later, when I plug my external hard disk on the laptop? I guess that it would be faster than usual backup solutions (rsync, unison, you name it) that scan the whole partitions every time. Do you feel the same annoyance when backing up laptops?"
ZFS: Snapshot + send (Score:2, Interesting)
Cleanest implementation of this I've seen is with ZFS.
You do a snapshot of your filesystem, and then do a zfs send to your remote backup server, which then replicates that snapshot by replaying the differences. If you are experiencing poor speed due to read/write buffering issues, pipe through mbuffer.
The only issue is that it requires that you have your OS on top of ZFS.
My solution (Score:1, Interesting)
Is to not try to keep 1TB of crap on a laptop... or anywhere for that matter. Travel light says me ;)
Re:Time Machine (Score:4, Interesting)
This doesn't match my experience. Time Machine fires up in the background, does its thing, and then stops shortly thereafter. Certainly much less than 15 minutes. More like five or less. This is on a new-ish iMac with a 3TB internal drive.
It wouldn't even be noticeable were it not for the fact that I can hear the TM destination drive (sitting on a shelf behind me) spin up once an hour.
Re:Time Machine (Score:4, Interesting)
This is my current experience with mine too. However during the prep stage it is making room on my time machine drive to receive the changes. Consolidating the older files will take time.
When my drive was new and had plenty of space, the prep stage was much shorter.
Whooosh (Score:4, Interesting)
Holy cow people, your missing the OP point. It's taking 15 minutes to SCAN the 1TB drive.
I've run into the same problem on windows and Linux. Especially for remote rsync updates on Linux on slow wireless connections. It's not the 1TB that kills since I can read 4TB drives with hundreds of movies in seconds. It's the amount of files that kill performance.
My solution on windows is to take some of the directories with 10,000 files and put them into an archive (think clipart directories). Zip, Truecrypt, tar, whatever. This speeds up reading the sub-directories immensely. Obviously, this only works for directories that are not accessed frequently. Also, FAT32 is much faster on 3000+ files in a directory than NTFS is. Most of my truecrypt volumes with LOTS of files are using FAT32 just because of the directory reading speed.
On Linux systems, I just run rsync on SUB-directories. I run the frequently accessed ones more often and the less-accessed directories less often. Simple, No. My rsyncs are all across the wire, so I need the speed. Plus some users are on cell-phone wireless plans, so need to minimize data usage.
Re:mdadm can do this (Score:5, Interesting)
Re:Just use Windows Backup (Score:4, Interesting)
Unless you're running Windows 8 or Server 2012, Windows Backup on Windows 7 and below is functionally obsolete due to the new 3TB + drives now in 4k sector Advanced Format technology.
Nice. So because you can buy large-capacity drives that immediately would "functionally obsolete" backup solutions even if a system does not have such a drive? Tell me, did you buy a new BMW when apple changed the connector for iPhone 5? You know, the old BMW are now "functionally obsolete".
Not that it matters much here anyway, because you got it wrong. Windows backup *will* backup to drives larger than 3TBs - as long as they use the 512e advanced formatting where it logically uses 512 bytes allocation units but physically 4096 bytes units. The solution is to use the GPT (GUID Partition Table) format. This will work for Vista and up.
The drives that are exclusively 4096 cannot be used with Windows 7 / Server 2012 - that's a limitation of the OS and not the backup software, however.
Re:find & diff (Score:4, Interesting)
Just curious, why do you require access time? I set 'noatime' on all partitions.