Forgot your password?
typodupeerror
Data Storage

Ask Slashdot: Simple Way To Backup 24TB of Data Onto USB HDDs ? 405

Posted by samzenpus
from the save-often dept.
An anonymous reader writes "Hi there ! I'm looking for a simple solution to backup a big data set consisting of files between 3MB and 20GB, for a total of 24TB, onto multiple hard drives (usb, firewire, whatever) I am aware of many backup tools which split the backup onto multiple DVDs with the infamous 'insert disc N and press continue', but I haven't come across one that can do it with external hard drives (insert next USB device...). OS not relevant, but Linux (console) or MacOS (GUI) preferred... Did I miss something or is there no such thing already done, and am I doomed to code it myself ?"
This discussion has been archived. No new comments can be posted.

Ask Slashdot: Simple Way To Backup 24TB of Data Onto USB HDDs ?

Comments Filter:
  • Julian? (Score:5, Funny)

    by WinstonWolfIT (1550079) on Friday August 10, 2012 @05:35AM (#40943569)

    Out on bail mate?

  • You know... (Score:5, Funny)

    by marsu_k (701360) on Friday August 10, 2012 @05:59AM (#40943683)
    Porn is a renewable resource, there's no need to store so much of it.
  • by jamesh (87723) on Friday August 10, 2012 @06:20AM (#40943779)

    If the OP's porn collection can be logically broken up at some level, eg:

    /porn/blonde
    /porn/brunette
    /porn/redhead

    then the backup software could create one job for each directory, and multiple USB disks could be attached at once giving increased throughput. USB3 also increases speed to the point where the 7200RPM disk itself will become the bottleneck.

    So at 100MB/second per disk write speed with 4 disks going at once (assuming the source disks are capable of this supplying this volume of data and there are no other throughput limitations), you could do it in 16 hours, or 24 hours with more realistic margins.

    If it turns out that the source data is not porn (unlikely) and is highly compressible, then it could be done in far less time.

    Bacula can do all of this.

  • by Anonymous Coward on Friday August 10, 2012 @06:39AM (#40943869)

    Or, he could watch the content as it is copied. At 600 Mbytes/hour (assuming standard mpeg compression), it would be a month of 24/7 nonstop action!

    "- Hey boss, I need to, uhh, work from home for the next four weeks to handle the backup..."

  • by Pieroxy (222434) on Friday August 10, 2012 @06:42AM (#40943883) Homepage

    then the backup software could create one job for each directory,

    Is that what we call a blow job?

  • I know (Score:2, Funny)

    by Anonymous Coward on Friday August 10, 2012 @07:23AM (#40944069)

    The iCloud! ;-)

  • by zapyon (575974) on Friday August 10, 2012 @07:28AM (#40944083)

    "Only wimps use tape[*] backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)"
    Linus Torvalds (1996) http://en.wikiquote.org/wiki/Linus_Torvalds [wikiquote.org]

    (Isn't that prescience of "The Cloud"?)

    ––––––––––
    * replace this with your favorite backup media of today ;-)

  • by ilikejam (762039) on Friday August 10, 2012 @07:45AM (#40944173) Homepage

    No. No it is not.

  • by Anonymous Coward on Friday August 10, 2012 @08:11AM (#40944325)

    Bacula can do all of this

    So he quantum leaps into you, and isn't allowed to leave until he performs the backup? Oh wait! Bacula, not Bakula.

  • by deniable (76198) on Friday August 10, 2012 @08:33AM (#40944469)
    Send error messages to a Blackberry and it's a RIM job.
  • by deniable (76198) on Friday August 10, 2012 @08:34AM (#40944475)
    Bacula went on to be Enterprise grade software.
  • by MMC Monster (602931) on Friday August 10, 2012 @08:48AM (#40944563)

    Maybe he's personally backing up CERN?

  • by hairyfeet (841228) <bassbeast1968@NOsPAM.gmail.com> on Friday August 10, 2012 @02:23PM (#40949193) Journal

    USB would be just the most retarded way to go for something like this, its too slow and he's gonna be swapping worse than when we used to have to back up things to CDs.

    I'm guessing he's going USB because he don't have the cash to buy a NAS of that size but you can always jury rig you a NAS, its really not hard. We did something similar at the last shop I worked at when the boss scored a ton of SCSI drives at an auction and ended up with nearly a Tb NAS when the average HDD was 40Gb. Here is how you do it..

    You take a couple of full size towers, bigger the better, preferably twinkies as it makes the job a LOT easier. You strip 'em to the frames and use a couple of spot welds to make them into one giant case along with another couple of weld to mount a shitload of drive cages into the case. Then you take a cheap server or even desktop board, all that matters is it has a shitload of PCI slots which you fill with controller cards, SCSI in our case but SATA today, mount the board along with a big PSU to feed the drives and voila! One big ass DIY NAS unit that can hold a huge pile of drives. Just to finish our white trash conversion we tied on a Walmart box fan to keep the sucker cool and stuck it in a corner, worked great.

    The only software that I think would work with USB is Paragon Drive Backup [drive-backup.com] as you can have it split by just about any size you want. They also have their own Linux based recovery media but damned if i know if you can get the software as a Linux installer, never ran into that situation to need it in that way. I know its worked great for me making OS images and backing up files and folders onto USB drives but if you're gonna be splitting to a ton of little drives then you are just gonna have to swap, no way out of that. If you want to fill the drives up then set Paragon to a small size, say 700Mb, but good fucking luck checking your backup as the amount of swapping you're gonna do is just insane.

To err is human -- to blame it on a computer is even more so.

Working...