Forgot your password?
typodupeerror
Data Storage

How the LHC Is Reviving Magnetic Tape 267

Posted by samzenpus
from the what's-old-is-new dept.
sandbagger writes "The Large Hadron Collider is the world's biggest science experiment. When spinning, it reportedly generates up to six gigs of data per second. Today's six-terabyte tape cartridges fill rapidly when you're creating that amount of material. The Economist reports that despite the advances in SSDs and hard drives, tape still seems to be the way to go when you need to store massive amounts of digital assets."
This discussion has been archived. No new comments can be posted.

How the LHC Is Reviving Magnetic Tape

Comments Filter:
  • by Gothmolly (148874) on Monday December 02, 2013 @12:17PM (#45574963)

    Of a station wagon loaded with tapes.

    Also, -1, Duh, because this is an obvious, stupid article.

    • by Danathar (267989)

      Why use a Station Wagon? Why not a 747?

    • by Isca (550291) on Monday December 02, 2013 @12:24PM (#45575029)
      Actually I found the article informative. I knew tapes were the cheapest and most cost effective backup solution but I didn't realize that they were so fast once the tap has been loaded.

      It's also interesting to see the advances in tape reading technology that they are striving for - it sounds as if it will keep pace with HD and SSD technology to keep staying relevant.
      • by SuricouRaven (1897204) on Monday December 02, 2013 @12:40PM (#45575215)

        Cheapest, sort of.

        The price of storage roughly follows the y=mx+c linear graph: m is the cost of the media, while c is the cost of the equipment needed to access it.

        For hard drives, it's easy: c=0. A drive is self-contained.

        For tape, c is large (Up to several thousand pounds for one tape drive), but m is smaller (Tape, purchased in bulk, is cheap).

        So if you're storing a small amount of data, a rack full of hard drives is cheaper. For larger amounts, tape is cheaper.

        This ignores issues of ease of access and management software.

        • by aaarrrgggh (9205)

          Tape works very well for applications where you have a large number of archival batches; us use the fixed resources (parent's "c") more per TB storage. Once you have a tape silo with 100,000+ slots, your incremental storage costs are very small.

        • interesting graph, but I think your explanation on C is a little muddy. I would just say C is the initial cost before any storage medium is acquired.

      • by dshk (838175) on Monday December 02, 2013 @01:00PM (#45575423)

        Yes, they are surprisingly fast. The maximum speed of a current Tandberg LTO-6 drive is 160 megabytes/s if the data is uncompressable. With the usual compressible data it can be about 320 megabytes/s (officially 400).

        These drives can even be too fast. The drives do speed matching, but they have a minimum speed, below that they start shoe-shining. One reason I have chosen an older generation, LTO-3 tape drive, instead of the current generation, because I cannot easily feed an LTO-6 with at least 60 MB/s, which is the minimum speed of the drive. Considering compression, that is about 120 MB/s, which saturates a 1Gb network.

        • by TooMuchToDo (882796) on Monday December 02, 2013 @01:45PM (#45575901)

          I used to work on data taking for the CMS detector at the LHC. We were using Storagetek tape silos [http://computing.fnal.gov/cdtracks/2009/january/images/robot.jpg [fnal.gov]] for long-term storage of data at Tier1.

          Tape allows for cheaper storage and large capacities, but you're then fighting contention issues (there are only so many robotic arms and tape drives for your tape library) as well as having data on tapes go bad without knowing it. When data is on disk, I can at least verify it immediately. Bit rot is definitely alive and well on tape.

          • by NeoMorphy (576507)

            You can use "logical block protection" and multiple copies so that you can save archive copies in protected vaults, which will increase your data integrity by having multiple copies in different locations as well as increased protection from bit rot(from cosmic rays at least). Multiple copies can be created simultaneously, at the cost of tape drives.

            You still have to read the entire disk copy to verify, which could take awhile if it's several TB in size. Though you still have several options to protect it f

        • by Doc Hopper (59070) <slashdot@barnson.org> on Monday December 02, 2013 @01:46PM (#45575911) Homepage Journal

          The drives do speed matching, but they have a minimum speed, below that they start shoe-shining.

          Agreed. At my work we do parallel streams to multiple Sun T10000 T2 tapes (T10K "C" drives) at 250Mbyte/sec uncompressed (500 megabytes per second compressed, more or less, usually quite a bit more). If for some reason we push less than about 120mbytes/sec, the tape rewind times cause all kinds of issues.

          We make the same kind of decision when choosing Sun T10000 "B" drives instead of "C" or the new "D" drives if the source cannot push data fast enough.

          I've long laughed at articles saying tape is dead. For large-scale* backup, retention, transport, and legal hold problems, there simply is no other solution that scales reasonably well.

          *My definition of "large-scale" for this specific context: hundreds of terabytes or more, much of it transported thousands of miles regularly. If you don't work with hundreds of terabytes and at least dozens of petabytes on a daily basis, you may suffer from optimistic delusions regarding disk storage capabilities, one which disk storage vendors are all too glad to reinforce, to the detriment of customers faced with half-baked solutions that cannot hope to meet their throughput requirements. Given "large-scale" data, there's no replacement for tape at present; everything else is a low-throughput also-ran, typically harboring enormous and unplanned complications. We're also heavy users of VTL, replication, cloning, S3-workalikes, and various disk technologies. Tape remains vital to large enterprise operations, and those predicting its imminent death have been the butt of jokes about marketing wonks for a decade and a half.

        • by mlts (1038732) *

          In a previous life, I had to go to 10GB in order to get anywhere near the speed of LTO-6 media.

          However, there is one advantage of LTO-4 and newer over LTO-3, and that is hardware tape encryption via SPIN/SPOUT commands. It doesn't sound like a big deal, but it gets rid of a lot of headaches fast, especially the fear of a tape falling off the back of the Iron Maiden truck doing its rounds.

          LTO 4 and 5 are approaching price points that may seem expensive, but for a SMB, a dedicated server with one of the driv

        • by Chris Pimlott (16212) on Monday December 02, 2013 @02:52PM (#45576507)

          These drives can even be too fast. The drives do speed matching, but they have a minimum speed, below that they start shoe-shining.

          Er, "shoe-shining"? What do you mean?

          • Re: (Score:3, Informative)

            by Anonymous Coward

            >"shoe-shining"

            When the tape drive repeatedly and quickly does forward and reverse operations over the same piece of tape due to data fault on the tape or some buffer problem (or other reasons, too). The analogy is to the quick back-and-forth of a shoe-shine rag that runs the same piece of rag over a shoe many times.

            Ah, memories...

          • by evilviper (135110)

            Er, "shoe-shining"? What do you mean?

            Would have been quicker and easier to look it up, yourself, than asking here, and waiting for a response:

            https://en.wikipedia.org/wiki/Tape_drive#shoe-shining [wikipedia.org]

            • by EETech1 (1179269)

              Asking the question here is kinda like saying "tell me a story about the olden days grandpa" sure you could go to the library and read a book, but hopefully a few folks here will get off their lawn long enough to tell us an amusing shoe-shining story, or reminisce about their experiences with some bad-ass (or slow-ass) hardware to add some depth and sense of community to the discussion you just don't get from a Google search.

              I'd rather hear it here, and have it inline with the discussion for everyone to enj

    • by mlts (1038732) *

      Station wagons are hard to find. I'd probably go with a Euro style van [1] or a crossover.

      [1]: Sprinters are the best, Ducatos/ProMasters are decent, and Transits would be OK... but won't be on this side of the pond until next summer.

  • i remember a few years back backup to "cheap" disk was all the rage. if you were backing up to tape you were seen as some kind of mental patient

    tape has its issues, but sucking up money like a trophy wife isn't one of them

    • I also sometimes get the "mental patient" stamp for saying that I still use optical discs.

      I just cringe the idea of storing long term archived data using an electric charge (flash, HDD, tape). Optical disc has also the benefit of being truly read-only so that you or a piece of malware cannot destroy the data afterwards by software.

      • by MightyMartian (840721) on Monday December 02, 2013 @12:31PM (#45575133) Journal

        It depends on the optical disc. If you fork out the money for an archival media like gold CDs or DVDs, then you can probably expect something like 20 to 40 years. All in all, from what I've read, tape still is king in long term storage.

        • by drinkypoo (153816)

          We've already seen some of the long-life optical discs fail long before their life was supposed to end. Tape in a cool, dark place is the answer.

          • by Doc Hopper (59070) <slashdot@barnson.org> on Monday December 02, 2013 @01:55PM (#45576009) Homepage Journal

            long-life optical discs fail... [store] tape in a cool, dark place...

            This, this, one-thousand times this. I've worked in data centers for a decade and a half, and seen innumerable optical media go bad within just a few years (typically about 3 years) even in DVD jukeboxes in climate-controlled environments. Meanwhile, we restore from fairly ancient tapes on a regular basis.

            In reality, most companies don't store tapes longer than 7 years anyway; that's the upper limit of typical audit liability. The data on the tapes may be older than that, kept indefinitely on-disk, but most large companies have a fairly aggressive destruction/over-write schedule for data on tape older than 7 years.

            It's very unlikely we'll need data off a tape 20 years from now, but kept in the right conditions -- like the bat-cave of a tape silo room housing tens of thousands of 10TB tapes a few feet away from me right now -- there's a really good chance the data will be readable. While we do have plenty of tape failures (hundreds per year), they are almost always caught at write-time by the verification head.

            On a modern tape drive, you usually have several dozen "heads" on any given tape drive, and there will be two sets of them each with its own mechanism to align it with a precision of just a few microns. Pretty amazing, really; if you drop by the Denver, CO area some time, the Oracle/Sun building engineers there can often arrange a tour of our tape testing facilities if you sign a NDA and represent a potential sale. Anyway, the second mechanism will be engaged on the tape in order to read what the first just wrote and verify it before it passes the "successful write" confirmation back up the fibre channel chain. This way you can guarantee you don't get "write once, read never" media.

        • It depends on the optical disc. If you fork out the money for an archival media like gold CDs or DVDs, then you can probably expect something like 20 to 40 years. All in all, from what I've read, tape still is king in long term storage.

          "Archival" optical discs are a scam. They fare no better than Office Depot brand shit.

      • Writeable optical storage has severe issues with longevity. The medium is chemical in nature, and degrades over time. Optical discs are fine for short term, but don't depend on them still being readable in even a year.

      • by MightyYar (622222) on Monday December 02, 2013 @12:44PM (#45575253)

        Just be careful - optical disks degrade, too. Years ago before hard drives became so incredibly dirt cheap, I would do my little video editing thing and then back up the project files to DVD. And not just any DVD - I did my homework and found the best-rated archival DVDs (sorry, don't remember the brand - only that they came from Japan). Anyway, I just sucked them back onto my NAS, and some of them had developed a teeny bit of unreadable data. Fortunately, I had made PAR2 files for everything. Between par2repair and ddrescue, I was able to recover the data. But the moral of the story is don't rely on optical disks to be magical storage that does not degrade.

        • Take a look at M-Disc. You'll need an M-Disc compatible burner, and the discs are more expensive, but the data is safe for up to 1,000 years. It's literally etched in stone. What is M-Disc? [mdisc.com]
          • by MightyYar (622222)

            I have never seen those. It's a great idea. I'm still skeptical, though. I've come to believe that the best way to protect your data is to keep it live, if you can afford it. I try to keep two local live copies and one remote. I also archive things, but I can't count on those IMHO.

            My most important files are family photos and videos. I try to keep the raw files live, and in addition every time I produce a DVD from the videos, I also copy any photos from the same time period onto the DVD. Then, I send the DV

      • by Joce640k (829181)

        I just cringe the idea of storing long term archived data using an electric charge (flash, HDD, tape).

        I think you'll find that two of those use a magnetic charge.

      • HDD and tape do not use electric charge. They use spin orientation, what is much more stable, and self sustaining. Flash do use charges, and optical disk, chemical reactions. Theoreticaly, spin orientation is the most stable of those options.

        By the way, where is the phase change memory IBM was promissing us 15 years ago? Moving atoms full nanometers from their original position... that would be stable.

      • by rickb928 (945187) on Monday December 02, 2013 @01:07PM (#45575493) Homepage Journal

        The bottom line in managing long-term archiving (5+ years) is that you need to both refresh and verify you storage, at several different levels.

        1. Shoot the initial copy.
        2. Copy this asap. "Copy1"
        3. Stash both in disparate locations.
        4. Go back to the 'original' on a 6-9 month schedule and verify it.
        5. Go back to the 'copy1' on a schedule and verify it on a different schedule.
        6. Go back to the 'original' on a different 9-12 month schedule and refresh(copy) it, stored to the other site.
        7. Go back to the 'copy1' on a different schedule and refresh (copy) it, stored to the other site.
        8. Repeat 4&5 on a year schedule. Do you need to re-write the data in 'current' formats and retain both original and new? Are you moving to new media?
        9. Repeat 6&7 on a year schedule. Ditto the rest of step 8.
        10. We should be at year 2 or 2.5. Repeat steps 1-9 once for a 6+/- year retention, again for 10+ year retention.

        Are you changing data formats, and is it possible to ensure integrity by copy8ing and archiving in new formats?
        As you change media, do you need to retain old media systems, or will you move to the new media?
        At what point is the data no longer valid, determined by the owners?
        Are the 'owners' the only stakeholders? If not, expand the set.

        In all of this, you have a dedicated media management system including media drives, copy/verify capabilities, and stand-in for restoration.

        This is all very interesting to me. Medical records in particular seem to be assumed to have a lifetime retention, but other than the date and nature of the event, how important are the details of your appendectomy performed at age 5 when you are 60? Is that benign tumor removed at age 12 important at age 45? How much LHC data collected in 2013 will be useful in 2023? Different criteria. Different processes.

        • by Doc Hopper (59070) <slashdot@barnson.org> on Monday December 02, 2013 @02:09PM (#45576103) Homepage Journal

          ...you need to both refresh and verify you storage...

          You came pretty close with the process, but for most businesses you're not quite there. Here are a few clarifications on the process.

          1. Typically large companies (including those, like us, with stringent HIPAA requirements) take two simultaneous copies from the original source. We don't copy a copy if it can be avoided, and we have enough tape drives to do this.
          2. We contract out with a local storage company to grab the tapes within a few days and store for the given retention period off-site. One copy usually remains on-site as well for long-term retention and rapid restoration. With plenty of capacity in the silo (tens of thousands of tapes in an Oracle/Sun SL8500), we are not terribly concerned about retention policies. If we get tight on space, we'll just expand the silo again.
          3. The same data usually still exists as on-disk media marked read-only, available for the legal folks who insisted we archive it in the first place. Often it also exists at a second geographical location thousands of miles (at minimum) from the first, with its own backup tapes. Plus it exists on two tapes at each site, one near-line and one off-site. Given tape reliability, three layers of data protection is typically sufficient. If "legal hold" is involved, we also insist that the disk array be kept on a valid support contract to reduce the risk of failed disks in the storage appliance.
          4. Retention policies dictate we keep around at least a few tape drives of every generation we've ever used which has tapes archived with our off-site storage facility. Even if they are not in the silo, they're in a storage closet waiting for us to bring them to life if needed up to twenty years later.

          I do this kind of thing all the time. Feel free to ping me at my easily-figured-out email address (firstname@lastname.org) if I can answer additional questions for you.

      • by mlts (1038732) *

        Join the asylum. I have had good luck with optical storage, and have been able to restore data from 1998 from burned CDs. This doesn't mean that I've not have had my share of coasters, but it is a decent, inexpensive medium to save data off securely.

        It may not be exotic, but WinRAR and Nero have done a decent job at keeping files that I don't want cluttering my NAS.

    • I've always insisted on a tape backup system. Hard drive backups certainly have their place, but tape cannot be beat for long-term archival storage. One of our weekly offsight backups goes into a safety deposit box, where sits a duplicate tape drive. I don't want to be searching around for a replacement while my organization is down and out due to some cataclysmic failure.

      • Per Murphy's law, you do need two tape drives in that deposit box.

        • by drinkypoo (153816)

          Per Murphy's law, you do need two tape drives in that deposit box.

          Only if you can plug the replacement in wrong. Murphy's law is often incorrectly interpreted to refer to failures. It isn't about that. It's about fuckups.

          • by Minwee (522556)
            And Finagle's Law states that anyone trying to explain the difference between it and Murphy's Law will get it wrong.
      • Hard drive storage > tape storage.

        How quickly can you verify the integrity of your off-site tape backup? I can verify my hot backups in S3/Glacier in seconds.

    • Check the undetected and uncorrectable bit error rates for cheap disk. Tape drives even read everything written to insure it got there and is correct allowing it to retry the operation till it works or it fails out the tape. There is no "consumer" grade tape anymore without all those nice enterprise features baked in.

      Current pricing is about $60 for 1.5TB ($120 for a 3tb drive) for consumer disk vs LTO5 $30 + 2k for a tape head (LTO6 is more expensive as the takes a 3x the cost).

      • by Doc Hopper (59070)

        Current pricing is about $60 for 1.5TB ($120 for a 3tb drive) for consumer disk vs LTO5 $30 + 2k for a tape head (LTO6 is more expensive as the takes a 3x the cost).

        $160 for a Sun T10000T2 tape, uncompressed capacity 5TB, typically holds a touch over 10TB in our usage (assuming you have something that can pump 250Mbyte/sec to the drive, which requires a bit of engineering to get right). That's $16/TB, and you can bring that down quite a bit if you buy in the "tens of thousands" kind of bulk we do.

        The drive

        • $32 a TB never count compression, 8.5TB with the newer tapes/heads and max "feature" not sure one price as it's a sole vendor product I wont go near it unless it's the only option.

    • by Joce640k (829181)

      tape has its issues, but sucking up money like a trophy wife isn't one of them

      It's OK for jobs like this where it doesn't really matter if you lose a few bits of data every now and again.

    • by tlhIngan (30335)

      i remember a few years back backup to "cheap" disk was all the rage. if you were backing up to tape you were seen as some kind of mental patient

      tape has its issues, but sucking up money like a trophy wife isn't one of them

      Depends how much you want to store.

      If it's just a little bit, using a DVD-R is perfectly adequate as a backup solution. Even BD-R for slightly larger amounts.

      But if you have to store more, say a few to double digit TB, hard drives might be a reasonable solution - they're quite cheap and of

      • by compro01 (777531)

        If it's just a little bit, using a DVD-R is perfectly adequate as a backup solution. Even BD-R for slightly larger amounts.

        And you're not needing to back it up for very long. Writable optical media doesn't have a very long archival lifespan.

      • by alen (225700)

        who in their right mind is going to store 4TB of company data on unRAIDed disk?

        first you have to buy the enterprise gear with the 4 hour or next day warranty. then RAID-5 at the minimum and probably RAID-6. and if your server runs out of storage its $3000 for a jbod plus $1000 for the RAID controller and then the disks

        and then you have to have a second backup server with the same data standing by just in case your primary server crashes. disks have this problem where if the OS crashed or something happens t

        • by mlts (1038732) *

          For enterprise grade SAN use, there are features like snapshotting done by the SAN controller (so if malware does strike, it can't get anywhere other than foul up the present LUNs), and asynchronous replication to a remote site (which a larger enterprise will need.)

          The bigger volumes, you want RAID 6 and hot spares. Otherwise, there is a good chance of getting caught with your pants down while a rebuild is taking place.

          Then there is archival storage. Storing stuff as an archive on hard disks means that th

      • by Doc Hopper (59070)

        There's a balance between cost per byte and initial acquisition cost. DVDs and BDs have extremely low acquisition costs, but relatively high cost per byte (you can get started with $100). Hard drives have higher costs, but lower cost per byte. Tape has the least cost per byte, but initial costs are quite high...

        Nailed it. I've never seen it expressed quite that concisely. Great work! Think I'll be using this quote regularly from now on.

    • I'm a fan of tapes too, partly because in the SMB space even the dumbest luser can change a tape, but changing out a disk drive on a Windows system *always* seems to be problematic.

      Usually you're stuck with USB for ease of use, and even USB2 blows for throughput and I have yet to see a new server with USB3. And then there's the whole clusterfuck with drive letter assignments and the crummy job backup software does with identifying backup media vs. needing to write to some specific path (which is as much a

    • by Darinbob (1142669)

      The mental patient view comes from people who are pretty close to becoming asylum regulars anyway. They have an extremely myopic viewpoint that comes from having a lack of experience and a fondness for new things. Many companies still back up to tape because it is what makes sense, even if it isn't cool and hip and the twenty somethings cry when you do it.

  • No shit Sherlock (Score:4, Informative)

    by morcego (260031) on Monday December 02, 2013 @12:42PM (#45575235)

    No one in the data retention business ever stopped using tapes. See the numbers on LTO units being sold, if you need proof.

    This is a shitty article.

  • by cyberchondriac (456626) on Monday December 02, 2013 @12:45PM (#45575259) Journal
    ..has been greatly exaggerated lately by trade journals. There are some backup scenarios for which hard disc backup just isn't viable.
    Viva la tape.
  • by Albanach (527650) on Monday December 02, 2013 @12:45PM (#45575265) Homepage

    A couple of years ago, Google restored lost gmail from tape [blogspot.com]. I'd expect that even with deduplication they must use a phenomonal amount of tape.

  • Magnetic tape has been alive and well. Most big companies and research labs use it daily.

    Sounds like the article writer knows nothing at all about corporate or industrial IT.

  • I'd love to see a Petabyte-Scale Tape Storage System that looked something like this, only modernized: http://youtu.be/Nq3mNYKR7FM [youtu.be]
    • by Doc Hopper (59070)

      I'd love to see a Petabyte-Scale Tape Storage System that looked something like this...

      You're thinking way, way, way too small. One of my tape silos a few feet away from me holds about 135 petabytes, and if we bought some expansion cabinets it would hold enormously more than that.

      IMHO, a SL8500 silo with dozens of robots shuffling tapes around under the LED lights looks way cooler than a pair of spinning reels. Here's a small sample: http://www.youtube.com/watch?v=OpVnk_GeCaw [youtube.com]

  • Or can I sell my cartridges on Craigslist now?
  • by rossdee (243626) on Monday December 02, 2013 @01:20PM (#45575625)

    " When spinning it reportedly generates up to six gigs of data per second."

    The LHC itself doesn't spin, rvrn though there are protons moving around the circular track at very near lightspeed. /pedant

  • by bravecanadian (638315) on Monday December 02, 2013 @01:21PM (#45575639)

    For *reliable* backup and archive purposes tape never went out of style.

  • by BenJeremy (181303) on Monday December 02, 2013 @01:37PM (#45575797)

    I've worked as a tape monkey in a large facility (Camp Foster RASC, Okinawa, circa 1989-90), so I know tapes do work well in the enterprise, but my experience with tapes in the consumer space in the 90s was anything but good. 90% of the tape backups made (using several different formats) using consumer-grade systems were corrupt and worthless.

    We took great care with the tapes, but when we checked them (thankfully never needed them, except one occasion), they were mostly all bad.

    Optical isn't much more reassuring as a backup media, given that optical discs tend to degrade over time.

    If somebody has a tape system that can store terabytes on a cartridge, reliably, for say... $10/TB or less, and the system costs less than $200, I'd look at it, though. Otherwise, it is still more worthwhile just to use hard drives to back up data (even at their inflated prices)

Man is the best computer we can put aboard a spacecraft ... and the only one that can be mass produced with unskilled labor. -- Wernher von Braun

Working...