Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Data Storage Intel Upgrades

There's No End In Sight For Data Storage Capacity (computerworld.com) 107

Posted by timothy from the heat-death-of-the-universe-comes-to-mind dept.
Lucas123 writes: Several key technologies are coming to market in the next three years that will ensure data storage will not only keep up with but exceed demand. Heat-assisted magnetic recording and bit-patterned media promise to increase hard drive capacity initially by 40% and later by 10-fold, or as Seagate's marketing proclaims: 20TB hard drives by 2020. At the same time, resistive RAM technologies, such as Intel/Micron's 3D XPoint, promise storage-class memory that's 1,000 times faster and more resilient than today's NAND flash, but it will be expensive — at first. Meanwhile, NAND flash makers have created roadmaps for 3D NAND technology that will grow to more than 100 layers in the next two to three generations, increasing performance and capacity while ultimately lowering costs to that of hard drives."Very soon flash will be cheaper than rotating media," said Siva Sivaram, executive vice president of memory at SanDisk.
This discussion has been archived. No new comments can be posted.

There's No End In Sight For Data Storage Capacity More

There's No End In Sight For Data Storage Capacity

Comments Filter:

  • Reliability? (Score:5, Insightful)

    by Tablizer ( 95088 ) on Wednesday March 09, 2016 @02:24PM (#51667519) Journal

    I'll generally take reliability over volume. I wish they'd work on that more.

    • Re: (Score:1)

      by Anonymous Coward
      Found this new technology that promises to aggregate disks and clouds to create a virtual and reliable storage pool: http://infinit.sh./ [infinit.sh.] Haven't tried it yet though. If someone has, it would be interesting to get feedback...

    • Re:Reliability? (Score:5, Interesting)

      by Distan ( 122159 ) on Wednesday March 09, 2016 @02:33PM (#51667565)

      The big purchasers of storage want the opposite. They prefer cheap capacity over reliability. All their data is replicated multiple time so losing a storage device is nothing to them, the data just auto-replicates to other devices.

      • Re: (Score:3)

        by tnk1 ( 899206 )

        Agreed. Since they are required to replicate data to ensure high availability anyway, the major cost is not the quality of the medium, but rather the costs of raw storage. For quality considerations to overcome capacity, they would need a medium that is utterly reliable, which is a very, very hard thing to prove. That means that dropping quality by some low percentage, if it reduces the price of storage appreciably, is an acceptable trade off.

        As an individual, we do not expect to buy a lot of storage, an

        • Re: (Score:2)

          by PCM2 ( 4486 )

          For many, if not most individuals, quality would be priceless because we aren't going to necessarily want to create a RAID array of our disks.

          Not this individual. My use case is different than yours, I guess. The amount of data I actually generate myself is miniscule. If I need big drives at all, it's to fill them up with media of various kinds. Since almost all of it would be trivial to re-acquire (re-rip from the original media, etc) I'd hardly shed any tears if my one, non-redundant drive went south. Storage, for me, simply isn't "mission critical."

          • I'm with you on this one. My home server has around 8TB of data on it and out of that probably 7.5TB is videos/music/games that I can redownload or rerip as necessary if I lose them.

            I use a manual system, personally. /home is on a RAID1. /mnt/media is a JBOD pool. Everything easily replaced goes in /mnt/media, everything important goes in /home/$user which is also backed up. /mnt/media just gets a file listing taken every night so if it blows up I know what I lost.

            • Re:Reliability? (Score:5, Insightful)

              by harrkev ( 623093 ) <{moc.liamg} {ta} {noslerrah.nivek}> on Wednesday March 09, 2016 @07:28PM (#51669333) Homepage

              My home server has around 8TB of data on it and out of that probably 7.5TB is videos/music/games that I can redownload or rerip as necessary if I lose them.

              So, the time required to feed optical discs into a computer and/or browse web pages to get downloads is worthless? Some of us value our time, and don't seem to have enough of it.

              • So, the time required to feed optical discs into a computer and/or browse web pages to get downloads is worthless? Some of us value our time, and don't seem to have enough of it.

                A valid point, but I have automation already set up to handle things for me. Sonarr handles my TV library, Couch Potato does the same for movies. As long as I don't lose their databases (which are small and easy to back up) I can restore my TV and movie collections basically at the push of a button. Those programs will then go out and light up my internet connection for a few days and the missing parts will (mostly) reappear. My music collection is synced to Google. The majority of the rest is cached S

    • hard drives have never been very reliable either and people still bought them

      • Re: (Score:1)

        by Tablizer ( 95088 )

        It's partly because they never had a choice, or didn't understand how unreliable typical drives are/were.

        Selling a more reliable drive at a bit more cost will not be given much attention UNLESS one first knows how unreliable typical drives are. Most consumers don't know.

        Note that I found RAID to often not be helpful, because if the OS gets hosed, which is common with Windows, the RAID won't work right either. Perhaps with enough expertise on configuring one can avoid that, but that's beyond the consumer lev

    • So you prefer SSD than? SSD reliability already far outstrips spinning disks, so you should be happy with the move to solid state.

    • Why, this isn't 2010.

    • If you believe the hype, 3D Xpoint nvram will be more reliable than either NAND or spinning rust.
      And (again, if you believe the hype) it will continue to get cheaper and cheaper, while NAND won't.

    • I'll generally take reliability over volume. I wish they'd work on that more.

      Not just reliability, but archival (write-once) media that are reliable for very long periods of time.

    • Re: (Score:2)

      by delt0r ( 999393 )
      Fork out for the Enterprise drives. Then you will see what it costs. I infact do fork out for enterprise drives for my backups. I have 2x4T and 2x2T dirves. they are quite a bit more expensive.

    • Re: (Score:2)

      by mcrbids ( 148650 )

      If you value reliability, you want ZFS and RAIDZ.

      • I use btrfs. It's still not faced the sort of testing that enterprise users would want, but I've put it through some pretty horrific abuses and it's came out almost undamaged, including a really awful hard drive that was randomly and silently dropping writes when under heavy load. Thank you, Seagate.

      • have you seen an enterprise setup built in the last 5 years that used raidz instead of mirrors?

    • Reliability on HDDs is tricky! Can you rely on an HDD or SSD like you rely your fridge or the toaster? They can live from 20 years to 1 month! It's electronics! An when they are not, they are mechanical. Even worse! Consider your phone. Do you know many phone's memories that fail? Have you seen many of them? No. So NAND is pretty much reliable. BUT even then they break sometimes! RAID solutions and backing up will always be no 1 priority for safe data storage.

    • Redundancy is better than reliability, and low cost-per GB means redundancy is easily affordable. Because no matter how much you spend on a drive, you're never going to hit 100% reliability, which means that you still have to do everything you're going right now in order to protect your data, with multiple levels of backup plus RAID if you need high availability. Cheap per-GB costs means you can build in all the reliability you want into a system yourself, and it's probably going to ultimately be cheaper

    • I'll generally take reliability over volume. I wish they'd work on that more.

      Why? As someone who has gone through some 30 HDDs over the year who's never lost any data in failures I have to ask: what are you doing wrong?

  • what is the point of streaming (Score:4, Interesting)

    by known_coward_69 ( 4151743 ) on Wednesday March 09, 2016 @02:27PM (#51667537)
    if i can simply carry around enough movies and music and TV to last me a few months to a year?

    • Re: (Score:2)

      by Jherico ( 39763 )
      No matter how great your local storage is, you can't carry *all the media*, meaning that at some point you have to pick the subset of media you want and transfer it over the network. There's nothing wrong with that, but I suspect that for the vast majority of consumers being able to browse and choose content at the time of watching will always be more convenient than trying to anticipate everything you're going to want to watch.

      • No matter how great your local storage is, you can't carry *all the media*

        Maybe not, but eventually, you could carry all the media you own.
        I already do this for music.
        If Moore's law continues to hold, in 20 years I'll be able to do the same for Movies, TV, and Video.

        I'll still download new stuff.
        If you want to call playing something as it's downloading "streaming" then I suppose I'll do that too.

    • Re: (Score:1)

      by Anonymous Coward

      What, you don't value being constantly tethered to an unreliable or unportable internet connection while your viewing habits are sold to the highest bidder? You don't like your 1950's communications platform being used as a square-peg round-hole solution to transfer large sections of data? You sir are living in the past, and must secretly be a luddite.

    • Or, at least, that will be the effect once all of the legal/available methods of ripping your own media are gone.

    • why stop there? (Score:4, Funny)

      by Comboman ( 895500 ) on Wednesday March 09, 2016 @03:08PM (#51667795)
      I'm just going to download and store a local copy of the internet. Then I can cancel my ISP service.

  • War of the marketing material... (Score:4, Insightful)

    by Junta ( 36770 ) on Wednesday March 09, 2016 @02:33PM (#51667567)

    HDD company says HDDs are going to continue to be the best price/capacity for a while, and it's going to be awesome, traditional SSD company says that's bollocks and HDDs will be completely obviated in the same timeframe, and Intel/Micron say 'screw nand, we got something better'

    It's great to see real advances in technology (unlike the various BS 'revolutions' that are commonplace in pure software), but it would be nice if once in a while a marketing person said something straightforward and honest.

    • Re: (Score:2)

      by Jherico ( 39763 )

      SSD company says ... HDDs will be completely obviated in the same timeframe

      No one said that. They said that the price of NAND based storage will drop to be competitive with HDDs, but I suspect that there's an unspoken assumption there of HDDs staying static... in other words, NAND storage in a few years will be equivalent to the price per GB of HDDs now.

      Even if NAND storage reaches price parity with HDDs, there will probably remain markets for both, since they have different performance behavior for the lifetime of the device, sine NAND cells will die over time. On the other

      • "Very soon flash will become cheaper than rotating media." In which case you don't buy rotating media anymore. SSD commands a premium because it's got advantages over HDD, so much so that many modern data center SAN racks are going all-SSD despite it costing 4 times as much.

        • Re: (Score:3)

          by mikael ( 484 )

          How long before they stick a couple of embedded CPU's or GPU's along with those SSD's?

          • How long before they stick a couple of embedded CPU's or GPU's along with those SSD's?

            Hah! Computer on a disk. It will take "upgrading my computer's hard drive" to a whole new level.

            • Re: (Score:2)

              by mikael ( 484 )

              Hard disk drives already have CPU microcontrollers to handle the cache management algorithms. Some companies actually tried putting some arithmetic logic on their VRAM chips so that pixel operations could be done without fetching across the data bus.

        • Re: (Score:3)

          by swb ( 14022 )

          I think they pay 4x because the value is worth more than 4x.

          You get a dramatic performance improvement which in itself provides efficiencies in service delivery, reduced power consumption. And the performance benefit itself is an exponential improvement, not just an incremental one.

          • This is why I never use the term "value" in economics. I've written full macroeconomic theories and a decent amount of market economics without talking about anything called "value".

            Of course, theories of value are so broken they eventually concluded that things have value for magical reasons.

    • There will be a place for HDDs, with their price advantage... but already, SSDs are storing stuff more densely than HDDS can. HDDs that use SMR or HAMR will be useful, either as a lower tier for storage, or be used for home NAS servers, just because they are relatively inexpensive, and at the low price point, a 3 TB WD Red HDD for a C-note is a lot cheaper than a 3TB SSD for that use. No, you won't want to run iSCSI from that array, but for stashing files and having some redundancy, it is good enough.

    • Re: (Score:2)

      by Kjella ( 173770 )

      Well the way I see it the main issue for HDDs is that they don't scale down well. I just looked at the prices now and the cheapest are around 500GB, but you can't get a 250GB drive for half or 125GB drive for a quarter of that but you can get a 128GB SSD for the same. And for a lot of people who are mostly streaming or working in business and not heavy into home video that's plenty. A HDD is still unmatched if you need many TBs of capacity, but hey most people just don't. No, we're not average.

    • but it would be nice if once in a while a marketing person said something straightforward and honest.

      It is all honest. From the point of view of each company they are putting the R&D into doing exactly what the marketing people are currently saying.

      The only real question is who's R&D will bare the fruits first:
      1. Will HDD vendors achieve the next boost in storage first.
      2. Will SDD vendors get 100 layer 3D NAND working first.
      3. Will Intel get their new memory format out first.

      No one is being dishonest.

  • 640K (Score:2)

    by sinij ( 911942 )
    Do we need all this storage capacity? No, this is serious question. We already have NSA admitting that all the data they collected is beyond their ability to analyze. I can assume Google and the likes are largely in the same boat.

    That is, we can produce a lot of data, but what if is it all GIGO?

    • I'm guessing they're assuming that the processing power of tomorrow will make short work of the data they're storing today.

    • If you have all the garbage it's no longer garbage (Score:5, Interesting)

      by Overzeetop ( 214511 ) on Wednesday March 09, 2016 @03:04PM (#51667755) Journal

      Garbage is actually filled with valuable substances. GIGO is more applicable to older, discrete entry systems which have specific analysis paths.

      Consider real garbage in your trash, or in transfer station, or in a landfill. It generally has no value - it's garbage. But at a larger and larger scale it begins to have, statsitically, more valuable material in it. Now consider minerals trapped in the earth. We regularly process millions of metric tons of earth to refine and process into the elements we need.

      A truck full of garbage is a smelly mess. An earth full of garbage is a resource which can be mined for nearly anything you need. That's why all the trash we put in Google has greater value as the total amount of garbage they collect increases. It's still garbage...but at that scale it can be refined into money.

    • Re:640K (Score:4, Insightful)

      by spire3661 ( 1038968 ) on Wednesday March 09, 2016 @03:47PM (#51668021) Journal
      Absolutely yes. I could harness a 1000 GHZ CPU, and 10,000 TB of storage TODAY no problem. IT would vastly increase my output. Yes, we need more, and that need is not going to abate for at least another 30 years without a quantum jump in storage tech. 4k/60 FPS video is BIG, imagine what 4K 360-degree VR video will take up, and then 8K. Its an unending thirst that will take us the better part of a century to quench.

    • This is, actually, the key to fighting constant monitoring by the NSA and other three-letter agencies, I believe: generate a lot of spurious data, too much for them to store, much less analyze.

      If I kept sending email of 10 MB files (I know, that's small nowadays) which were randomly generated and had no meaning , and then erased them once they reached the other side (e.g. maybe a different email account of mine), then that's no skin off my back since I know I shouldn't care. Any monitoring agencies, though

      • Re: (Score:2)

        by KGIII ( 973947 )

        Hmm... I like this idea.

        10 MB file created, encrypted, mailed, sent to /dev/null.
        Recipient encrypts it again, mails to next person, sent to /dev/null.
        Daisy chain it so more people can participate.
        Every third (or so) time it's encrypted, the next one gets deleted and a new 10 MB random file is generated.
        Have it go in circles BUT also have it 'cross-fire' so that it's sending data to random participants.
        Get more than 1 file moving around.
        Make it tit-easy and let people join/participate.
        It can all be done over

  • Will it ever get cheap again? (Score:5, Insightful)

    by grumbel ( 592662 ) <grumbel+slashdot@gmail.com> on Wednesday March 09, 2016 @03:23PM (#51667891) Homepage

    Back in 2011 I could get a 1.5TB drive for 45€, now five years later the best I can get is 3TB for 90€. Double the storage for double the price. If I just want to spend 50€ I only get 1TB. It's nice that we now have 6TB and 8TB drives, but they aren't cheap and so far haven't really lowered the price of the smaller drives and given how long this has already taken I am not even sure if HDDs will ever get cheap again before SSDs will take that space.

    • Re: (Score:2)

      by djbckr ( 673156 )
      Just last week bought a 5TB drive for $130 (USD). Not long ago, a 1TB drive was the same price. I'm not sure what you are saying - it seems pretty cheap to me.
    • Price per GB has resumed dropping [cbsistatic.com] since the effect of the Thailand flooding and HDD consolidation in 2011-2012. Quite frankly, that price adjustment upwards was needed, as the HDD industry had some of the slimmest margins in the electronics industry (around 1%-3%, vs 5%-10% for electronics overall). Slim margins = less money companies are able to devote to R&D = slower rate of capacity improvement.

      And don't fret about the rate of price drops slowing down since 2009 in the graph. The y-axis on that

      • Re: (Score:2)

        by grumbel ( 592662 )

        Price per GB has resumed dropping [cbsistatic.com] since the effect of the Thailand flooding and HDD consolidation in 2011-2012.

        The most frustrating part isn't so much the price per gigabyte, as that is back to pre-flood levels, but that you only get that good GB/$ rate in the $100 price range, while you used to get that in the $50 range. If you only need 1 or 2TB you are still paying quite a bit more then five years ago, it's only with 3TB-8TB drives that you get a slightly better rate then pre-flood.

    • Back in 2011 I could get a 1.5TB drive for 45€, now five years later the best I can get is 3TB for 90€. Double the storage for double the price.

      Just as well there was no major natural disaster skewing your statistics in that time frame ....

  • "Very soon flash will be cheaper than rotating media," said Siva Sivaram, executive vice president of memory at SanDisk.

    I can hardly wait. Just as core replaced drum and ram replaced drum I am excited to get rid of spinning platters. The day I can get a 3TB SSD for around $100 will be a great day.
    Of course it might be possible to get a 40TB HD for that price by then...

    • Perhaps misplaced ... but I remember the time when we had races for the smallest disk size (as in form factor). Something like 1" disks. The first iPods had super small disks, and basically read one track and stopped the disk then. Fotos of such disks usually where comparing it with a small coin, "a dime" or "one Mark" (that was pre Euro). That was not that long ago.

      Steve would say: is that not incredible? No, I think his term was "amazing" ... anyway.

    • 40TB HDD? It's apt to fail before you read all the data off of it... Ok, I'm only slightly joking there. The IOPS will still be piss poor on spinning media, so it will only get good throughput with large files, current benchmarks on 8TB drives show around a 150MBps read and write, if that holds the same, it will take over 70 hours to transfer all 40TB, in the best case scenarios.

      • Re: (Score:2)

        by LWATCDR ( 28044 )

        Ok then an 8TB 2.5" HDD for $90.
        Even at today's prices I see HDDs as more of archive storage. You put large amounts of data like media on them. Also if IOPS count then you will just keep adding spindles to an array.

  • that will ensure data storage will not only keep up with but exceed demand

    If that were to happen, the bottom would drop out of the market; they're obviously not going to produce so much that they shoot themselves in the foot.

  • I was talking with someone that works with storage systems, he mentioned other technologies to replace HD and their looking into massive petabytes (sp?) of storage capacity. I asked with so much data, when does anyone have enough time to look back at it. His reply, "they will not."

    OK some data will be examined, I'm thinking from my perspective. I have about three major email accounts. One has 360 messages, I recently trashed 800 but few years ago I archived 2000 messages (but haven't looked at them since)

  • Data storage capacities are going up. News at 11.

  • Seagate claimed 60TB by 2016 (Score:4, Interesting)

    by Anonymous Coward on Wednesday March 09, 2016 @07:48PM (#51669433)

    This post on Slashdot reminds me of previous post on Slashdot [slashdot.org] from 2012 about 60TB 3.5 inch and 10TB 2.5 inch drives by 2016. Again it was based on Seagate's 2006 patent on heat assisted magnetic recording. I think it is more realistic at his point to expect LTO tape to provide 220TB storage [slashdot.org] than Seagate making good on it's dream of HAMR drives. This latest claim should be an indication of how much trouble Seagate is having with their heat-assisted technology. They are essentially giving themselves a 4 year extension to achieve only a third of the results that they previously claimed.

    • It goes back even farther than that!

      Here's an article from 2010 reporting Seagate promising 100TB HAMR hard drives:
      http://www.myce.com/news/seaga... [myce.com]

      Here's an article from 2006 reporting Seagate promising HAMR hard drives in "a few years":
      http://webcache.googleusercont... [googleusercontent.com]

      Also I'm puzzled by the claims about hundred layer 3D NAND chips. I can see how a hundred-layer chip would increase density and therefore could reduce access latency, but I don't see how it could significantly reduce cost-per-bit. Sure, th

  • The sad aspect of this story is that developers grossly underestimate our demand for 'selfies'.

Slashdot Top Deals

The computer is to the information industry roughly what the central power station is to the electrical industry. -- Peter Drucker

Close