Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Data Storage

Linux Support for Hybrid Hard Drives? 86

Posted by Cliff from the better-storage-tech dept.
christoofar asks: "HHDD (Hybrid Hard Disk Drive) technology has been receiving some buzz lately. The concept is not new, but Samsung has been working on a consumer version of HHDD that everyone can use. HHDDs are disk drives that carry onboard RAM (in this case, NAND flash) which is non-volatile and offers to speed boot times and writes to the disk. This carries enormous benefit to laptop users who need to keep their disk activity to a minimum in order to preserve battery life. Given that Microsoft is adding support for Hybrid Hard Drives in their upcoming Windows Vista release, what efforts are being undertaken in the Linux realm to use this new storage technology?"
This discussion has been archived. No new comments can be posted.

Linux Support for Hybrid Hard Drives? More

Linux Support for Hybrid Hard Drives?

Comments Filter:

  • It should just work (Score:5, Insightful)

    by saskboy ( 600063 ) on Saturday March 04, 2006 @12:43AM (#14848390) Homepage Journal
    Why aren't the drives designed in a way that the drive technology worries about the details of retrieving and writing files, and sends them the way USB, IDE, SATA, or SCSI drives already do? Why would these drives need special drivers?

    • Re:It should just work (Score:5, Insightful)

      by mrchaotica ( 681592 ) on Saturday March 04, 2006 @12:48AM (#14848416)
      I would guess that it's because having a software-based queuing and caching system is more flexible. For example, if you want to optimize operating system startup, then you need some way to specify that the operating system is what you want in the flash memory, instead of some other data.

      Or if you've got a laptop, you maybe want to cache writes to the flash if you're using the battery, but write directly to the disk if you're hooked up to the mains. Since the operating system knows about this but the drive doesn't, the operating system needs to control the drive.
      • For example, if you want to optimize operating system startup, then you need some way to specify that the operating system is what you want in the flash memory, instead of some other data.

        Still sounds pretty much like disk caching. During bootup, the system's going to be asking for the OS anyway*, so if you've got a big enough amount of RAM to load it into, you can just do that, can't you?

        *I realize that this does not explain current boot times. Windows, Mac OS X et cetera are obviously spending some amou
        • It is not disk caching - the OS requests that certain files be kept in that memory at all times. This memory would not be changed until it is told to change. This is so that files that have to be read, either to boot up or continue running the os are easily accessed, and quickly accessed. The drive would do disk caching as well, seperately.

      • The simple solution is to just make a specific part of the hybrid drive be RAM based. It would default to being sectors starting at sector 0. Then the method to change it would be to partition the drive with a special partition code for a partition that will change the sector locations to be in RAM to what they are for that partition, up to the end of the partition or the RAM capacity, whichever is first. Then RE-partition it back to what the OS needs, which will not make changes on the hybrid drive (it

      • I would guess that it's because having a software-based queuing and caching system is more flexible. For example, if you want to optimize operating system startup, then you need some way to specify that the operating system is what you want in the flash memory, instead of some other data.

        You could easily have a drive that appears to the OS as two drives; one "drive" would always write to NAND, while the other would be the real disk, with any free space on the NAND part being used as the persistent cache. It

    • Re:It should just work (Score:4, Interesting)

      by AKAImBatman ( 238306 ) * <akaimbatman@gmaYEATSil.com minus poet> on Saturday March 04, 2006 @01:49AM (#14848624) Homepage Journal
      I'm guessing that it's not so much that it can't work that way, as it is that it would kill performance.

      Today's operating systems try to do exactly what this drive is doing by pumping all reads and writes through the paging system. The problem is that the OS can't take a full performance boost from this or data will be lost in a crash or power loss. You may remember that EXT2FS could easily lose all the data you'd recently written if it wasn't unmounted properly. (It ticked me off when I tried to move RedHat 5.2 packages to a special partition, then found out that they had all disappeared on reboot because I hadn't explicitly unmounted the disk. *grumble*)

      Since this technology has little to no chance of losing data in a power failure, the OS can be modified to write the blocks immediately. This could easily result in a performance increase of 2 to 3 times what you normally see today. The improvements in writing meta-data alone could easily provide this increase.
      • ...then found out that they had all disappeared on reboot because I hadn't explicitly unmounted the disk. *grumble*

        The SYNC command forces the system to flush all buffers to disk. Once it returns, you know that all currently cached data has been at least scheduled to be written to disk. (current documentation claims that it doesn't return until the data has actually been written). Once data goes to the HD, it may still take a second or more for it to clear out of the internal drive cache.

        for older kern

        • On W2K3 there is an option for enabling proper syncing (if your hard disk is enterprise suitable). It is enabled by default and the performance drop from W2K to W2K3 is shocking, until you disable it. Which means: don't use W2K for anything important - upgrade your servers to W2K3, take the performance hit, but keep your data safe. Microsoft ought to give you a refund on your W2K Server purchase as it is not fit for the purpose for which it is sold.

          Having flash in front of the platters will allow this same
          • If you do it properly, you shouldn't have that much of a performance hit. Linux also has subroutines for explicitly flushing data on a specifid File Descriptor to disk (for situations where the order is important, like databases).

            Between the horrific speed differences between RAM and disk and the importance (nay -- criticality) of permanent storage, disk caching is an especially touchy space to work the tradeoffs.

            If you don't cache at all, then you get horrific speed hits. If you cache too eagerly, th

            • I saw it under email loads, so it could well only be syncing for important data (ie, ensuring the email is in the queue folder an on disk before telling the SMTP client that it has queued it).
              • That barely makes sense. You should be able to write to disk far faster than you read network data -- unless you're reading from a RAIDed mail server over a gigabit LAN.
                • unless you're reading from a RAIDed mail server over a gigabit LAN.

                  Which is how most businesses are set up these days...

                • The HD is ofcourse used for other things as well... it has to get done writing something and then seek.. yada yada...

                  I'd think that most email systems are multi-threaded so that a mail message waiting to be put on disk wouldn't block another one from comming... /. had an article on why vista won't suck.. and the ability to use a usb thumb drive for system purposes was mentioned.

                  I would think that the flash would be used for moderately frequently read items... (because superfrequently read stuff would be in

      • It would not kill performance at all. It might fail to improve it if the OS is paying no attention to what is going on with the drive. If the OS queues the writes to the RAM-ified sectors, then you get the old performanc as if this was not a hybrid drive. Reads can't get better if the page is already in system RAM anyway. But I could argue that an OS that queues writes for a reason other than the drive being busy or powered down is doing queueing badly, anyway. If the drive is idle and a process writes

    • There are a couple good replies to you already, but I'll add one more place the OS might be able to improve its performance if it knew about the hybridness of the disk.

      I have a hunch (this is unsupported by data; you'd want to do some profiling before making a decision to do this) that you would want swap reads and writes to go right to the platter rather than flash. One reason is that flash does "wear out" after a while, and moving swap so that it doesn't use the flash would likely increase its lifetime su
      • This would only have an effect if your swap gets used. During regular usage patterns ( so most of the time), my swap doesn't get used, with 512M of RAM.
        • I regularily see the amount of memory allocated to Firefox alone over 500 MB...

          Currently Windows reports 920 MB pagefile usage. I think that's probably total memory use, not the amount that's swapped out, but I can almost guarantee that there's something on disk. Windows seems to be very bad with swapping; before I added another GB of RAM (for a total of 1.5) it paged fairly frequently. After coming back from work and unlocking it, it would take probably 10 seconds to page everything in from disk.

          A laptop (
          • just one reason why I stick to Linux :) I guess it could be argued that the parent poster was thinking about the performance differences and what not if Windows (and not Linux) moved the swap to the platter, but that's irrelevant anyway, because we have no say in what MS does with the technology, and the article's about Linux. Thing is, Linux will probably have to be developed on the technology only after it is released. Unless Samsung is collaborating with someone, I assume they wouldn't have access to
    • If access to drive contents are bound to runtime drivers then that content can be made bound to DRM. Am I the only one noticing a nice little trend happening...

      --Neth
  • I'm sure they will be support for it by the time it is really *needed*. If it is an open and available spec, support could be coding before these things start becoming popular. Otherwise, as the linux community doesn't enjoy the advantages of MS, it will be reverse engineered after release as someone's need dictates. Honestly, how is this a useful question? These drives seem to have limited use outside of laptops and there shouldn't *Need* to be special support, as other posters have said. It should adh
    • I'd like to see linux have support for (and have hardware makers create) what SUN used to call NFS Accelerators. Basically add-on NV (or battery backed) memory sufficent to transparently cache write information. On my laptop I'd put it in my ExpressCard(tm) slot.

      Then again I have been considering using an SD type card to contain the journal for my ext3 file system.

      Actually, that combination; an ext3 file system in full data write-back journal mode, a solid-state flash device for that journal, and a "large
      • Is flash memory actually faster than a decent hard disk? I assume the seek time is better, but my recollection is that the transfer rates aren't anything to write home about.

        ian

        • I was thinking more about avoiding drive spin-up. I am not so impatient for "my OS to boot" and more concerned about lengthening the drive lifespan by having it be (e.g. STAY) spun-down for longer. If you had a two-gig SD card and could get the ext3 journal flush time to be "forever if the drive is stopped and there is at least 10% of the log space left" [and if you have enough main memory that you don't need to page/swap] then you could run the laptop virtually all day with the drive spun down if you wer
  • If they don't even the writes across the media, bits will wear out faster at say the beginning of the memory. Go through a few million writes in a hurry. No thanks for me! Or am I wrong about that?

  • What to put in there? (Score:5, Interesting)

    by aspoon ( 794081 ) on Saturday March 04, 2006 @02:18AM (#14848695)
    The question is not whether Linux will support the drive -- someone will eventually write one. The question is what files to put in there to make it boot faster. Perhaps should be done is to put the entire swap partition in there, and put those "hibernation" files in the swap. So it would resume fast, and would also make normal operations faster with the faster swap.
    • would also make normal operations faster with the faster swap.

      If you're swapping while running normal applications, something is wrong with your system. If an increase in swap speed is immediately noticeable as better system performance, you need more memory. I do like the idea of suspending to flash though.

      For the pendantic: sure, OCCASINALLY you might have to dip into swap. And sure, OCCASIONALLY you'll have to pull in some really stale pages. But these are supposed to be rare events. But if these ar

      • If you're swapping while running normal applications, something is wrong with your system.

        Either that, or you're just dirt poor and trying to surf the net on 5-year-old equipment.

      • how about 'boot state' recorder to flash.. as in, unless you install software or add hardware, your system memory at bootstate can be written to a flashchip the same size as your ram, and you can boot, or reboot, as fast as you can re-dump the chip into ram.

        add new hardware? fine, as soon as yer system is booted for real, save it to the bootchip.

      • If you're swapping while running normal applications, something is wrong with your system. If an increase in swap speed is immediately noticeable as better system performance, you need more memory.

        And of course, memory is free, so we'll just all run out and solve that problem right away. :)
    • At the time we loose our cycles on slashdot, the MTD project [infradead.org] probably already has support for these devices.

      Oh, and it's a very bad idea to put swap in a Flash device, unless you want to wear out the sectors pretty quickly and render the chip useless.

      • It sounds like they want to mostly cache files that get read every boot-up, which would tend to not get changed much. This would avoid the however-many-writes limitation of flash.
        • That is exactly how this work. The reason it's so useful for a Windows box arises from one of the ways in which Windows makes interesting use of files on disk.

          When a DLL is loaded into memory from disk, it's only relocated if there's already another DLL loaded in the same slot in virtual memory. If there's no conflict with the DLL, it's not relocated; code pages are read directly from disk and never written back. Instead, when they're swapped out (it the machine doesn't have enough RAM), the page is simpl

  • Why wait? Hack one up yourself! (Score:5, Interesting)

    by Myself ( 57572 ) on Saturday March 04, 2006 @02:36AM (#14848741) Journal
    You shouldn't have to wait for the drives to come out. Laptop drive controllers can address a master and a slave, they just don't have a slave drive connected in normal usage. I was just looking for an adapter [logicsupply.com] to let me put a pair of CF cards in place of a 2.5" hard drive. (I can only find the single-card version in a 2.5" form factor, all the dual-card ones are for 3.5" mounting.) I figure, put a solid-state card in one slot and a microdrive in the other, and I've got a hybrid-drive laptop, right?

    I just got rid of an old toughbook cf-25 [wikispaces.com] that would've been perfect for this, as the drive mounting is gel and would easily accomodate an oddly shaped adapter instead of a regular drive. Or for the truly insane, a CF card piggybacked on a regular 2.5" drive! All I need is the ability to home-brew those little flex cables, and I'd be in business.
    • It eats a PCMCIA slot, but I use a CF card in a PCMCIA adapter (cheap at about 8 dollars) in my CF-47 whose hard disk died. I boot from live CD.
      If you hard disk is working, then you have two hds.
      Partitioning CF cards is also useful. On my Portege I run a Damn Small Linux frugal install on the first partition and storage in the second.

  • Nice to see (Score:3, Insightful)

    by Telvin_3d ( 855514 ) on Saturday March 04, 2006 @02:37AM (#14848744)
    With all of the improvements in RAM, on-die cache and processor speed, the bottleneck in performence is HD speed. Anything that helps boot that is a welcome improvement.
  • Just HD partitioning should be enough really.

    There are two ways that RAM can speed up a HD: as cache and as mirror. According to what I've read, it serves both purposes but mostly as mirror. And since mirror will represent a specific, unmoving window of the drive, a partition scheme that fits the situation is all that's really needed to best take advantage of it. Now deciding WHAT works best in that area of the drive is a matter for the really smart people. I'm not one of them...

    But that said, I don't k
    • Why not just do that now by adding the memory to your system(if it can handle it)? It shouldn't be to hard to setup a shell script that will load a portion of your hard drive into memory. I wonder if you could setup unionfs to write the changes to the hard drive? You could probably setup proccess acounting or something to track program usage and load only the most commonly used programs. But, to get to the point, the value of the flash ram is that you can accelerate boot time by having the data already in
    • Windows doesn't open executables for writing when you run them. In fact, when you run a program, it prevents writing entirely. Windows loads executables through the memory-mapped file interface. It does this so it can share code pages between different processes, and get away with paging out part of a program without actually writing to the page file first.

      Windows programs that modify their own EXE have to do some crazy tricks. While you cannot modify or delete a running program, you *can* rename or mov

  • Write caching in flash... (Score:2, Interesting)

    by rew ( 6140 )
    I'm not sure if write caching in the flash chips is a good idea...

    Flash chips write at around 1Mb per second. Tops. Modern harddrives write around 50Mb per second. You'd need quite a lot of flash chips in parallel to top that.

    Of course, you can cache acouple of writes on the laptop drive to prevent having to spin up the disk. But it is not going to be a speed issue.
    • For many application, like the windows swap file, that read and write small random block, flash disks offer huge performance improvment, because their seek times are much better then regular hard drives.

      The sustained write speed does not really matter for the small block operations.

    • Re:Write caching in flash... (Score:5, Informative)

      by greg1104 ( 461138 ) <gsmith@gregsmith.com> on Saturday March 04, 2006 @06:45AM (#14849217) Homepage
      Samsung's OneNAND design uses some interesting implementation details to run faster than is normally expected with flash. Their specs at http://www.samsung.com/Products/Semiconductor/OneN AND/index.htm [samsung.com] suggest writes at 9.3MB/s and reads at 108MB/s; plenty fast for many applications even without running multiples in parallel. It's certainly much slower than writing directly to the drive if the drive is active. However, if the drive isn't spun up at the moment and the amount of data to be written fits in the NAND flash cache, I could see this being a net performance boost over the spinup/seek time combination of the hard drive.

      They also spec 100,000 erase cycles before it's worn out. As was noted by an underrated poster the last time [slashdot.org] this came up, intelligent flash controller designs like this can cope with bad bits and assuring level usage of the memory much better than what you normally see in random hunk of flash.
      An analysis at http://www.sudhian.com/printdocs.cfm?aid=686 [sudhian.com] suggests 33 years of usage for a typical worker. When you run the numbers it doesn't sound that difficult to create a design that would likely outlast the mechanical parts of a standard hard drive.
      • Also consider that flash uses considerably less power to operate than a hard drive, meaning that your laptop could survive a lot longer without having to spin up the mechanical drive.
    • 50Mb/s yes, but only sequentially. As soon as you're seeking you get a few ms of delay on a HDD, but a few us or ns on anything solid state. So basically if you're encoding video a HDD or RAID array would work great, but if you're reading/writing lots of little files (say, in swap, or in a DB server) flash would beat it hands down.

  • Other good uses for this (Score:4, Interesting)

    by nial-in-a-box ( 588883 ) on Saturday March 04, 2006 @03:52AM (#14848906) Homepage
    I think the real demand for flash-cache hard drives should be in the area of data loss prevention. Anyone who has any doubts about the fact that laptop hard drives are still way too fragile for the job needs to work at a help desk for a week. Any way to keep those drives spun down as much as possible should be seen as a good idea, since speed does not really matter much if you lose your data and/or need a new drive every three months simply because you like to move your laptop while it is running.

    I know this just adds another point of failure to the mix with the addition of flash memory. However, with the apparent improvements in the quality of flash memory, I would expect one of these drives to outlast a current laptop drive by at least 50%. (Note: this is just idle speculation, but I don't know of any solid real-world statistics on laptop hard drive lifespan. I'm guessing laptop makers don't want us to know, either.)

    I think that the point of drivers is, therefore, valid. There needs to be some sort of intelligence behind this system to allow frequently needed files to be held in cache in order for this to work effectively. Sure, you could build a drive that could try this on its own, but odds are you would totally throw out any performance or power advantage by doing so.

  • Another use? (Score:3, Insightful)

    by agentcdog ( 885108 ) on Saturday March 04, 2006 @04:12AM (#14848940)
    So I see a completely different picture here. There are a lot of files on the HDD that are never rewritten. System files, etc. This is where having hybrid drives really helps. Put the main boot files, executables and libraries which are accessed when staring things up in there. So no fast saves, but no load time.
    • This has nothing to do about speed, you will not gain speed by using the flash, it's just a way to write data to the harddrive without spinning up. This will earn you
      1. power consumption (alot)
      2. less wear on the HD

      nothing else.
  • Seeing as I can't buy one of these drives from my usual shopping websites, why would I care? Obviously the technology is very new. I would much rather let Samsung and Windows sort out all the defects at their cost and embarrassment than risk buying a kludgey piece of hardware for my linux computer.
  • Its not a hybrid drive, but the clever one I saw was a hard disk-backed 2 gig "solid state" drive. It was a board in a 1U rackmount box with 2 gigs of regular ram, a battery and a 2 gig hard disk.

    All reads and writes are to the ram. The hard disk isn't even running. When power is lost for more than 15 seconds, the hard disk spins up and the contents of the ram are written out to disk. When power comes back on, the drive spins up, the contents of the disk are loaded back in to ram and operation resumes.

  • It looks like the answer is "yes" (Score:3, Informative)

    by NetRAVEN5000 ( 905777 ) on Saturday March 04, 2006 @09:30PM (#14851881) Homepage
    I've got the 2.6.15.4 kernel and there's a spot to enable support for "Memory Technology Devices".

    Here's the description:
    "Memory Technology Devices are flash, RAM and similar chips, often used for solid state file systems on embedded devices. This option will provide the generic support for MTD drivers to register themselves with the kernel and for potential users of MTD devices to enumerate the devices which are present and obtain a handle on them. It will also allow you to select individual drivers for particular hardware and users of MTD devices."

    It's got options for a whole bunch of things, such as "FTL (Flash Translation Layer)", "NFTL (NAND Flash Translation Layer)", and "INFTL (Inverse NAND Flash Translation Layer)".

    Given that it says the hard drives will use "a one-gigabit OneNAND flash chip" (according to the article), it sounds like it will work.

  • i use a 12 hp dot matrix and stone tablets [modified] ive had some of my bakups for 15+ years and no deterioratin equivalent to 100000rpm hdd read 35000 write
  • I would think the easiest way to accomplish the goal of HHDDs is to simply have 2 drives, one made of Flash RAM and one traditional.... You use the flash drive for your OS and programs, and then store all your data on the traditional drive.

Slashdot Top Deals

Without life, Biology itself would be impossible.

Close