Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
Data Storage Programming IT Technology

Revisiting the Five-Minute Rule 153

In 1987, a study published by Jim Gray and Gianfranco Putzolu evaluated the trade-offs between holding data in memory and storing it on a disk. Known widely as the "five-minute rule," their research was updated and expanded 10 years later. Now, as jamie points out, Communications of the ACM is running an article by Goetz Graefe with another decennial update, evaluating the rule using hardware and software typical of 2007, with an eye toward how flash memory will affect the situation. An excerpt from Graefe's conclusion: "The 20-year-old five-minute rule for RAM and disks still holds, but for ever-larger disk pages. Moreover, it should be augmented by two new five-minute rules: one for small pages moving between RAM and flash memory and one for large pages moving between flash memory and traditional disks. For small pages moving between RAM and disk, Gray and Putzolu were amazingly accurate in predicting a five-hour break-even point two decades into the future. Research into flash memory and its place in system architectures is urgent and important. Within a few years, flash memory will be used to fill the gap between traditional RAM and traditional disk drives in many operating systems, file systems, and database systems."
This discussion has been archived. No new comments can be posted.

Revisiting the Five-Minute Rule

Comments Filter:
  • for those wondering: (Score:5, Informative)

    by wjh31 ( 1372867 ) on Sunday July 05, 2009 @10:04AM (#28586385) Homepage
    http://en.wikipedia.org/wiki/Five-minute_rule [wikipedia.org]

    "The 5-minute random rule: cache randomly accessed disk pages that are re-used every 5 minutes."
    • by salahx ( 100975 ) on Sunday July 05, 2009 @10:07AM (#28586403)
      The more useful 5 second rule [wikipedia.org].
      • Nor the funkier 3 minute rule [youtube.com].

      • by johannesg ( 664142 ) on Sunday July 05, 2009 @01:58PM (#28587705)

        The more useful 5 second rule [wikipedia.org].

        That's just utterly disgusting. Do people in the US really believe that you can eat food that's fallen on the floor if you pick it up fast enough?

        • by SUB7IME ( 604466 ) on Sunday July 05, 2009 @02:30PM (#28587925)
          I know, right? What a disgusting waste of perfectly good food that has been on the ground for only 10 seconds!
        • by michaelhood ( 667393 ) on Sunday July 05, 2009 @02:39PM (#28587983)

          I'd rather be a disgusting American than a naive European with no sense of humor..

          • by squizzar ( 1031726 ) on Monday July 06, 2009 @05:08AM (#28592455)

            As a not so naive Brit, we have our much superior 3 second rule for the same thing.

            I'd like to know what the GP has on the floors in his house that is so toxic that the tiny amount that will rub off on food is detrimental for your health? How on earth do people survive in places without nice sealed floors and cleaning chemicals? You'd think we'd have evolved some method of protecting our bodies against stuff like that.

            • by lxs ( 131946 )

              Are those Imperial or metric seconds?

            • by Jamu ( 852752 )
              That's how long it takes the mouse to spot the potato chip, scurry across the floor, and then to sit on the potato chip and nibble away at the edges.
        • Re: (Score:3, Funny)

          by jweller ( 926629 )

          No silly, but if you catch it on the bounce, it's like it never happened

        • Do people in the US really believe that you can eat food that's fallen on the floor if you pick it up fast enough?

          Well, if you accept that my brother fits the definition of "person", and various snacks (pretzels, chips, hard candy (still wrapped)) as "food", then some people in the US don't restrict themselves to 5 seco^Wminu^W^da^Wmonths.

          Though I am sure the attitude is not limited to the US.

        • by jo42 ( 227475 )

          You need to watch the Mythbusters episode on the 5 second rule.

        • Re: (Score:3, Insightful)

          by Kaboom13 ( 235759 )

          Well the confusion is, unlike your dirt-floor huts full of your own feces, in real countries like America we have clean tile floors with miraculous inventions we call "mops".

        • by tomhudson ( 43916 ) <barbara DOT huds ... a-hudson DOT com> on Sunday July 05, 2009 @05:22PM (#28589083) Journal
          Carpets don't transfer as many bacteria [nytimes.com]

          Scientists have put the commonly-cited five-second rule to the test. They found that food that comes into contact with a tile or wood floor does pick up large amounts of bacteria. Food doesn't pick up many germs when it hits carpet, but it does pick up carpet fuzz.

          Since this is slashdot, I'd bet most will pick bacteria over carpet fuzz any day ... after all, if it doesn't look fuzzy ...

          or this ... [scienceline.org]

          many people believe that gastric acid enzymes found in the stomach are strong enough to destroy the "small, harmless" amount of bacteria that could gather on a piece of food in five seconds. But are these bacteria really harmless?

          In 2003, Jillian Clarke, then a high school senior, decided she wanted to find out. During an internship at the University of Illinois at Urbana-Champaign, she and a doctoral candidate, Meredith Agle, took swab samples from floors all over the campus, including labs, hallways, and bathrooms, and found that the amount of bacteria on the floors was very low. When she published her research, she concluded that if a piece of food falls on a relatively clean floor, the five-second rule is, in fact, applicable.

        • Re: (Score:3, Funny)

          Normal floors are probably safer than your hands. If it's in your own home and you just dropped it or something, I can't even really imagine why it would be mentally bothersome.

          It's not really an issue if you have a dog, though.

        • Re: (Score:2, Insightful)

          by jonadab ( 583620 )
          > Do people in the US really believe that you can eat food
          > that's fallen on the floor if you pick it up fast enough?

          Short answer: no, but yes.

          Long answer: There are two kinds of people. Some people won't eat food that has fallen off the plate onto the table, their lap, the chair, the floor, wherever. If it fell, it's "dirty", and they throw it away.

          There are also people who think this is silly. If there's no visible dirt on it, or if you can brush it off, hey, it's still food. Unless it fell in
      • The lesser known 5 minutes 5 second rule combines the two: It states that if the case is left off a desktop computer for more than 5 minutes and 5 seconds Pizza and coke will spontaneously migrate from a computer lab desk and contaminate your RAM, CPU and motherboard.

    • by Twinbee ( 767046 )

      Will the 5 minute rule increase to 10 or even 20 when RAM becomes mega-cheap, and 64 bit OSs take off to take advantage of >4GB memory addressing?

      • by Korin43 ( 881732 )
        I really wish programs would just leave everything in memory and let the operating system page it out as it needs to. I have 4 Gb of memory and even with 'preload' set to prefetch everything, I'm still only using 1 Gb right now (it was around 200 Mb before I installed preload). And Ubuntu doesn't start swapping until it needs to, so no swap used.
  • Flash memory? (Score:2, Insightful)

    by NitroWolf ( 72977 )

    I couldn't quite figure out if the article willfully ignored the advent of SSDs or was written before they were available and not updated to include them (but it appears the article was updated to include other current technology).

    Given the fact that SSDs are likely going to replace rotational media for most applications in the future, it makes this article basically meaningless, at least insofar as the fact that flash memory and the disk are/will be synonymous. As the article is basically predicated aroun

    • What article? (Score:3, Insightful)

      by argent ( 18001 )

      The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?

      • Re:What article? (Score:5, Insightful)

        by amorsen ( 7485 ) <benny+slashdot@amorsen.dk> on Sunday July 05, 2009 @10:51AM (#28586635)

        The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?

        The article treats flash as something you place in between hard drives and memory. This turned out not to happen (with a few exceptions). SSD's simply replace hard drives. Hybrid systems are rare, and it doesn't look like they will become more common -- either you can live with the slowness of hard drives, or you can't. The mainstream will switch to SSD's for everything except backup applications.

        There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.

        • The grandparent was talking about hybrids, but also something new. If and when MRAM becomes possible the technological hard drive wether spinning or flash is gone.

          Indeed I am waiting for a true hybrid system to be built. One that has the OS installed in read only flash and applications on a separate drive. you might ask why? but then stop to realize what would happen if viruses couldn't overwrite the system settings. that to clean up a virus all you had to do was to reboot.

          • I am waiting for a true hybrid system to be built. One that has the OS installed in read only flash and applications on a separate drive. you might ask why? but then stop to realize what would happen if viruses couldn't overwrite the system settings. that to clean up a virus all you had to do was to reboot.

            How would such a hybrid system correct a discovered defect in the operating system?

            • by vipw ( 228 )

              I propose a physical switch to toggle between read/write and read only. Even if it could be controlled remotely by a management server, it would provide a nice increase in security.

              • Re: (Score:3, Informative)

                by tepples ( 727027 )

                I propose a physical switch to toggle between read/write and read only.

                And have social engineers disguise malware as OS updates or dancing bunnies [msdn.com], prompting the home user who doesn't understand risks to flip the switch to see the dancing bunnies.

                • And have social engineers disguise malware as OS updates or dancing bunnies, prompting the home user who doesn't understand risks to flip the switch to see the dancing bunnies.

                  You can't make something 100% foolproof, the world will just invent a better fool. But you can make the lives of the non-fools better.

                • propose a physical switch to toggle between read/write and read only.

                  And have social engineers disguise malware as OS updates or dancing bunnies [msdn.com], prompting the home user who doesn't understand risks to flip the switch to see the dancing bunnies.

                  If I'm the admin of that system, I'd install a key switch and keep the key for myself.

                  Better yet, I would install a keypad that is synchronized to my RSA keyfob - that way I could give the user a one-time code over the phone if I needed the button pushe

                  • by tepples ( 727027 )
                    Recap: The article discusses the divide between flash and disk. Another user proposed an alternate divide that puts the operating system on a flash drive, write-protected by a physical switch between "allow updates" and "block 'updates' by malware", and the applications and data on disk, in a way reminiscent of TOS on Atari ST computers. This has its advantages, but I'm explaining how protection against malware isn't necessarily one of them.

                    If I'm the admin of that system, I'd install a key switch and keep the key for myself.

                    That might work for you, but it won't work for everyone. Should the

                    • Should the owner of a home PC be the PC's admin by default?

                      Not my father or sister. It would save me lots of headaches and desperate phone calls if I had that switch.

                      In fact, it might be worthwhile to offer this as a feature - software publishers could form a guild to guarantee software authenticity. Have an optional phone number you can call w/ the serial # of your keyswitch and the code of the sfw you're installing. They'd give you a code for the switch that only opened the areas of the disk and/or regi

                    • by tepples ( 727027 )

                      In fact, it might be worthwhile to offer this as a feature - software publishers could form a guild to guarantee software authenticity.

                      How would developers of free software [gnu.org] gain access to a publisher in the guild? And who can afford to make multiple international long-distance telephone calls, one to each publisher's headquarters?

            • by neomunk ( 913773 )

              Presumably by the tech flipping a hardware rw/ro switch on the drive after proper isolation conditions are met.

              Yes it's a pain, but much like dentistry it's a preventative pain that spreads a small controlled annoyance over a planned schedule as opposed to a big problem cropping up all at once unexpectedly (and usually at the worst possible time).

              • by tepples ( 727027 )

                Presumably by the tech flipping a hardware rw/ro switch on the drive after proper isolation conditions are met.

                If such a system were deployed in home PCs, how much would it cost for the tech to visit each user and flip the switch? I see no way to make such on-site service cost-effective.

          • Indeed I am waiting for a true hybrid system to be built. One that has the OS installed in read only flash and applications on a separate drive. you might ask why? but then stop to realize what would happen if viruses couldn't overwrite the system settings. that to clean up a virus all you had to do was to reboot.

            You mean like the way I run Windows under VMware and roll back to a snapshot?

          • by drsmithy ( 35869 )

            One that has the OS installed in read only flash and applications on a separate drive. you might ask why? but then stop to realize what would happen if viruses couldn't overwrite the system settings. that to clean up a virus all you had to do was to reboot.

            Why would rebooting clean up a virus that had inserted itself into the user's data and programs ?

            Or are you proposing a system when the user can't write to anything as well ? We have those already, they're called consoles.

        • ZFS (Score:2, Interesting)

          by Anonymous Coward

          The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?

          The article treats flash as something you place in between hard drives and memory. This turned out not to happen (with a few exceptions). SSD's simply replace hard drives. Hybrid systems are rare, and it doesn't look like they will become more common -- either you can live with the slowness of hard drives, or you can't. The mainstream will switch to SSD's for everything except backup applications.

          There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.

          Except if you're using ZFS. You can put a (SLC or MLC) SSD drive into just about any system and tell it to act as a (write or read) cache.

        • Re: (Score:2, Insightful)

          Flash is still 10-20x the price for the same GB. Recent developments have increased max magnetic density 1000x current. While you may be happy with 128GB, think of what you could do with 128TB. Store all your HD movies, lectures, conversations, life. Keep all your web history, including pix and html. And then there's the "killer app" we haven't reached yet...

          tOM

        • There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.

          FWIW, Sun's ZFS has the ability to automagically use flash drives as intermediate stage in front of rotating disks.

        • by bwcbwc ( 601780 )

          The other problem with the role ascribed to flash memory in the article is the write-wearing of flash. If flash memory decays with fewer writes than disk memory, there's a trade-off between the overall lifetime of the disk subsystem and the use of flash memory for caching/buffering. If you do most of your active writes to flash as a buffer before doing sector-based writes to magnetic disk, sure your performance is better, but you shorten the lifetime of the disk subsystem.

          A more likely scenario is to have a

        • Re:What article? (Score:5, Informative)

          by wwwillem ( 253720 ) on Sunday July 05, 2009 @03:06PM (#28588201) Homepage

          Hybrid systems are rare, and it doesn't look like they will become more common.

          You're probably right when we talk about desktop PCs and laptops. I'm sure the latter will be SSD only in 5-10 years time, and desktops are also losing terrain quickly against laptops.

          But when we look at datacenter grade enterprise storage, hybrid systems are currently picking up fast. The advantage is that because of the fast 'flash memory cache' you can use SATA disks instead of the FC/SCSI drives, where the former are both much bigger and much cheaper. Instead of 300 146GB 15K FC disks, you only need 30 1.5 TB 7200 RPM SATA disks. For the same capacity this results in much lower power bills, less DC floor-space costs and much better performance.

          If you say "There are some hybrid SAN's, but they're damn expensive.", have a look at what [shameless plug-on] Sun is doing [sun.com], and yes, I work for Sun [plug-off]. But other storage vendors (NetApps, EMC, IBM, etc.) are starting to do similar things.

          So the whole "storage-stack" gets more and more hybrid and integrated. It consists of the full gamut of DRAM, flash memory, hard drives and finally tape. Each of these have their own strength and are used best in combination.

          • by drsmithy ( 35869 )

            But other storage vendors (NetApps, EMC, IBM, etc.) are starting to do similar things.

            Actually, EMC seem to be most emphatically NOT doing it and are just classifying flash as another tier to be managed by the storage admin,

        • by DrDitto ( 962751 )
          Ya...if you run a 5-petabyte storage system, let me know if you replace all of that with flash. Heck, anybody who stores > 1 TB will be using disk drives (and that is a lot of people given video, 16-megapixel cameras, etc.).
        • Isn't ReadyBoost [wikipedia.org] essentially a hybrid system?

          Also I rememeber that one of the main disadvantages of Btrfs over ZFS was that I doesn't support using SSD to speed up overall access, while ZFS does [sun.com].
          • Readyboost and Superfetch are really just hacks to get around the 3GB or so ceiling in 32 bit Vista due to incomplete support of the Pentium Pro and later processors (PAE extension). With the 64 bit versions (or the server 32 bit versions, or any OS produced by anyone other than Microsoft in the last decade) you can use real memory instead for improved performance. Consider that you are grabbing all that stuff from disk and doing the relatively slow write to flash to save time when it needs to go into mem
            • The point of the original story was that SSD is cheaper than ran. Clearly moving everything from SSD into ram is going to make things faster, but so would moving everything from HDD to ram. Its just a matter of cost vs benefit.

              I am not too sure how the cost/benefit of ReadyBoost stacks up, but I'd guess that plugging in a decent flash-drive would be cheaper than trying to find obsolete Laptop ram.

              The wikipedia entry for Superfetch [wikipedia.org] doesn't mention anything about only being 32bit, and it sound like it w
            • by drsmithy ( 35869 )

              Readyboost and Superfetch are really just hacks to get around the 3GB or so ceiling in 32 bit Vista due to incomplete support of the Pentium Pro and later processors (PAE extension). With the 64 bit versions (or the server 32 bit versions, or any OS produced by anyone other than Microsoft in the last decade) you can use real memory instead for improved performance. Consider that you are grabbing all that stuff from disk and doing the relatively slow write to flash to save time when it needs to go into memo

        • by drsmithy ( 35869 )

          There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.

          Er, what ? NetApp and Sun are two examples of vendors with "hybrid" NAS/SANs that use SSDs as an extra caching level, and they're _very_ price competitive with pure-flash SANs. Particularly if data volume is important.

          Or are you calling a Linux machine stuffed with Intel X25-Es and an iSCSI target a "SAN" ?

    • I'm somewhat curious as to why people would moderate the original post off topic? It's specifically addresses the article and is ABOUT the article. How is it off topic?

      Given the fact that every other post in the article is modded offtopic, I suspect someone has gone through and just modded everything off topic.

      Either way, the point still stands. The article fails to recognize or address SSDs in any way, shape or form. As such, the article is basically mostly irrelevant in 2009 and going forward. It's i

      • Re:Flash memory? (Score:4, Informative)

        by Courageous ( 228506 ) on Sunday July 05, 2009 @11:59AM (#28586987)

        I couldn't quite figure out if the article willfully ignored the advent of SSDs or was written before they were available... As for the post by Argent, I wasn't sure if that was addressed to me or not - if so, I have no idea what you're talking about. Your post has absolutely nothing to do to with my original response.
         
        Argent's post refers to "flash memory". You said the article ignored SSD's, however it did not. "Flash memory" is the technology that SSD's are composed of. Did you not know this? "Flash memory" is all over the article.


        Flash SSD's will not replace SATA drives anytime within the next 4-6 years. In technology time, that's such long period of time, it would be quite difficult to make a credible projection for the consumer market space. For servers, where the segment is dominated by 10K/15K drives, you can expect flip over within 18 months.


        C//

        • Why would SSD's not take over within the next few years? I expect that in one, two years time, all new notebooks will be flash based. Desktops will soon follow, possibly with a few year of a HDD next to the SSD for mass storage.

          I for one do not need a 350 GB HDD on my notebook such as I have now. I would love to have better power life and snappier response though. 80GB is comfy for me, and has been so for 5 years. Movies and stuff I put on an external drive, and transfer when I need it. Netbooks already d

          • Re: (Score:3, Insightful)

            by Courageous ( 228506 )

            Depends what segment you're talking about. There isn't one market for HDD's, there are many. In both the enterprise archival and consumer mass storage segments, drives are sold pretty much $/TB. In that segment, you won't see significant penetration for 4-6 years. In other segments, sooner. I agree with you: I want a flash SSD for my laptop currently; it just hasn't quite yet reached the right price point. It will soon. And you're right: it's not $/GB that will be the deciding factor there.

            There's also no r

          • Re: (Score:3, Insightful)

            by Eskarel ( 565631 )

            If you've still got an external(or internal) physical HDD, then SSD's have not taken over. They've become a part of a new solution, not replaced an old solution.

    • by smaddox ( 928261 )

      I'm not sure that is entirely true. High capacity SSDs will need to write large chunks to have good write speeds, and to reduce wear. Having a separate small chunk cache will still be necessary.

    • Re: (Score:2, Insightful)

      by m.dillon ( 147925 )

      SSDs are not likely to replace traditional hard drives any time soon, if ever. The cost differential is simply too high. While it is true that SSDs can effectively replace HDs on systems which do not require large amounts of storage (say, 64G-256G), the fact of the matter is that a 256G SSD is still three times as expensive as a 2TB hard drive. That is a 24:1 cost factor.

      At the same time SSDs are becoming capable of replacing systems which do not have large storage requirements, HDs are becoming far more

  • by vaporland ( 713337 ) on Sunday July 05, 2009 @10:30AM (#28586531) Homepage
    five minutes is an awful long time for food to remain on the floor before you pick it up to eat it...
    • Not as long as you scrape all the dog hair off first.

    • It's just a bit dirty, it's still good, it's still good!

    • If you'd RTFA in the last article about the 5-second rule, then you'd know that it is a modern formulation of Genghis Khan's 12-hour rule, stating that you should not eat meat that had been on the floor for more than 12 hours. You'd also have learned that the amount of bacteria transmitted from a floor in five second and five minutes is about the same.
  • These days, the database is not what it used to be. Local clients use shared memory. JVMs and entire web servers are incorporated directly into the database executables. The old concept of the separation of the database from its clients no longer applies.

    When you are running a database, what business does the OS have, deciding what data is to be paged in or not? The database is in a far better position to make these decisions, and it can be based on much better rules than "5-minute" heuristics.

  • The way I see it, the advent of SSD storage gives us the ability to extend the cache layering abstraction we already use into a smooth continuum between the cpu's L1 cache (L2, L3, dram simms, etc...) and traditional HD-based disk cache. SDD doesn't quite close the gap but it actually fills a major portion of it.

    The article makes the mistake of assuming fairly small SSDs, but is otherwise spot-on. It isn't possible to use tiny SSDs in the 8G range as a paging medium for caching memory, it simply isn't eno

    • I expect that SSDs will in fact replace HDDs. But probably not with FLASH. Other solid state non-volatile random access memory technologies are likely to come around, hopefully with better wear and density characteristics. <tongue-in-cheek>There is a general fallacy here where some people seem to believe that SSDs can only ever be composed of FLASH</tongue-in-cheek>
    • by Eskarel ( 565631 )

      I'm not sure where people get this acceptable wear levels for SSD's thing.

      All the math I've seen indicates that, presuming reasonable wear algorithms, if you write the volume of your data to disk every day your drive will last for about 30 years and that it will scale linearly with shifts up or down in that amount. An ordinary hard disk lasts about 5 years, so for your 8 GB disk you'd have to be writing approximately 48 GB of data to it every single day in order for it to not last longer than a current HD,

    • by swilver ( 617741 )

      I don't quite see how it would allow you to more fully utilize resources. This is usually the argument given by people that think paging somehow will increase your systems performance because it makes more optimal use of all the memory.

      Unfortunately, this is only true if you have the perfect paging algorithm. It basically requires that you have a paging algorithm that is almost psychic in its ability to predict what future data may or may not be needed. Current paging algorithms, although no doubt comple

  • Advanced "universal" memory technologies (fast, non-volitile) such as MRAM, FRAM, maybe RRAM will alter this landscape significantly. While some are available now, we'll have to wait a few more technology generations before they have the density to realistically compete with hard drives or even Flash.

Real programmers don't comment their code. It was hard to write, it should be hard to understand.

Working...