Revisiting the Five-Minute Rule

In 1987, a study published by Jim Gray and Gianfranco Putzolu evaluated the trade-offs between holding data in memory and storing it on a disk. Known widely as the "five-minute rule," their research was updated and expanded 10 years later. Now, as jamie points out, Communications of the ACM is running an article by Goetz Graefe with another decennial update, evaluating the rule using hardware and software typical of 2007, with an eye toward how flash memory will affect the situation. An excerpt from Graefe's conclusion: "The 20-year-old five-minute rule for RAM and disks still holds, but for ever-larger disk pages. Moreover, it should be augmented by two new five-minute rules: one for small pages moving between RAM and flash memory and one for large pages moving between flash memory and traditional disks. For small pages moving between RAM and disk, Gray and Putzolu were amazingly accurate in predicting a five-hour break-even point two decades into the future. Research into flash memory and its place in system architectures is urgent and important. Within a few years, flash memory will be used to fill the gap between traditional RAM and traditional disk drives in many operating systems, file systems, and database systems."

Re:Flash memory?

[Rockoon]

You ask for evidence. How about some evidence that SSD's are being used anywhere in the way that you describe.. as a cache between rotational media and ram.

There are millions of people already using SSD's as (superior) drive replacements.

Do you really want evidence of the fact that SSD's are already replacing drives, that many millions of them have been sold specifically for that purpose, that even companies like Apple offer SSD's as alternatives solutions to rotational media in their standard packages?

ZFS

[Anonymous Coward]

The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?

The article treats flash as something you place in between hard drives and memory. This turned out not to happen (with a few exceptions). SSD's simply replace hard drives. Hybrid systems are rare, and it doesn't look like they will become more common -- either you can live with the slowness of hard drives, or you can't. The mainstream will switch to SSD's for everything except backup applications.

There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.

Except if you're using ZFS. You can put a (SLC or MLC) SSD drive into just about any system and tell it to act as a (write or read) cache.

Re:Not to be confused with

[johannesg]

The more useful 5 second rule [wikipedia.org].

That's just utterly disgusting. Do people in the US really believe that you can eat food that's fallen on the floor if you pick it up fast enough?

Re:What article?

[Anonymous Coward]

Flash replacing spinning drives is something I've heard of for a few years now, but it turns out to have not happened yet. Instead, it overlaps optical and magnetic drives due to price and capacity issues. It's obliterated all other temporary portable storage (floppies, cd-rw, dvd-rw), but it doesn't compete with the big reusable media on storage-per-dollar (hard drives are an order of magnitude cheaper) and it doesn't compete with long term cheap write-only storage (blank DVDs are still an order of magnitude cheaper in their common form... $.35 or so per blank single sided disc that holds 4 gig, vs $5-$15 for a 4 gig USB flash stick. And in newer tech, $15 will get you 50 gig of blu-ray). So for that kind of thing, flash isn't going to overtake magnetic storage for a long while yet, unless either flash gets a huge tech boost above and beyond its current rate of improvement, or hard drives hit a wall.

But the more expensive flash can slot right into the memory hierarchy, because its capacity and speed and price (and power draw?) fit right in between main memory and hard drives. The stuff that'd be silly for most hard drive use - $200 for a measly 32 gig of high performance flash - is still much cheaper than the equivalent capacity RAM, but much faster than the equivalent capacity of magnetic hard drive. While this may not be needed much in current desktops - we don't really need 32 gig of extra cache yet for our web browsers - it could certainly be immediately useful in servers. Discs are horrifically slower than RAM, after all.

AFAIK, they're already fooling around with using flash caches in databases.

Re:OS patches?

[tepples]

MSDN? Noted. The author of that article has a vested interest in an operating system prone to such exploits.

Ad hominem.

the Linux user is somewhat less likely to a: have admin rights or to b: grant admin rights to a program if he has them.

In the case of Linux on the home desktop, the owner of the PC has admin rights, and I don't see how granting setuid is any harder to social-engineer out of an inexperienced Linux user than out of an inexperienced Windows user.

When I look at bunnies, I see food.

What about dancing scantily clad people of the appropriate sex?

the king is dead

[epine]

What I love about slashdot is its scalability. The discussion ranges anywhere from the design of a Google data center in 2015 to some guy's psychological stance toward his next netbook purchase in 2009. Sometimes it's unclear which end of the spectrum is under debate, but the discussion happily progresses in a state of astral superposition. When this gets too confusing, even for slashdot, the moderation system helps to sort things out. For example, if the comment

Flash memory is set to replace rotational media.

is moderated +1 insightful, then we know we're talking about some guy's future netbook purchase. Or if the same comment is moderated -1 troll, then we know we're talking about Google data centers in 2015.

Flash memory begins to fade - ZDNet.co.uk [zdnet.co.uk] from 2005

"The scaling laws are not favourable to flash," said Tom Lee, an associate professor of electrical engineering at Stanford University and a founder of Matrix Semiconductor, which makes a 3D memory chip that performs flash-like functions. "The noises are getting louder now, so it looks like manufacturers are already in that new age of diminished gains."

Numonyx Breakthrough Delivers First 45nm NOR Flash Memory Chips [benchmarkreviews.com] from Jan 2009

"Numonyx engineers overcame major scaling limitations by developing new process techniques to produce the 7th generation MLC NOR flash on the industry's most advanced 45nm technology, and to be the first to bring the cost and performance benefits to our customers."
...
"At a time when the entire industry grapples with the scalability of all flash memory technologies, ..."

Brewster Kahle [ted.com]

I think Brewster Kahle is going to jump off a bridge when he learns that Seagate is exiting the disk drive business in 2010. If you think CERN or EOS cost a lot of money, try updating the budget with SSD specified as the primary storage layer.

A useful way to view this transition is the long tail on steroids. 99% of the world's stored information will be held by a few hundred mega-scale institutions (NASA, Google, CERN, GenBank) on rotating hard drives, while 99% of the world's gadgets have no hard drive at all.

The same thing happened in software. The C language represents a tiny sliver of source code written over the last ten years, but if you could measure the number of machine instructions executed by language of origin, C would continue to represent a very large slice of the pie. A major factor in the success of scripting languages is that the problems these languages don't handle well can be off-loaded to a well established compiled language. If you cherry pick your niche, it's amazing how much more convenient it looks compared to the ancestral technology which didn't.

I thought the paper was quite good, and more relevant than 99% of what I read these days. I'm always interested in analysis of hybrid solutions. In the engineering world, there is a de facto allergy to hybrid solutions. We tend to achieve the best result by scaling a single virtue to the max, rather than engaging in the jello-like trade-offs involved in balancing complementary virtues. I first began to think about this when ethernet trounced ATM by the simple measure of vastly over-provisioning bandwidth.

The exception to this is on the large scale where operational costs exceed all other costs, such as major data centers.

This is one of the reasons why progress in ecology is so painfully achieved: ecological systems almost always demand hybrid solutions, and we're not terribly comfortable with this. Engineers prefer monarchy. In ecological systems, life is complicated, and you can't just sit there and

Carpets are safer ...

[tomhudson]

Carpets don't transfer as many bacteria [nytimes.com]

Scientists have put the commonly-cited five-second rule to the test. They found that food that comes into contact with a tile or wood floor does pick up large amounts of bacteria. Food doesn't pick up many germs when it hits carpet, but it does pick up carpet fuzz.

Since this is slashdot, I'd bet most will pick bacteria over carpet fuzz any day ... after all, if it doesn't look fuzzy ...

or this ... [scienceline.org]

many people believe that gastric acid enzymes found in the stomach are strong enough to destroy the "small, harmless" amount of bacteria that could gather on a piece of food in five seconds. But are these bacteria really harmless?

In 2003, Jillian Clarke, then a high school senior, decided she wanted to find out. During an internship at the University of Illinois at Urbana-Champaign, she and a doctoral candidate, Meredith Agle, took swab samples from floors all over the campus, including labs, hallways, and bathrooms, and found that the amount of bacteria on the floors was very low. When she published her research, she concluded that if a piece of food falls on a relatively clean floor, the five-second rule is, in fact, applicable.