In 1987, a study published by Jim Gray and Gianfranco Putzolu evaluated the trade-offs between holding data in memory and storing it on a disk. Known widely as the "five-minute rule," their research was updated and expanded 10 years later. Now, as jamie points out,
Communications of the ACM is running an article by Goetz Graefe with another decennial update,
evaluating the rule using hardware and software typical of 2007, with an eye toward how flash memory will affect the situation. An excerpt from Graefe's conclusion:
"The 20-year-old five-minute rule for RAM and disks still holds, but for ever-larger disk pages. Moreover, it should be augmented by two new five-minute rules: one for small pages moving between RAM and flash memory and one for large pages moving between flash memory and traditional disks. For small pages moving between RAM and disk, Gray and Putzolu were amazingly accurate in predicting a five-hour break-even point two decades into the future. Research into flash memory and its place in system architectures is urgent and important. Within a few years, flash memory will be used to fill the gap between traditional RAM and traditional disk drives in many operating systems, file systems, and database systems."
for those wondering: (Score:5, Informative)
"The 5-minute random rule: cache randomly accessed disk pages that are re-used every 5 minutes."
Not to be confused with (Score:5, Funny)
Parent
Re: Not to be confused with (Score:2)
Nor the funkier 3 minute rule [youtube.com].
Re:Not to be confused with (Score:4, Interesting)
The more useful 5 second rule [wikipedia.org].
That's just utterly disgusting. Do people in the US really believe that you can eat food that's fallen on the floor if you pick it up fast enough?
Parent
Re:Not to be confused with (Score:4, Insightful)
Parent
Re:Not to be confused with (Score:5, Funny)
I'd rather be a disgusting American than a naive European with no sense of humor..
Parent
Re:Not to be confused with (Score:4, Insightful)
As a not so naive Brit, we have our much superior 3 second rule for the same thing.
I'd like to know what the GP has on the floors in his house that is so toxic that the tiny amount that will rub off on food is detrimental for your health? How on earth do people survive in places without nice sealed floors and cleaning chemicals? You'd think we'd have evolved some method of protecting our bodies against stuff like that.
Parent
Re: (Score:3, Funny)
No silly, but if you catch it on the bounce, it's like it never happened
Re: (Score:3, Insightful)
Well the confusion is, unlike your dirt-floor huts full of your own feces, in real countries like America we have clean tile floors with miraculous inventions we call "mops".
Carpets are safer ... (Score:4, Interesting)
Since this is slashdot, I'd bet most will pick bacteria over carpet fuzz any day ... after all, if it doesn't look fuzzy ...
or this ... [scienceline.org]
Parent
Re: (Score:3, Funny)
Normal floors are probably safer than your hands. If it's in your own home and you just dropped it or something, I can't even really imagine why it would be mentally bothersome.
It's not really an issue if you have a dog, though.
Flash memory? (Score:2, Insightful)
I couldn't quite figure out if the article willfully ignored the advent of SSDs or was written before they were available and not updated to include them (but it appears the article was updated to include other current technology).
Given the fact that SSDs are likely going to replace rotational media for most applications in the future, it makes this article basically meaningless, at least insofar as the fact that flash memory and the disk are/will be synonymous. As the article is basically predicated aroun
What article? (Score:3, Insightful)
The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?
Re:What article? (Score:5, Insightful)
The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?
The article treats flash as something you place in between hard drives and memory. This turned out not to happen (with a few exceptions). SSD's simply replace hard drives. Hybrid systems are rare, and it doesn't look like they will become more common -- either you can live with the slowness of hard drives, or you can't. The mainstream will switch to SSD's for everything except backup applications.
There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.
Parent
Re: (Score:2)
The grandparent was talking about hybrids, but also something new. If and when MRAM becomes possible the technological hard drive wether spinning or flash is gone.
Indeed I am waiting for a true hybrid system to be built. One that has the OS installed in read only flash and applications on a separate drive. you might ask why? but then stop to realize what would happen if viruses couldn't overwrite the system settings. that to clean up a virus all you had to do was to reboot.
OS patches? (Score:2)
I am waiting for a true hybrid system to be built. One that has the OS installed in read only flash and applications on a separate drive. you might ask why? but then stop to realize what would happen if viruses couldn't overwrite the system settings. that to clean up a virus all you had to do was to reboot.
How would such a hybrid system correct a discovered defect in the operating system?
Re: (Score:2)
I propose a physical switch to toggle between read/write and read only. Even if it could be controlled remotely by a management server, it would provide a nice increase in security.
Re: (Score:3, Informative)
I propose a physical switch to toggle between read/write and read only.
And have social engineers disguise malware as OS updates or dancing bunnies [msdn.com], prompting the home user who doesn't understand risks to flip the switch to see the dancing bunnies.
Re: (Score:3, Interesting)
MSDN? Noted. The author of that article has a vested interest in an operating system prone to such exploits.
Ad hominem.
the Linux user is somewhat less likely to a: have admin rights or to b: grant admin rights to a program if he has them.
In the case of Linux on the home desktop, the owner of the PC has admin rights, and I don't see how granting setuid is any harder to social-engineer out of an inexperienced Linux user than out of an inexperienced Windows user.
When I look at bunnies, I see food.
What about dancing scantily clad people of the appropriate sex?
Re: (Score:2)
Presumably by the tech flipping a hardware rw/ro switch on the drive after proper isolation conditions are met.
Yes it's a pain, but much like dentistry it's a preventative pain that spreads a small controlled annoyance over a planned schedule as opposed to a big problem cropping up all at once unexpectedly (and usually at the worst possible time).
Re: (Score:2)
Presumably by the tech flipping a hardware rw/ro switch on the drive after proper isolation conditions are met.
If such a system were deployed in home PCs, how much would it cost for the tech to visit each user and flip the switch? I see no way to make such on-site service cost-effective.
ZFS (Score:2, Interesting)
The article I read spent a good deal of time talking about flash memory. What article are YOU referring to?
The article treats flash as something you place in between hard drives and memory. This turned out not to happen (with a few exceptions). SSD's simply replace hard drives. Hybrid systems are rare, and it doesn't look like they will become more common -- either you can live with the slowness of hard drives, or you can't. The mainstream will switch to SSD's for everything except backup applications.
There are some hybrid SAN's, but they're damn expensive. At that price they have a hard time competing with simpler pure-flash SAN's.
Except if you're using ZFS. You can put a (SLC or MLC) SSD drive into just about any system and tell it to act as a (write or read) cache.
Re:What article? (Score:5, Informative)
Hybrid systems are rare, and it doesn't look like they will become more common.
You're probably right when we talk about desktop PCs and laptops. I'm sure the latter will be SSD only in 5-10 years time, and desktops are also losing terrain quickly against laptops.
But when we look at datacenter grade enterprise storage, hybrid systems are currently picking up fast. The advantage is that because of the fast 'flash memory cache' you can use SATA disks instead of the FC/SCSI drives, where the former are both much bigger and much cheaper. Instead of 300 146GB 15K FC disks, you only need 30 1.5 TB 7200 RPM SATA disks. For the same capacity this results in much lower power bills, less DC floor-space costs and much better performance.
If you say "There are some hybrid SAN's, but they're damn expensive.", have a look at what [shameless plug-on] Sun is doing [sun.com], and yes, I work for Sun [plug-off]. But other storage vendors (NetApps, EMC, IBM, etc.) are starting to do similar things.
So the whole "storage-stack" gets more and more hybrid and integrated. It consists of the full gamut of DRAM, flash memory, hard drives and finally tape. Each of these have their own strength and are used best in combination.
Parent
Re: (Score:2)
What's the problem? If you need to store 1TB and you can live with the slowness of hard drives, you use hard drives. If you can't, you use SSD's -- or possibly an array of small fast hard drives, but SSD's will most often be cheaper.
What you don't do is build a hybrid system with automatic page migration between SSD and hard drives -- and that is what the article assumes will be commonly used. Hierarchial storage has a very small niche, and SSD's won't make it more popular.
Hence the article is useless.
Re: (Score:2, Funny)
Wow...almost nothing but offtopic and redundant posts so far.
Well, this is /. What do you expect?
Natalie Portman and hot grits.
Re: (Score:2)
I'm somewhat curious as to why people would moderate the original post off topic? It's specifically addresses the article and is ABOUT the article. How is it off topic?
Given the fact that every other post in the article is modded offtopic, I suspect someone has gone through and just modded everything off topic.
Either way, the point still stands. The article fails to recognize or address SSDs in any way, shape or form. As such, the article is basically mostly irrelevant in 2009 and going forward. It's i
Re:Flash memory? (Score:4, Informative)
I couldn't quite figure out if the article willfully ignored the advent of SSDs or was written before they were available... As for the post by Argent, I wasn't sure if that was addressed to me or not - if so, I have no idea what you're talking about. Your post has absolutely nothing to do to with my original response.
Argent's post refers to "flash memory". You said the article ignored SSD's, however it did not. "Flash memory" is the technology that SSD's are composed of. Did you not know this? "Flash memory" is all over the article.
Flash SSD's will not replace SATA drives anytime within the next 4-6 years. In technology time, that's such long period of time, it would be quite difficult to make a credible projection for the consumer market space. For servers, where the segment is dominated by 10K/15K drives, you can expect flip over within 18 months.
C//
Parent
Re: (Score:3, Insightful)
Depends what segment you're talking about. There isn't one market for HDD's, there are many. In both the enterprise archival and consumer mass storage segments, drives are sold pretty much $/TB. In that segment, you won't see significant penetration for 4-6 years. In other segments, sooner. I agree with you: I want a flash SSD for my laptop currently; it just hasn't quite yet reached the right price point. It will soon. And you're right: it's not $/GB that will be the deciding factor there.
There's also no r
Re: (Score:3, Insightful)
If you've still got an external(or internal) physical HDD, then SSD's have not taken over. They've become a part of a new solution, not replaced an old solution.
Re: (Score:2)
No shit, that is exactly my point.
SSDs are set to replace rotational media. That was what I stated.
So... to make it more clear to the people with reading comprehension issues:
Flash memory is set to replace rotational media.
This article indicates that Flash Memory (AKA SSDs) are only going to be an intermediary between rotational media and RAM. This is clearly not going to be the case going forward... this is WHY I wrote the GP post and pointed out that it's invalid.
WTF. Seriously. Are people that incapa
External hard drives == memory hierarchy (Score:3, Insightful)
This article indicates that Flash Memory (AKA SSDs) are only going to be an intermediary between rotational media and RAM.
If your handheld device or subnotebook PC has only an internal SSD and no internal hard drive, then you will store any data that doesn't fit on your SSD on a hard drive plugged into a Hi-Speed USB port, copying it to the SSD when it is needed. For example, you'd keep the video footage that you are editing on the SSD and other projects on the hard drive. That sounds to me like a memory hierarchy, albeit one that occasionally requires manual intervention to connect the (offline) long-term mass storage to the
Re: (Score:2)
the king is dead (Score:3, Interesting)
What I love about slashdot is its scalability. The discussion ranges anywhere from the design of a Google data center in 2015 to some guy's psychological stance toward his next netbook purchase in 2009. Sometimes it's unclear which end of the spectrum is under debate, but the discussion happily progresses in a state of astral superposition. When this gets too confusing, even for slashdot, the moderation system helps to sort things out. For example, if the comment
Flash memory is set to replace rotational media.
is moderated +1 insightful, then we know
Re: (Score:3, Interesting)
There are millions of people already using SSD's as (superior) drive replacements.
Do you really want evidence of the fact that SSD's are already replacing drives, that many millions of them have been sold specifically for that purpose, that even companies like Apple offer SSD's as alternatives solutions to rotational media in their standard packages?
Re: (Score:2)
I'm not sure that is entirely true. High capacity SSDs will need to write large chunks to have good write speeds, and to reduce wear. Having a separate small chunk cache will still be necessary.
I thought you were referring to the 5 second rule (Score:4, Funny)
Re: (Score:2)
Not as long as you scrape all the dog hair off first.
Re: (Score:2)
It's just a bit dirty, it's still good, it's still good!
Re:I thought you were referring to the 5 second ru (Score:2)
Re: (Score:2)
Do the old premises still apply? (Score:2)
These days, the database is not what it used to be. Local clients use shared memory. JVMs and entire web servers are incorporated directly into the database executables. The old concept of the separation of the database from its clients no longer applies.
When you are running a database, what business does the OS have, deciding what data is to be paged in or not? The database is in a far better position to make these decisions, and it can be based on much better rules than "5-minute" heuristics.
Extending the memory cache abstraction to paging (Score:2)
The way I see it, the advent of SSD storage gives us the ability to extend the cache layering abstraction we already use into a smooth continuum between the cpu's L1 cache (L2, L3, dram simms, etc...) and traditional HD-based disk cache. SDD doesn't quite close the gap but it actually fills a major portion of it.
The article makes the mistake of assuming fairly small SSDs, but is otherwise spot-on. It isn't possible to use tiny SSDs in the 8G range as a paging medium for caching memory, it simply isn't eno
Re: (Score:2)
So I RTFA to see if it answered any of my questions and it seems that they are just saying that they don't have the answers either. I don't know if I agree with their use of flash as extended ram. Why not just make it a super fast drive that sits close to the cpu and give it the illusion of being an IDE drive so us normal users can just make it a swap partition. Oh, and give the chip a socket so we can upgrade/replace it later. I don't think it would cost manufacturers a great deal more to add one more sock
Erase traffic burns up SSDs (Score:2)
I don't know if I agree with their use of flash as extended ram. Why not just make it a super fast drive that sits close to the cpu and give it the illusion of being an IDE drive so us normal users can just make it a swap partition.
A swap partition has a lot more erase traffic than a data partition, and access patterns high in erases are thought to wear out SSDs faster than hard drives.
Re: (Score:2)
Does swap really erase? I was always just under the impression that if a block was no longer used it was just demapped. If the OS decides to write to that block again it just merely overwrites that block. Unless I'm missing something here? I thought the problem with running swap on a flash was the sheer number of writes involved, not the erases. The problem with erasing, from what I can tell, is when you have a temp folder that keeps erasing and further reducing the number of free whole blocks by fragmentin
rarely-rewritten logical sectors (Score:2)
Does swap really erase? I was always just under the impression that if a block was no longer used it was just demapped. If the OS decides to write to that block again it just merely overwrites that block.
A flash block must be erased before it is overwritten, and each sector is guaranteed for only about 100,000 erases. Lots of writes are fine on larger SSDs because the wear leveling program in the drive's controller will move rarely-rewritten logical sectors (such as free space and read-only files) to more-worn physical sectors. But if you're devoting an entire device to swap, there won't be a lot of rarely-rewritten logical sectors for the controller to make use of.
Re: (Score:2)
But if you're devoting an entire device to swap, there won't be a lot of rarely-rewritten logical sectors for the controller to make use of.
Only if you have a very unusual swap load, where the swap space is 100% full and every swap operation is exactly that; an exchange of some pages between memory and disk. In practice, a swap partition is likely to be largely full of data that isn't ever accessed. For example, if an app leaks some memory, this will be swapped out after not being used for a while and never swapped back in. If an application spends most of its time idle, it will be swapped out and stay on disk for a long time, being occasion
Re: (Score:2)
Obviously, the problem with running Puppy in a ramdisk and not having it write back until shutdown is that if it crashes you'll lose your data. You can maybe do a full install to avoid that, I'm not sure ... it'd be nice if it could persist data to storage on request but I don't know if it can.
Re: (Score:2)
It's small, wicked fast, includes Firefox.
To be pedantic, it's the Mozilla SeaMonkey suite. Puppy's a great distro though, especially for older machines.
Re: (Score:2)
It's small, wicked fast, includes Firefox.
To be pedantic, it's the Mozilla SeaMonkey suite. Puppy's a great distro though, especially for older machines.
Hrmmm, OK! I imagine using a combined suite makes sense, space-wise, perhaps? I always liked using SeaMonkey, before the days of Firefox, so that's cool.
I *think* the version I actually used did have Firefox but that was a very long time ago now. When I used it I'm not sure if they had a package manager, even!
I should really burn a new Puppy CD so that I'm prepared when this computer dies ;-) Or, given the speed of the device isn't an issue if running from a ramdisk, I could buy a dirt cheap 512MB USB s