Wear Leveling, RAID Can Wipe Out SSD Advantage 168
storagedude writes "This article discusses using solid state disks in enterprise storage networks. A couple of problems noted by the author: wear leveling can eat up most of a drive's bandwidth and make write performance no faster than a hard drive, and using SSDs with RAID controllers brings up its own set of problems. 'Even the highest-performance RAID controllers today cannot support the IOPS of just three of the fastest SSDs. I am not talking about a disk tray; I am talking about the whole RAID controller. If you want full performance of expensive SSDs, you need to take your $50,000 or $100,000 RAID controller and not overpopulate it with too many drives. In fact, most vendors today have between 16 and 60 drives in a disk tray and you cannot even populate a whole tray. Add to this that some RAID vendor's disk trays are only designed for the performance of disk drives and you might find that you need a disk tray per SSD drive at a huge cost.'"
Little Flawed study. (Score:5, Insightful)
This assumes that RAID controller manufacturers won't be making any changes though.
RAID for years has relied on millisecond access times. So why spend a lot of money on an ASIC & Subsystem that can go faster? So taking a RAID card designed for slow (relatively) spinning disks and attaching them to SSD of course the RAID card is going to be a bottleneck.
However subsystems are going to be designed to work with SSD that has much higher access times. When that happens, this so called 'bottleneck' is gone. You know every major disk subsystem vendor is working on these. Sounds like a disk vendor is sponsoring 'studies' to convince people not to invest in SSD technologies now knowing that a lot of companies are looking at big purchases this year because of the age of equipment after the downturn.
Re: (Score:3, Insightful)
The article is talking about stuff that's available today. They aren't saying "SSDs will never be suitable", they're saying they aren't suitable today. Why? Because none of the hardware infrastructure available is fast enough.
Re:Little Flawed study. (Score:5, Interesting)
Sure, but why do you put 60 drives in a RAID?
Because hard disks, even the high end ones, have quite low IOPS. You can attain the same performance level with much fewer SSDs. If what you need is IOPS and not lots of storage that's a good thing even. You reach the required level with much fewer drives, so you need less power, less space and less cooling.
Re:Little Flawed study. (Score:5, Interesting)
I agree. 60 drives in RAID0 are going to see between 150 and 200 IOPS/drive, maybe more for 2.5" drives right? So that's 12,000 IOPS.
The X25-E, the new Sandforce controller, and I believe some of the newer Indilinx controllers can all do that with one SSD.
$/GB is crap, $/IOPS is amazing.
Re: (Score:2)
The next problem is that large RAID 0 arrays will suffer from word-width. The reason that ATA and SCSI buses were parallel buses had to do with chip fanout and cable length (slew rate). When SATA and SAS arrived, they use single fast clocks to frame out data and re-lengthen the cable by convenience.
When you get a bunch of drives that can now be accessed faster than the interface IO rate, the interface IO rate has to change and that starts putting RAID controller technology into the same realm of difficulty
just use the edge of the disk (Score:3, Interesting)
Re: (Score:2)
Yes, disks are cheap
The power to run and cool them, and the space to hold them on your data center floor, are not.
Re: (Score:2)
Quoted for truth. For applications where GB are not needed but IOPS is, IOPS is the only thing that matters. Booting and applications usually need less than 40GB. We are already at that point where the total cost is more than worth it from a total system cost perspective. Games are there, or nearly there. I highly doubt that someone is going to have more than 160GB of games they play regularly, in which case, SSD are a logical choice for games. Low minimum FPS is what destro
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
They are suitable today. You just don't raid them using 50-100K RAID controllers.
Anyway, the "Enterprise Storage" bunch will probably stick both SSDs and TB SATA drives in their systems for the speed and capacity (and charge $$$$$$). I think some are doing it already.
Or you could stick a few SSDs in a decent x86 server with 10 Gbps NICs, and now you can
Re:Little Flawed study. (Score:5, Interesting)
You missed half the point. SSD use wear leveling and other techniques that are very effective on the desktop but in a high IO environment, the current wear leveling techniques reduce SSD performance to well below what you get on the desktop.
I really think that this is just a result of the current trend to put high performance SSD on the desktop. When the market re-focuses these problems will disolve.
This also goes for RAID controllers. If you have 8 ports and SAS 3Gb links, then you need to process 24Gb and a IO/s of current 15k SAS drives. Lets just assume for easy math that this requires a 500Mhz RAID Processor. What would be the point of putting in a 2Ghz Processor? What if you increase the IO/s by 100x and double the bandwidth? now you need to handle 48Gb/s throughput and 100x the IO and that requires 2x 3Ghz Processors.
Its just takes time for the market players to react to each technology increase. New raid controllers will come out that can handle these things. maybe the current raid cpus have been using a commodity chip (powerpc often enough) because it was fast enough to handle these things and the new technologies are going to require more specific processors. Maybe you need to get cell chips or nvidia GPUs in there, whatever it takes.
I admit it would be pretty interesting to see the new Dell/LSI 100Gb SAS powered by Nvidia logo in Gen12 Dell servers.
Re:Little Flawed study. (Score:4, Informative)
I clicked on this thinking this guy has done some testing... somewhere. Nope, nothing, no mention of benchmarks or what hardware he used. I'm sure some of he said is true. But I'd really like to see the data that he gets the
I have seen almost 4 to 1. That means that the write performance might drop to 60 MB/sec and the wear leveling could take 240 MB/sec.
from. I'd also really like to know what controllers he's tested with, wheather or not they have TRIM support (perhaps none do yet), what drives he used, if he had a BBU and write-back enabled etc etc etc.
Until he give us the sources and the facts this is nothing but a FUD piece. Yes, wear levelling will eat up some bandwidth, thats hardly news... show us the data about how much and which drives are best
Re: (Score:3, Interesting)
I dont think I missed the point. I am just a little more patient than most I guess. I don't think SSDs are ready from a cost/performance standpoint vs enterprise SAS 15k drives due to the market's focus.
The OP may not have listed the hardware and disks but each controller has info published on max throughput.
This is very comparable to running U320 SCSI disks on a U160 card. The performance bottleneck is often NOT the U160 interface but rather that the controller was not over engineered for its time. The
Re: (Score:2)
A fair number of the desktop stuff can take sustained writes for quite a long while- e.g. the entire disk or more.
http://benchmarkreviews.com/index.php?option=com_content&task=view&id=454&Itemid=60&limit=1&limitstart=10 [benchmarkreviews.com]
If that's not enough, some of the desktop benchmarks/tests, involve writing to the entire disk first, and then seeing how far the performance drops.
e.g.
http://www.anandtech.com/printarticle.aspx?i=3702 [anandtech.com]
See: "New vs. Used Performance - Hardly an Issue"
They're not cheap, but
Re: (Score:2)
But that isnt indicative of enterprise loads. Enterprise loads such as databases do many many seeks and tend to have long queues as many clients request the data. Size and throughput are less important for these loads than seek time (though still critical).
A desktop system can only(realistically) have a similar load in synthetic benchmarks.
The server vs desktop loads on disks are so different that they cant be directly compared. A great desktop drive can be a terrible server drive and vice versa.
Re: (Score:3, Informative)
Did you even read the link?
"I saw 4KB random write speed drop from 50MB/s down to 45MB/s. Sequential write speed remained similarly untouched. But now I've gone and ruined the surprise."
That's for random writes. 4KB random writes at 45MB/sec is 11520 writes per second.
A 15000rpm drive doing 4KB r
Re: (Score:3, Interesting)
Re: (Score:3, Interesting)
how about opensolaris with ZFS. you get a high performance iSCSI target and a filesystem with re-ordered writes that improves IO performance by reducing seeks plus optional deduplication and compression.
Additional gains can be had from seperate log and cache disks and with 8+ core platforms already available you can blow a traditional RAID card out of the water.
One nice thing about software raid is it is completely agnostic to controller failure. If you need to recover a raid after a controller failure, y
Re: (Score:3, Informative)
Re: (Score:3, Interesting)
WOW NICE motherboard there, TWO io hubs to give seven of x8 electrical/x16 mechanical slots along with a x8 link for the onboard SAS and an x4 link for the onboard dual port gigabit.
http://www.supermicro.com/products/motherboard/QPI/5500/X8DTH-i.cfm [supermicro.com]
Re: (Score:3, Insightful)
Re: (Score:2)
Not to be rude, but I'm guessing there's only 20-30 engineers in the world who have any idea what current wear leveling state-of-the-art is, and how it affects performance.
There's a huge variety in the quality of controllers in the marketplace, and just because one design has a given advantage or flaw, doesn't mean others share those attributes.
Re: (Score:2)
I dont think its about 'quality' so much as target. Desktop drives and server drives are different things and wear leveling between the two could be significantly different. Right now, I think that everything has been targeting desktops and notebooks and those products are adapted to the server space rather than being developed from the ground up for servers.
Also, what indication do you have that 20-30 people are experts in wear leveling? I would expect that many engineers as EACH flash memory vendor and
Re: (Score:2)
Welcome to the multi-tiered storage world. There are places and applications where SSDs are a perfect fit, and places where they are not. Eventually server builders will find a place where both work in tandem to give you the performance you were wanting to begin with. SSD is a fully cached drive. That's not necessary in all applications. For some applications, TB's of RAM is the better option. Combinations of various storage technology will find their niche market. SSDs are not financially practical for all
Re: (Score:3, Informative)
ceph [newdream.net], XIV, and other distributed storage controller models are available today, and avoid controller bottlenecks.
Re: (Score:2)
Why is it so hard for developers of ports and interface standards to get it super fast, first time round? It's not like there's a power issue and there's no worry about having to make things small enough (as with say the CPU).
For example, let's take USB:
USB 1: 12 Mbit/s
USB 2: 480 Mbit/s
USB 3: 4 Gbit/s
Same goes for video and SATA etc. Perhaps I'm being naive, but it seems like they're all a bit short-sighted. They should develop for the hardware of the future, not artificially limit the speed to what current
Re: (Score:3, Informative)
Why is it so hard for developers of ports and interface standards to get it super fast, first time round? It's not like there's a power issue and there's no worry about having to make things small enough (as with say the CPU).
There IS a power issue, and most importantly there's a price issue. The interface electronics limit speed. Even today, 10Gbps ethernet (10Gbase-T) is quite expensive and power hungry. 40Gbps ethernet isn't even possible with copper right now. They couldn't have made USB 3 40 Gbps instead of 4, the technology just isn't there. In 5 years maybe, in 10 years almost certainly.
USB 1 could have been made 100Mbps, but the others were close to what was affordable at the time.
Re: (Score:2)
Isn't it possible to make a wired connection where speed is determined by the amount of data sent. If true, then the power would only increase as the hardware became more sophisticated. Therefore, the USB3 interface would only eat up minimal power in the USB1 computer age (perhaps as little as USB 1 itself).
Re: (Score:3, Informative)
You would still have to run a sophisticated DSP, unless you kept entirely separate chips for USB1, USB2, and USB3. The DSP would eat lots of power even when working at USB1-speed.
Also, we're talking hundreds or thousands of dollars for a USB3-DSP in the USB1 era.
Re: (Score:2)
Even if the bottleneck moves from disk to controller the overall performance will improve. So it's not that SSD:s are bad, it's just that the controllers needs to keep up with them.
On the other hand - raid controllers are used for reliability and not just for performance. And in many cases it's a tradeoff - large reliable storage is one thing while high performance is another. Sometimes you want both and then it gets expensive, but if you can live with just one of the alternatives you will get off relativel
Duh (Score:2, Interesting)
RAID means "Redundant Array of Inexpensive Disks".
Re:Duh (Score:4, Informative)
Re: (Score:2)
Fully acceptable to illiterates, you mean.
Re: (Score:2)
Yes, because 15k RPM SAS drives are OH so inexpensive, right?
Starting out at 1.9$/GB for a 73.5 GB drive [newegg.com] is certainly inexpensive. Especially when you have to pay an insane 9.332 cent/GB for a 750 GB hard drive [newegg.com].
By your definition, you could NEVER EVER use RAID on expensive hard drives. Which obviously means that you are an idiot.
Re: (Score:2)
"Inexpensive" does not have an absolute definition, and if you think it does, I think you need to reexamine who the "idiot" is.
The idea has nothing whatsoever to do with the absolute price of any particular disk. The idea is that one can use comparatively inexpensive disks to create an array that would be prohibitively expensive were it to be replaced with a single drive of identical capacity and performance.
Also, that link to a 750GB drive is for a 640GB drive. I know kindergarten is hard, but one would th
Re: (Score:2)
Re: (Score:3, Funny)
Re: (Score:2)
Correction: (Score:5, Informative)
Wear Leveling, RAID Can Wipe Out SSD Advantage for enterprise.
While it may not be efficient to slap together a platter of 16 SSDs, it is worthwhile to upgrade personal computers to use an SSD.
Re: (Score:2)
I agree. You shouldn't be using consumer grade SSDs for servers - unless it's a game server or something. (Ex: TF2)
Do you know why RE (RAID Edition) HDDs exist? They strip out all the write recovery and stuff, which could mess up speeds, IOPS, and seek times, and instead streamline the drives for performance predictability. That makes it far easier for RAID controllers to manage dozens of them.
SSDs have a similar thing going. You're an enterprise and need massive IOPS? Buy enterprise-level SSDs - like the
Re: (Score:2)
The lousy thing about PCIe SSD's is that modern servers don't have enough PCIe slots. 1U servers often have only one free slot, and blade servers often have zero. The only blade vendor with decent PCIe expandability is Sun, and their blade density isn't fantastic.
Re: (Score:2)
True, but ioDrives have an IOPS edge that is massive. If that's what you need, then find a way to make it work.
Heh... Sun... how typical. :P
Re: (Score:2)
There are server boards out there with pelnty of PCIe slots e.g. http://www.supermicro.com/products/motherboard/QPI/5500/X8DTH-i.cfm [supermicro.com] (shamelessly grabbed from a board in a picture linked from another post here)
Yes you will need a case tall enough to take cards without risers (which means 3U afaict) but I would guess getting the same IOPS any other way would take up way more than 3U of rackspace.
Re: (Score:3, Interesting)
I'm working for an ISP, rack units are way too expensive to waste 3 on a server. Heck they're too expensive to waste 1 on a server.
The advantage over enterprise SATA/SAS SSD's isn't large enough for us at least. We would have to go to 6 socket motherboards to get the same CPU density.
Re: (Score:2)
Wear Leveling, RAID Can Wipe Out SSD Advantage for enterprise.
While it may not be efficient to slap together a platter of 16 SSDs, it is worthwhile to upgrade personal computers to use an SSD.
If there's a benefit, why wouldn't you upgrade your enterprise servers too?
We just built a "lab server" running ESXi 4, and instead of a SAN, we used 2x SSDs in a (stripe) RAID. The controller was some low-end LSI chip based one.
That thing was blazing fast -- faster than any SAN I have ever seen, and we were hitting it hard. Think six users simultaneously building VMs, installing operating systems, running backups AND restores, and even running database defrags.
It's possible that we weren't quite getting 'p
Re:Correction: (Score:4, Insightful)
I see this rather often on Slashdot and elsewhere. It's becoming a part of our collective culture it seems.
Increasingly, it's not good enough that you said what you did say, and chose not to say what you clearly haven't said. There's this unspoken expectation that you also have to actively disclaim things you clearly are not claiming, otherwise some clever individual who really wants to be "right" is going to assume that your lack of a disclaimer amounts to tacit support of whatever was not disclaimed. This leads to a great deal of both intentional trolling and unintentional creation of strawmen. Both invite unnecessary follow-up posts designed to correct unfounded assumptions.
I wonder if this comes from modern politics where the audience is generally "hostile" in the sense that it's eager to twist words and demagogue positions with which it may disagree. That's a poor substitute for good reasoning, for showing that there are substantive reasons to disagree. So much of politics is done by handling complex, nuanced issues with 20-second soundbites that I can see how it happens there. On Slashdot, it seems to lower the quality of discussion for no good reason.
Re: (Score:2)
Someone using their PC for word processing, Web browsing and email will see significant gains in overall system responsiveness, load times and above all system boot times.
If you ever witnessed a common Vista32 laptop booting in under 40 seconds, you know the use of SSDs.
Re: (Score:2)
Re: (Score:2)
The needs of enterprise customers will ensure that magnetic HDDs will continue to exist for years to come.
I just don't see it happening. HDD's are lousy for the enterprise, simply because of the laughably low IOPS. Yes you can compensate by buying 10 times as many disks, but SSD's aren't 10 times as expensive as 15k disks anymore. And yes, SSD's are going to saturate the RAID controllers -- but why would it ever be an advantage for HDD's that they're too slow to even saturate the lousy 500MHz PPC chips that are sold under the pretense of making RAID faster?
Re: (Score:3, Informative)
You obviously don't manage a SAN and I'm starting to think you've never seen one. SSD's are nice, but typical FC/SAS/SATA drives will be around for a long time to come. IOPS aren't all that matters in a SAN, space matters as well.
IE: Typical SANS are setup in tiers. In my case, we use Compellent SANs. New writes and active data is written to 15K FC drives. They are fast, expensive, and we have less total capacity than the SATA totals. After about 2 weeks, the inactive blocks that were on the FC drive
Re: (Score:2)
Except the EEE's often had drives that were slower than most USB 2.0 Flash drives. I know, I have a 900 and the first thing I did was replace it with a higher performance drive.
Only A Matter of Time (Score:3)
Seek time (Score:5, Informative)
The real advantage of solid state storage is seek time, not read/write times. They don't beat conventional drives by much at sustained IO. Maybe this will change in the future. RAID just isn't meant for SSD devices. RAID is a fix for the unreliable nature of magnetic disks.
Re:Seek time (Score:4, Informative)
Re: (Score:2)
True, though if what you need is sequential read/write performance then RAID0 will do that well at less cost and much higher capacity than an SSD. Normally the reason why you want that is because you're doing video capture or something similar that takes ungodly amounts of space, so RAID0 is pretty much a slam dunk here. It's the random read/write performance that is the reason for getting an SSD. In the 4k random read/write tests - which are easier for me to understand than IOPS as reading and writing lots
Re: (Score:2)
The new 64MB cache WD Black drives have wicked sustained read speeds. Close to 140MB/sec.
But when dealing with small files, you still notice the IOPS limit.
The cheaper SSDs won't do as well with a sustained write situation (Ex: Recording 12 security camera feeds) as a traditional HDD will.
Re:Seek time (Score:5, Insightful)
One of the fastest platters on the market today is the Seagate 15,000 RPM Cheetah and that one runs at about $1/GB. Some of the 15K drives go for $3/GB.
SSD's are running about $3/GB across the board at the top end, a cost not dissimilar from the top end platters, but they perform much better.
I understand that many people dont want to drop more than $120 on a drive, but many of the vocal ones are letting their unwillingness to do so contaminate their criticism. SSD's are actually priced competitively vs the top performing platter drives.
Re: (Score:2)
They don't beat conventional drives by much at sustained IO.
umm, err?
Which platter drive did you have in mind that performs similar to a high performance SSD's? Even Seagates 15K Cheetah only pushes 100 to 150MB/sec sustained read and write. The latest performance SSD's (such as the SATA2 Colossus) are have sustained writes at "only" 220MB/sec and with better performance (260MB/sec) literally everywhere else.
Re: (Score:2)
Re: (Score:2)
$100 for a terabyte sounds great and all, but you cant get a fast one for that price. You wont be doing sustained writing 120MB/sec writing to those 7.2K drives. You will be lucky to get 80MB/sec on the fastest portions of the drive and will average around 60MB/sec.
That SSD thats pushing 220MB/sec sustained writing is 4x the performance on that one metric, and even faster on every other metric.
Re: (Score:2)
Your benchmarks for 7.2k drives is defiantly outdated. My 2 drive raid-0 is doing 400MB/sec on the fastest part, and 250MB/sec at the slowest with an average of 309MB/sec. These aren't the latest drives, nor are they highend SCSI/SAS drive, just typical seagate desktop drives. My single Intel 80GB SSD is 220-200MB from start to finish.
Re: (Score:3, Insightful)
Which Seagate drives would this be? Those numbers sound very high for typical desktop drives.
Besides sustained sequential speed is one thing but what really gives a responsive "feel" on
the desktop is random access and any one of the post-JMicron-stutter SSDs will stomp even a small RAID of
dual-ported enterprise drives into the dirt on random reads and writes, especially combined with the order of magnitude
faster access time of an SSD.
This study seems deeply confused in a specific way (Score:5, Insightful)
"Even the highest-performance RAID controllers today cannot support the IOPS of just three of the fastest SSDs. I am not talking about a disk tray; I am talking about the whole RAID controller. If you want full performance of expensive SSDs, you need to take your $50,000 or $100,000 RAID controller and not overpopulate it with too many drives. In fact, most vendors today have between 16 and 60 drives in a disk tray and you cannot even populate a whole tray. Add to this that some RAID vendor's disk trays are only designed for the performance of disk drives and you might find that you need a disk tray per SSD drive at a huge cost."
That sounds pretty dire. And, it does in fact mean that SSDs won't be neat drop-in replacements for some legacy infrastructures. However, step back for a minute: Why did traditional systems have 50k or 100k RAID controllers connected to large numbers of HDDs? Mostly because the IOPs on an HDD, even a 15K RPM monster, sucked horribly. If 3 SSDs can swamp a RAID controller that could handle 60 drives, that is an overwhelmingly good thing. In fact, you might be able to ditch the pricey raid controller entirely, or move to a much smaller one, if 3 SDDs can do the work of 60HDDs.
Now, for systems where bulk storage capacity is the point of the exercise, the ability to hang tray after tray full of disks off the RAID controller is necessary. However, that isn't the place where you would be buying expensive SSDs. Even the SSD vendors aren't even pretending that SSDs can cut it as capacity kings. For systems that are judged by their IOPS, though, the fact that the tradition involved hanging huge numbers (of often mostly empty, reading and writing only to the parts of the platter with the best access times) HDDs off extremely expensive RAID controllers shows that the past sucked, not that SSDs are bad.
For the obligatory car analogy: shortly after the début of the automobile, manufacturers of horse-drawn carriages noted the fatal flaw of the new technology: "With a horse drawn carriage, a single buggy whip will server to keep you moving for months, even years with the right horses. If you try to power your car with buggy whips, though, you could end up burning several buggy whips per mile, at huge expense, just to keep the engine running..."
Re:This study seems deeply confused in a specific (Score:5, Insightful)
And we don't have to use Highlander Rules when considering drive technologies. There's no reason that one has to build a storage array right now out of purely SSD or purely HDD. Sun showed in some of their storage products that by combining a few SSDs with several slower, large capacity HDDs and ZFS, they could satisfy many workloads for a lot less money. (Pretty much the only thing a hybrid storage pool like that can't do is sustain very high IOPS of random reads across a huge pool of data with no read locality at all.)
I hope we see more filesystems support transparent hybrid storage like this...
Re:This study seems deeply confused in a specific (Score:4, Insightful)
I'll be interested to see, actually, how well the traditional 15K RPM SCSI/SAS enterprise screamer style HDDs hold up in the future. For applications where IOPS are supreme, SSDs(and, in extreme cases, DRAM based devices) are rapidly making them obsolete in performance terms and price/performance terms are getting increasingly ugly for them. The costs of fabricating flash chips are continuing to fall, the costs of building mechanical devices that can handle what those drives can aren't as much. For applications where sheer size or cost/GB are supreme, the fact that you can put SATA drives on SAS controllers is super convenient. It allows you to build monstrous, and still pretty zippy for loads that are low on random read/write and high on sustained read or write(like backups and nearline storage), storage capacity for impressively small amounts of money.
Is there a viable niche for the very high end HDDs, or will they be murdered from above by their solid state competitors, and from below by vast arrays of their cheap, cool running, and fairly low power, consumer derived SATA counterparts?
Also, since no punning opportunity should be left unexploited, I'll note that most enterprise devices are designed to run headless without any issues at all, so Highlander rules cannot possibly apply.
Re: (Score:3, Insightful)
Re: (Score:2)
Correct me if i am incorrect.
Re: (Score:2)
My (possibly incorrect?) understanding was that 3-4x 7200rpm drives arent a drop-in replacement for a 15k in all situations
Not all but I would think most.
If you are comparing a big array of 15K drives with an even bigger array of 7.2K drives I would think it likely that the application in question is one that is capable of generating large numbers of requests in parallel (most likely some kind of database server).
Re: (Score:2)
All I want to know is who is making RAID cards that cost $50,000 to $100,000? Or is he describing a complete system and calling it a RAID card?
Re: (Score:2)
He's clearly talking about SAN controllers like EMC Clariion or IBM DS5000; if you don't look too carefully you might mistake them for RAID controllers.
Well you know it is confused (Score:2)
Just based on the face that it says "$50,000 or $100,000 RAID controller." Ummm what? Where the hell do you spend that kind of money on a RAID controller? A RAID controller for a few disks is a couple hundred bucks at most. For high end controllers you are talking a few thousands. Like Adaptec's 5805Z which has a dual core 1.2GHz chip on it for all the RAID calculations and supports up to 256 disks. Cost? About $1000 from Adaptec. Or how about the 3Ware 9690SA-8E, 8 external SAS connectors for shelves with
Re: (Score:2)
Where the hell do you spend that kind of money on a RAID controller?
EMC Clariion CX3-80 will run you between $13k and $130k depending on how it is configured.
In other news... (Score:5, Funny)
... researchers have found that putting a Formula One engine into a Mack truck wipes out the advantages of the 19,000 rpm.
Re: (Score:2)
Software RAID? (Score:2)
So does anyone know if this applies to software RAID configurations?
Just curious...
Re: (Score:2)
Re: (Score:2)
This. I'm surprised no one has mentioned it. I don't think there's a RAID controller on the market that supports pass-through TRIM. Which is going to be one hell of a wakeup call when an admin finds the batch job took ten times longer than usual. I had this happen with an X25-M, I had stopped paying attention to the log file's end time for various steps, and one day I woke up to it running past 9AM (from the initial times of taking a mere ten minutes when starting at 5AM.)
Re: (Score:2)
One problem I see is that afaict trim can't just be passed through. The controller needs to take careful steps to handle trim in a way that keeps the raid data consistent.
ZFS sidesteps the whole RAID controller problem (Score:4, Insightful)
If you use ZFS with SSDs, it scales very nicely. There isn't a bottleneck at a raid controller. You can slam a pile of controllers into a chassis if you have bandwidth problems because you've bought 100 SSDs - by having the RAID management outside the controller, ZFS can unify the whole lot in one giant high performance array.
Re: (Score:2)
That's not the problem, the problem is a lot of the high end controllers have 8, 16, 24, etc SAS ports. If you were to plug SSDs into all of those ports, you'd swamp the card, whether you treat the disks as JBOD or let the controller handle it. And the storage vendors who make real nice SANs did the same thing. They have one controller managing dozens of HDDs because their performance is so abysmal.
Re: (Score:3, Interesting)
If you use ZFS with SSDs, it scales very nicely. There isn't a bottleneck at a raid controller. You can slam a pile of controllers into a chassis if you have bandwidth problems because you've bought 100 SSDs - by having the RAID management outside the controller, ZFS can unify the whole lot in one giant high performance array.
If performance is that critical, you'd be foolish to use ZFS. Get a real high-performance file system. One that's also mature and can actually be recovered if it ever does fail catastrophically. (Yes, ZFS can fail catastrophically. Just Google "ZFS data loss"...)
If you want to stay with Sun, use QFS. You can even use the same filesystems as an HSM, because SAMFS is really just QFS with tapes (don't use disk archives unless you've got more money than sense...).
Or you can use IBM's GPFS.
If you really wan
Re: (Score:3, Informative)
I just did. On the first page, I got just one result on the first page relating to an event from January 2008 - Joyent. And they managed to recover their data. I did another search - "ZFS lost my data". One example running on FreeBSD 7.2, in which ZFS was not yet production ready. Other examples existed in wh
Re: (Score:2)
Raid controllers obsolete? (Score:2)
Even the highest-performance RAID controllers today cannot support the IOPS of just three of the fastest SSDs.
In the old days, raid controllers were faster than doing it in software.
Now a days, aren't software controllers faster than hardware? So, just do software raid? In my very unscientific tests of SSDs I have not been able to max out the server CPU when running bonnie++ so I guess software can handle it better?
Even worse, it seems difficult to purchase "real hardware raid" cards since marketing departments have flooded the market with essentially multiport win-SATA cards that require weird drivers because th
Re:Raid controllers obsolete? (Score:5, Informative)
The advantage of hardware RAID, at least with RAID 5, is the battery backup. When you write a RAID stripe, you need to write the whole thing atomically. If the writes work on some drives and fail on others, you can't recover the stripe. The checksum will fail, and you'll know that the stripe is damaged, but you won't know what it should be. With a decent RAID controller, the entire write cache will be battery backed, so if the power goes out you just replay the stuff that's still in RAM when the array comes back online. With software RAID, you'd just lose the last few writes, (potentially) leaving your filesystem in an inconsistent state.
This is not a problem with ZFS, because it handles transactions at a lower layer so you either complete a transaction or lose the transaction, the disk is never in an inconsistent state.
Re: (Score:2)
Just my thought. Hardware RAID adds latency and limits throughput if you use SSDs. On the other hand, server CPUs often have cycles to spare and are much faster than the CPU on the RAID controller. I've yet to see the dual quad cores with hyperthreading going over 40% in our servers.
Now all we need is a VFS layer that smartly decides where to store files and/or uses a fast disk as a cache to a slower disk. Like a unionfs with automatic migration?
ditch the controller (Score:2)
RAID for what? (Score:2)
If using RAID for mirroring drives, well, you must also consider the fail rate of drives, as it is all about fault tolerance, no? It is reported that SSDs are far more durable, so the question should be, what does it take to match the fault tolerance of HDD RAID with an SSD RAID, and only after that, can we truly compare the pros and cons of their performance sacrifices.
On a side note, you can now get a sony laptop that comes equipped with a RAID 0 quad SSD drive.
http://www.sonystyle.com/webapp/wcs/stores/s [sonystyle.com]
Re: (Score:2)
Do SSDs really have a lower failure rate than HDDs? I mean, how many times can it be assumed that I can write to a specific sector on each? I'd be interested in a report in which this has been tested by writing various devices to destruction, rather than by quoting manufacturer predictions.
Don't give me wear levelling arguments, as they assume that I'm not frequently changing all the data on the medium.
Re: (Score:3, Interesting)
A 500GB SSD can be entirely over-written ("changing all the data on the medium") over 10,000 times. No wear leveling needed here. 10K writes is the low end for modern flash.
Lets suppose you can write 200MB/sec to this drive. Thats about average for the top enders right now.
It will take 2,500 seconds to overwrite this entire drive once. Thats about 42 minutes.
So how long to overwrite it 10,000 times?
Thats 25,000,000 seconds.
Thats 416,667 minutes.
Thats 6,944 hours.
T
Re: (Score:3, Insightful)
289 *days* of constant 24/7 writing to use of the flash.
This assumes the case of repeated sequential write to blocks 1 to n, where no wear levelling occurs. Consider that I first write once to 100% of the disk, then repeatedly: write sequentially to the first 25% of the disk n times, then write to the remaining 75% of the disk once. Dynamic wear levelling is out. How is a typical static wear levelling algorithm likely to kick in in a way which prevents an unacceptable slowdown during one pass, while at the same time squeezing out max writes to all physical bloc
Re: (Score:3, Funny)
Re: (Score:2)
MSA are low performance crap anyway. here's a quarter kid, get yourself an EVA
Re: (Score:2)
just put regular spinning disk in it. Point is working for an HP VAR I've had the misfortune to set up many database and middleware systems on MSA and performance is appalling compared to EVA
Re: (Score:2)
A lot of those features are available for a lot less than $100,000. But what you don't get, usually, is the same level of support.
... because it is SSD + RAID (Score:2)
While the ioDrive may offer great performance, I hate their marketing.
http://www.fusionio.com/products/iodrive/ [fusionio.com]
"It's not SSD + RAID, it's solid state memory in parallel channels!"
No, it's not X25-M's on an Adaptec card. However, it is NAND flash with a bunch of parallel channels. It's the exact same idea behind SSD + RAID, it's just above the level that you'll get with "regular" S
Re: (Score:2)
Perhaps the most common is RAID5, because it offers increased performance, with redundancy that can withstand any single drive going down.
Re: (Score:2)
Re: (Score:2)
Obviously it wouldn't be cool if it was really slow, but isn't data redundancy the primary purpose?
Not really, performance has been the driver for the growth of companies like EMC. 99.999% availability is an absolute requirement as well. Say I have built a banking system. It absolutely, positively must be able to handle 30000 transactions per second, and those transactions require data from over 20 TB of account data. The only way to do that with magnetic disks us by lashing a fuck-ton of them together in parallel. Until very recently, only high-end arrays from EMC, hitachi, etc. could do this sort if th