AOL Spends $1M On Solid State Memory SAN 158
Lucas123 writes "AOL recently completed the roll out of a 50TB SAN made entirely of NAND flash in order to address performance issues with its relational database. While the flash memory fixed the problem, it didn't come cheap, at about four times the cost of a typical Fibre Channel disk array with the same capacity, and it performs at about 250,000 IOPS. One reason the flash SAN is so fast is that it doesn't use a SAS or PCIe backbone, but instead has a proprietary interface that offers up 5 to 6Gb/s throughput. AOL's senior operations architect said the SAN cost about $20 per gigabyte of capacity, or about $1 million. But, as he puts it, 'It's very easy to fall in love with this stuff once you're on it.'"
Re:AOL? (Score:5, Informative)
It is called HDSL... (Score:4, Informative)
You can read more about that here:
http://www.google.com/search?q=High-Speed+Data+Link [google.com]
Really? (Score:4, Informative)
My impression has been that this has been what has been going on for some time now with all the larger database operations, and one of the reasons why SSD have not yet come down in price is that all the best units and tech are going to the big companies as fast as they can get it from the manufacturers. I wouldn't be surprised to see someone like Google saying something like "yawn, 50TB" and saying that they have PETABYTE versions already out there.
If you run a Database of any size, especially ones with large read to write ratios, SSD would only make things faster. And speed counts.
Re:AOL? (Score:4, Informative)
Re:AOL? (Score:5, Informative)
AOL is Advertising.com and some flagship sites. And yes, they still have dialup users. The access business is steadily decreasing, but its pretty profitable since they basically stopped upgrading it and now just sort of run it.
If they maintain their current path, yes, they will eventually disappear and fail, but the process is much longer than you might think. Not all of their acquisitions were as retarded as Bebo.
What they probably need the SAN for is the Advertising business. That is profitable and requires a shitload of storage. They don't need that for their websites.
Finite number of program-erase cycles? (Score:3, Informative)
I wonder what the read/write rating is vs. a hard disk?
Wikipedia puts flash at 1,000,000 program-erase cycles [wikipedia.org]
Re:AOL? (Score:4, Informative)
Neither. AOL separated into its own company again.
As a very casual observer it seems like the entire TW/AOL debacle could not have been mismanged worse - well I guess both companies could have gone titsup, but that's about it. TW vastly overpaid for AOL when AOL was at its peak (160 billion dollars). Then, just as AOL had started to climb out of the bottom they spun it off for a song ($2.5 billion). Since then AOL has been doing a decent enough job of reinventing itself as "new media" company - the kind of thing TW seems to be struggling with.
That's why corporate CEO's get the big bucks though!
Re:Sas bandwidth constrained??? (Score:3, Informative)
Google found differently in their massive hard drive survey [engadget.com]... sometimes drives would just up and die with no SMART warnings. Also the most common SSD failure-case is lack of writes, at least you can retrieve data off the drive as opposed to a completely opaque device if the platter is frozen.
Re:interface? (Score:1, Informative)
If you look at the Violin Memory website (vmem.com) you can see a Memory Array presents a PCIe interface. What AOL is doing is using a Violin SAN head which connects to multiple Memory Array's and then presents Fibre Channel to their EMC VPLEX (Storage Virtualization Layer) and then they can provision to their individual internal customers as needed.
I think there is some confusion in what is available from the VPLEX point of view which can aggregate multiple Memory Arrays to present whatever performance profile they want - not what is available from an individual box.
Comment removed (Score:2, Informative)
Re:DB Performance Issues (Score:1, Informative)
Also, even if the software/dba guys can tune all the apps* there is the opportunity cost of having them spend time on a problem that is not specific to AOL's expanding business (delivering content in attractive / magnetic ways) and spend it on a technical problem that may actually just be a growth problem.
If you're the manager in charge of this decision, you embrace the crisis, spend the cash to get yourself some new capacity and keep rolling. If you want to tune the apps/dbs as well, you do that in parallel, install the new hardware and get the performance bump that validates your strategic choice. Then you roll out the performance improvements and make everyone even happier.
* Remember: they're serving up data for an unknown number of applications here; tuning might encompass a huge amount of db profiling, multiple application teams, and god knows how many load interdepencies, both technical and interpersonal.
Re:Sas bandwidth constrained??? (Score:3, Informative)
Google found differently in their massive hard drive survey [engadget.com]... sometimes drives would just up and die with no SMART warnings. Also the most common SSD failure-case is lack of writes, at least you can retrieve data off the drive as opposed to a completely opaque device if the platter is frozen.
Yeah, I've seen quite the opposite. Let me preface this with saying that I'm strictly talking about consumer and midrange drives, I've seen very few SCSI and SAS drives die without warning.
In the past 10 years, in a company with about 200 nodes, I can literally count on one hand the amount of hard drives that have given any SMART warnings leading up to their imminent failure. They pretty much always die while the OS accumulates log entries of bad blocks and I/O errors. Most of the time it was either death by shock, or death by manufacturer defect (Maxtor!). The former, SSD drives are pretty much immune to BTW. I would prefer an SSD in a road warrior or college student's laptop any day over a conventional HDD.
Re:AOL? (Score:2, Informative)
AOL bought TW. It was a very shrewd move for AOL, because TW had a much higher intrinsic value to set a floor on the stock price when the internet bubble burst.
Re:What? (Score:5, Informative)
I have a feeling AOL just spend $1,000,000 on something they didn't really need as well.
They admitted as much in the article. They decided that it was cheaper to improve the hardware throughput than to spend the money on developers to try to trim the demand. They were also probably losing money by not meeting SLAs and a quick fix was cheaper in the long run. They also reduced power and cooling requirements as well, so there may be some long term payback there as well. The free publicity certainly didn't hurt either
Re:AOL? (Score:3, Informative)
This isn't true - what the AC says is true. TW was bought by AOL when AOL could leverage its 160 billion dollar fairy-dust value into tangible assets. If they hadn't done so they would have been gone years ago. It was a brilliant move by AOL and at the time, TW thought it was a great deal as well. TW got suckered, as we know now. But in that day and age it looked like a good move: AOL had the internet savvy, TW the IP - combine them and rule the Internet. Ofcourse, that didn't quite go as planned.