Going Deep Inside Xserve Apple Drive Modules 243
adamengst writes "If you've had an Xserve drive fail, you may have considered saving some money by putting a replacement drive inside its Apple Drive Module. That may be a false economy, though. TidBITS explains why, while pinning Apple down on exactly what goes into Apple Drive Modules and why they cost so much more than bare retail drives."
Article text (Score:5, Informative)
While I hate to copy it, the server being pummeled and reporting errors for 9/10 requests doesn't lead to ad revenue either, so here goes:
About a year ago, we bought an Intel-based Xserve with a pair of 80 GB SATA drives to act as our primary Web server. When the boot drive went flaky on us in October 2008, we were able to recover from the backup on the second drive and off-site backups, if a little shakily (see "TidBITS Outage Causes Editors Outrage", 2008-10-07). But although we were able to bring the machine back online, we didn't trust the drive that had failed. Since the Xserve has three drive bays, the obvious solution was to purchase another drive. Sounds simple, doesn't it? Not so much.
You cannot buy a bare hard drive and insert it into an Xserve, as you can with a Mac Pro (and having just added a drive to my new Mac Pro, I can say that Apple did a stunningly nice job in making it easy to add drives, especially in comparison to the awful approach they used in the Power Mac G5). Instead, Xserves require Apple Drive Modules, which are custom carriers containing drives.
For users accustomed to buying inexpensive hard drives, Apple's pricing on the Apple Drive Modules comes as a bit of a shock. An 80 GB SATA ADM costs $200 from Apple, and a 1 TB SATA ADM costs $450. In comparison, a bare 80 GB SATA drive can be purchased for a measly $35, and a 1 TB drive is only about $100. That would seem to point toward buying a new SATA drive and swapping it into the bad drive's ADM. However, when I started down that path, a number of problems arose, such that I bailed on a quick solution and simply purchased a new 80 GB SATA ADM to replace the bad one.
First, I wasn't sure whether my Xserve had SATA drives, as I thought, because System Profiler on the Xserve shows nothing on the SATA bus, instead including all drives on the SAS bus. (SAS stands for Serial Attached SCSI, and is a high-performance data transfer technology that supports fast SCSI drives and is downward compatible with SATA drives.) After some discussion with knowledgeable folks on the MacEnterprise list and careful reading of the drive details in the SAS section of System Profiler, it became clear that both SAS and SATA drives are shown in the SAS section, with SATA drives having "ATA" as the Manufacturer, and showing "Yes" in the SATA Device line.
Second, once I knew that I had SATA drives in my ADMs, I started investigating if there were any gotchas involved in replacing the drives. There turned out to be surprisingly little hard information about this, with some people having replaced an ADM's drive with no trouble and others experiencing performance or reliability issues. I did find a few discussions about how replacing drives isn't recommended, but giving no solid sources.
Confused, I contacted Apple to discuss why ADMs are so expensive in comparison to bare drives, exactly what an ADM does, what Apple recommends users do with failing ADMs, and whether or not replacing a drive in one is a good idea. That conversation revealed a great deal of interesting information about the ADM and shed some light on what people with flaky ADM drives should do.
Drive Selection -- The most important fact to know about ADMs is that Apple doesn't use just any drives. We've all benefited from the amazingly low cost of storage. But whenever manufacturers compete on price, they cut corners every way they can to reduce costs. Although drive reliability is generally good, everyone who buys bare drives regularly has a drive vendor they refuse to patronize due to bad experiences in the past. (As is often the case, these people all hate different vendors, depending on which one was having a bad run at any given time.)
Since the Xserve is designed to be in constant use - 24 hours a day, 7 days a week, for years at a time - Apple doesn't use the least expensive drives available, since those drives are designed for more normal duty cycles in desktop computers - 8 to 10 hours per day, with variable use during that time. Instead, Apple wor
Re:Here we go again (Score:5, Informative)
The pixie dust is in the controller, not the platter.
Re:Nothing to see here (Score:2, Informative)
Cheap compred to EMC (Score:3, Informative)
Summary fails (Score:5, Informative)
The summary makes it seems that there's no rhyme or reason why Apple charges more for their HDs and why can't the consumer simply replace it with a standard SATA HD. If you RTFA, it goes into a long list of reasons why. Whether you accept Apple's reasoning is another matter.
To begin, XServes use a HD module called ADM rather than simple HDs. On the new MacPros, they also use a module but those modules are designed to replace the HDs inside. For the XServe you apparently can't get a bare drive alone, you have to replace the whole module. The author begins to list the reasons:
And then the author concludes:
Custom Firmware (Score:3, Informative)
Been happening for years.
Back in the late 80's, in addition to my dev job, I admin'ed a Motorola Delta 3600 box. We were looking for a little more space, manual said that it would take a Seagate ST-251N 40MB SCSI drive. So we bought on off-the-shelf.
It wouldn't work. It turns out that Motorola had custom firmware for those 251Ns.
So it's been going on for at least 20 years.
Re:Because Apple says so? (Score:3, Informative)
Because Xserves won't run with commodity drives...
Except that the article clearly says that an XServe will work with them.
Well, that was a waste of 5 minutes. (Score:3, Informative)
Go to any server manufacturer's website (or a retailer if they sell through the retail channel). Dell, HP, IBM, I don't care. Any of them.
Price up equipment sold specifically for servers. Note particularly the price they charge for a larger/faster hard disk.
Go on, I'll wait.
Right, now go onto your favourite cheap & cheerful parts supplier and look at how much they charge for a hard disk.
Is it really the exact same disk with that much price discrepancy? Well, I (along with a lot of sysadmins) would dearly love to believe that it isn't. Whether or not that's true I honestly couldn't say.
What I can say is that if you do go out and buy the cheapest disks you can to populate the server, warranty support from the OEM is going to suddenly become "Oh, you plugged some random disk in? Go away and come back when all the disks are from us". Which starts to look rather expensive rather quickly when the RAID's knackered and you need to resurrect the system as quickly as possible. If your job is on the line, it's soon looking even more expensive, and nobody wants to say "I was sacked from my last job because I cut one too many corners on a system that was critical to the business" in an interview.
It's not so much of a problem for the Googles of this world who write their own applications to live on huge clusters which have component systems being added and removed all the time. Most of us, however, don't have that luxury.
Re:Because Apple says so? (Score:1, Informative)
Actually, Xserves will run with commodity drives. As the server drives filled up to capacity, I always tried to find an replacement match against the list of drives that Apple used in the Xserves but failing that I'd match a newer version. Some were enterprise and some were regular off-the-shelf units. A pair of 400 GB Seagate ST3400832AS that I installed into an Xserve almost 3 years ago are still running without a hitch.
As far as tweaking firmware goes, a lot of that can be done with the drive manufactures disk utilities. The only issues I ever had was that I had to slow down some of the newer drives from 3.0 Gb SATA to 1.5 GB SATA to use in the older Xserves. The "drive specific" rubber grommets did make me laugh - if vibration is a concern at that scale, then just use 'Sorbothane' for all of the grommets.
All that being said, I regularly replace clients drives around the 3 year mark - mainly, because I don't have a lot of faith in the longevity of any drives that are currently manufactured.
Re:Here we go again (Score:2, Informative)
Re:Article text (Score:4, Informative)
Extra hardware in the carrier? Again, show me a net benefit for the extra money.
Dude... he is telling you an ADM integrates with XServe using SMART, skipping read-after-write, requesting more airflow if needed, and adapting to OSX block size. That costs extra on a XServe and in other high end machines, thats all.
And yes, try 15k RPM 24/7 without custom rubbers, see what happens.
Re:Well, that was a waste of 5 minutes. (Score:3, Informative)
It's not just the IT industry: go to Mercedes and ask them for warranty support after you've fitted third party replacement parts. I personally know several people who's Skoda warranties have been voided after they've remapped the ECU: understandable if they were expecting warranty support after engine damage caused by a remap (which is highly unlikely) but they've had warranties turned down for failed seat mounting rails.
Large companies like charging you as much as they can - no surprise there.
Site back up (Score:3, Informative)
TidBITS system guy here. Sorry for the troubles. We had a glitch in our Apache min/max/spare/etc settings that was triggered for the first time by Slashdot traffic. (A combination of a new method to zoom images and AJAX produced a very high set of spawned children for each new visitor.)
Re:Here we go again (Score:3, Informative)
Re:Having worked in the disk mines... (Score:4, Informative)
In theory a SCSI disk should not be much better than ATA but the reality is the best made, more reliable parts go to the high end more profitable products.
There are, typically, huge differences between your average SCSI and ATA disk beyond just manufacturing quality. USENIX paper here [hil.unb.ca]. Differences range from disk interface command richness, to reliability under wider operating conditions, to materials and assembly. They do the same thing, but they aren't even close to being the same thing.
Re:Attempt at refuting summary fails (Score:4, Informative)
Parts of this may be true (it's impossible to say, since they don't specify exactly what they're comparing) -- but even if parts are true, it's misleading at best. In fact, some of it doesn't even seem to make sense. Let's look at real price lists from Apple [apple.com] and dell [dell.com].
First we note that Dell doesn't seem to offer a 73 GB drive at all, so it's not entirely clear what they're comparing. The most likely possibility appears to be Apple's smallest option, a 73 GB SAS ADM ($300) to Dell's smallest, a 146 GB SAS ($349). While it's certainly true that the prices are comparable, it's also true that the Dell drive is twice as big.
For 300 GB SAS drives, the Apple site shows $650 while the Dell site shows $699. While Apple's price is lower, it's certainly not even close to $200 lower. To get a $200 price difference, it looks like they compared the full price of a 300 GB drive for the Dell to upgrade price for the Apple (i.e. the price difference for changing from the stock drive to the 300 GB drive).
For 1 TB hard drives, they have something of a point, but not a very good one. Apple's price for a 1 TB SATA drive is $450, while Dell's is $639. They fail to note, however, that Dell also lists a 1 TB SAS drive (an option not available for the XServe) for $679. Taking this into account, it looks a great deal as if Dell is simply doing their best to encourage their higher-end customers to use enterprise-class SAS drives by offering them at a purely nominal incremental cost over SATA drives.
The original article attempts to portray the situation as Apple offering prices that are at least as good as, and often better than the competition. In reality, there appears to be only one reasonable configuration where the Apple is likely to be competitive: the one using 300 GB SAS. At the low end, the Dell offers twice as big of a drive as the Apple for a purely nominal price difference. For lots of storage, the Apple offers only SATA drives where Dell offers SAS. If you're storing 1 TB of data (or more) the incremental cost of SAS is usually fully justified. There are undoubtedly exceptions, but they're not particularly common.
Re:Its simple. (Score:2, Informative)
Oh you had problems, you just didn't realize it.
I won't argue that the price is justified, its not if your time is worthless.
However, 'server class' drives ARE higher quality than consumer grade drives. Just like Intel has been known to take chips that 'failed' a higher class of tests and rebadge them as slower processors, drive manufactures take drive components that are lower quality for whatever reason and use them in consumer level drives, saving the more reliable parts for server class drives.
You may not have noticed that you were replacing more drives or components then needed, but that doesn't mean it wasn't happening. Your kind of claiming that just because you came out of a coma and had no memory of the car accident you were involved in last month that it didn't happen.
Take the cost of the consumer level drives failing more often so you need more of them, the cost of your time to replace, and the cost of having to play phone tag with several manufactures because you didn't use what they want, and it very quickly adds up to more than just paying the extra price for the better drives. That is unless you work for free, then it may be a better deal for your employer.
There is a LOT more to replacing a drive in a critical system than the drive itself, only an armature wouldn't recognize that.
Let me summarize your post:
Guy without a clue read slashdot article about OEM drive costs
Guy proceeds to provide some anecdotal reference from a course he took in collage from junior admin.
Guy then proceeds to attempt to enlighten everyone on slashdot about his experiences
Everyone on slashdot with any sort of real responsibility at their job, with real data that would actually cause loss of revenue if the data was unavailable for any length of time laughs at the guy as he clearly hasn't actually been an admin in any sort of capacity that matters.
Your collage desktop support job doesn't give you a clue as to what it means to be a real admin, please don't try to pretend you know. Its clear from your post that you have not had to work an environment where reliability is the key performance indicator. Now days, when your entire business depends on the data you have, saving a tiny amount of cash on a cheap drive looks pretty stupid after you spend several hours of phone tag to find out that the RAID is having issues because you bought a cheap drive to 'save money'. The money you lost the company on that one failure could easily cost more in lost work or sales than any amount you're going to save by buying cheap drives throughout your entire life.
There are places to buy the cheapest available and places not to, real admins with real experience no the difference. Just like in every other industry, you have the people that know what they are doing from years of experience and then people like this that don't know what they are doing but think they do because they learned it in collage.
Final thoughts: Forget what you learned in collage, collage is so skewed from the rest of the world that as most graduates are just now starting to realize, you're getting screwed by going and wasting your time with the crap you got fed there by people who don't actually work in the fields they claim to know about.
Re:Nothing to see here (Score:1, Informative)
What a FANBOI sucker !! (Score:1, Informative)
The fanboi will believe anything some apple soda-jerk tells him, and gladly pay more for the "I'm an idiot" baseball cap with the extra special, specially-selected size strap. What an idiot!!