Build Your Own 135TB RAID6 Storage Pod For $7,384 239
An anonymous reader writes "Backblaze, the cloud-based backup provider, has revealed how it continues to undercut its competitors: by building its own 135TB Storage Pods which cost just $7,384 in parts. Backblaze has provided almost all of the information that you need to make your own Storage Pod, including 45 3TB hard drives, three PCIe SATA II cards, and nine backplane multipliers, but without Backblaze's proprietary management software you'll probably have to use FreeNAS, or cobble together your own software solution... A couple of years ago they showed how to make their first-generation, 67TB Storage Pods"
Re:Can't actually store 135TB of data (Score:3, Informative)
A manufacturer's terabyte would be 1,000,000,000,000 bytes.
Re:Can't actually store 135TB of data (Score:4, Informative)
Common usage for the past 50 years has been that, in the context of computer memory capacity, 'tera-" is to be interpreted as 2^40 (with "giga-" being 2^30, and so on). You'll note that I included a sidenote on 'tebibytes" to appease revisionists like you.
PS: It's rather ironic that someone accusing me of bastardizing SI prefixes can't even spell 'terabytes" properly. Unless you're somehow referring to Earth Bytes or something.
Re:Feelin' HOT HOT HOT (Score:4, Informative)
This is nothing new. You've never been in a datacenter before, kid. You can ask a grownup one day and he can take you there and you will feel the heat. And NOISE. No offense, but I think you're one of those gamer kids who builds rigs for max FPS, with esoteric water cooling and silent fans everywhere.
Yeah, no, you don't need to pamper your hardware that much. Even laptop drives work way hot (60C+) for years with no issue.
Most servers are built that way too. The Sun x4500 is extremely densely packed. And there are hundreds running just fine.
Backblaze is speaking about scalability in SF (Score:4, Informative)
If you're in the SF Bay Area check out http://geeksessions.com/ [geeksessions.com] where Gleb Budman from Backblaze will be speaking about the Storage Pod and their approach to Network & Infrastructure scalability along with engineers from Zynga, Yahoo!, and Boundary. This event will also have a live stream on geeksessions.com.
Full Disclosure: This is my event.
50% discount to the event (about $8 bucks and free beer) for the Slashdot crowd here: http://gs22.eventbrite.com/?discount=slashdot [eventbrite.com]
Original blog post (Score:5, Informative)
Here [backblaze.com] is a link to Backblaze's actual blog entry for the new pods 135TB, and here [backblaze.com] is the original 67TB pods. The blog article is actually quite fascinating. Apparently they are employee owned, use entirely off-the-shelf parts (except for the case, looks like), and recommend Hitachi drives (Deskstar 5K3000 HDS5C3030ALA630) as having the lowest failure rate of any manufacturer (less than 1% they say).
I found it kinda amusing that ext4's 16TB volume limit was an "issue" for them. Not because its surprising, but because... well, its 16TB. The whole blog post is actually recommended reading for anyone looking to build their own data pods like this. It really does a good job showing their personal experience in the field and problems/not problems they have. For instance: apparently heat isn't an issue, as 2 fans are able to keep an entire pod within the recommended temperature (although they actually use 6). It'll be interesting to see what happens as some of their pods get older, as I suspect that their failure rate will get pretty high fairly soon (their oldest drives are currently 4 years old, I expect when they hit 5-6 years failures will start becoming much more common.) All in all, pretty cool. Oh, and it shows how much Amazon/ Dell price gouges, but that shouldn't really shock anyone. Except the amount. A petabyte for three years is $94,000 with Backblaze, and $2,466,000 with Amazon.
P.S. I suspect they use ext4 over ZFS because ZFS, despite the built in data checks, isn't mature enough for them yet. They mention they used to use JFS before switching to ext4, so I suspect they have done some pretty extensive checking on this.
Re:7K for software raid? and why a low end cpu? (Score:5, Informative)
Hardware RAID controllers are stupid in this context. The only place they make sense is in a workstation, where you want your CPU for doing work, and if the controller dies you restore from backups or just reinstall. Using software RAID means never having to try to get a rebuilder software to convert the RAID from one format to another because the old controller isn't available any more, or because you can't get one when you really need one to get that project data out so you can ship and bill.
Re:Can't actually store 135TB of data (Score:5, Informative)
Hitachi:
"Capacity - One GB is equal to one billion bytes and one TB equals 1,000GB (one trillion bytes) when referring to hard drive capacity."
Western Digital:
"As used for storage capacity, one megabyte (MB) = one million bytes, one gigabyte (GB) = one billion bytes, and one terabyte (TB) = one trillion bytes."
Seagate (PDF product sheets):
"When referring to hard drive capacity, one gigabyte, or GB, equals one billion bytes and one terabyte, or TB, equals one trillion bytes."
So no, no and more no. Sometimes there really should be a "-1, Wrong" moderation...