Slashdot Log In
RAID Vs. JBOD Vs. Standard HDDs
Posted by
kdawson
on Mon Jun 04, 2007 07:21 PM
from the media-center-storage dept.
from the media-center-storage dept.
Ravengbc writes "I am in the process of planning and buying some hardware to build a media center/media server. While there are still quite a few things on it that I haven't decided on, such as motherboard/processor, and windows XP vs. Linux, right now my debate is about storage. I'm wanting to have as much storage as possible, but redundancy seems to be important too." Read on for this reader's questions about the tradeoffs among straight HDDs, RAID 5, and JBOD.
At first I was thinking about just putting in a bunch HDDs. Then I started thinking about doing a RAID array, looking at RAID 5. However, some of the stuff I was initially told about RAID 5, I am now learning is not true. Some of the limitations I'm learning about: RAID 5 drives are limited to the size of the smallest drive in the array. And the way things are looking, even if I gradually replace all of the drives with larger ones, the array will still read the original size. For example, say I have 3x500gb drives in RAID 5 and over time replace all of them with 1TB drives. Instead of reading one big 3tb drive, it will still read 1.5tb. Is this true? I also considered using JBOD simply because I can use different size HDDs and have them all appear to be one large one, but there is no redundancy with this, which has me leaning away from it. If y'all were building a system for this purpose, how many drives and what size drives would you use and would you do some form of RAID, or what?
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Two words: RAID 0 (Score:5, Funny)
Nuh-Uh (Score:5, Funny)
Raid 0 won't protect you, man!
Parent
Re: (Score:3, Funny)
Perhaps a Post-It note on the monitor to remind myself.
Re:Nuh-Uh (Score:5, Funny)
Freaky, that's the exact excuse I used for failing my CS binary course
Parent
Re:Two words: RAID 0 (Score:5, Funny)
For the love of God and all that's holy will someone mod this 'Funny' instead of Informative? I get the joke, but there's always somebody who won't!
(Then again... maybe people who won't oughta make a 10 disk RAID 0, hell mod it insightful sucka!)
Parent
Re:Two words: RAID 0 (Score:5, Funny)
I would, of course, be using a different 10-drive raid-0 pack to record the tragedy -- but I'd be safe because it's my disk pack (which makes it impervious to catastrophic failure).
Parent
Re:Two words: RAID 0 (Score:5, Funny)
Parent
Re:Two words: RAID 0 (Score:4, Funny)
Parent
Re:KISS it (Score:5, Informative)
3ware [3ware.com] made some pretty good cards.
Parent
Re:KISS it (Score:5, Insightful)
But the minute or so of uptime you get by not having to power down the computer is more than made up when the controller chip on your beautiful RAID controller sizzles. Using Linux software RAID lets you plug the drive(s) into another computer of a completely different chipset, boot up, and continue operations as though nothing had ever gone wrong. IMHO, this is far preferable to the effective lock-in presented to you by hardware controllers.
For me, it's all 100% software RAID 1.
Parent
Re:KISS it (Score:5, Informative)
Eh?
LVM [wikipedia.org] and RAID [wikipedia.org] are orthogonal solutions, and don't do the same thing. LVM will let you make a single larger partition out of a number of real partitions, and before anyone says that's the same as RAID0, I should point out that RAID0 is not a real RAID level (as it has no redundancy). The circumstances for failure for LVM and RAID0 (JBOD too) are basically the same - if one part fails, you will quite possibly lose the whole lot.
As for hardware RAID, I would not necessarily recommend that either, as it moves the single point of failure without resolving the problem. Replacing a broken controller with something compatible some years down the road can prove impossible, especially with onboard controllers. There's also the fact that a number of RAID controller cards are buggy and others do most os the work in software drivers anyway! Performance is also no longer a reason to use a pure hardware RAID solution, especially now that multi-core machines are available cheaply.
Hot-swap is still someting that requires a good hardware solution, but that's about it. Good (and well supported) RAID products cost good money too, and for most of us it's just not worth doing - better to use software RAID, buy more RAM, and pocket the rest.
-- Steve
Parent
Re:KISS it (Score:4, Informative)
Parent
Re:Two words: RAID 0 (Score:5, Insightful)
I.e. 3 500 GB drives in a RAID 5 doesn't give you 1.5 TB. (RAID 0 dose that). With RAID 5 you only get 1 TB.
Parent
Do some research first? (Score:5, Informative)
Re:Do some research first? (Score:5, Funny)
Nonsense! Everything you need to know is in the RAID 5 song:
(My friend Rich actually came up with this. I like him too much to slashdot him, though.)
Parent
Duh (Score:5, Insightful)
That said, RAID is not a replacement for proper backup. RAID is just a first line of defense to avoid downtime.
Re: (Score:3, Insightful)
A good point. Consider, though, that most people don't run terabyte-size tape backup at home. It's not like it's business critical data, so RAID-5 is probably sufficient.
Re: (Score:3, Informative)
Re: (Score:3, Insightful)
Re:Duh (Score:5, Funny)
Hell yeah.
Cool but untested tech is always best recommended to others before you try it. Preferably LOTS of others.
Parent
Re:Duh (Score:4, Insightful)
RAID is just a first line of defense to reduce downtime.
Parent
Don't worry about losing your media files (Score:5, Funny)
Re: (Score:3, Insightful)
Get what you need for *NOW* not for later (Score:5, Insightful)
With computers, the stupidest thing you can do is spend extra money to prepare for your needs for tomorrow. Buy for what you need now, and by the time you outgrow it, things will be cheaper, faster and larger.
By the way RAID 5 is a pain in the ass unless you have physical hotswap capability, which I highly doubt.
Re:Get what you need for *NOW* not for later (Score:4, Interesting)
With recent kernels, you can hotswap drives on nvidia sata controllers (common onboard). I believe several other chipsets had support for this added in recent kernels too. Then you can swap drives live and rebuild as needed.
One more important note - if you're using more than about 8 drives (I personally recommend 6), I would use raid 6 instead of 5. You often get read errors from one of your "good" drives during a rebuild after a single drive failure. Having a 2nd parity drive (that's what raid 6 gives you) solves this problem.
Parent
Re: (Score:3, Insightful)
500 GB isn't that much space any more. If he's thinking of making an HDTV MythTV box, for instance, full-res HDTV streams will require a lot of space to store in real-time. It would probably be too computationally intensive to recode them into MPEG4 on the fly.
RAID 5 is damned easy. (Score:5, Interesting)
Until another few years go by and you want to buy more storage. Then you're basically stuck with doubling it, clumsily -- or migrating away and essentially throwing out the old drives.
RAID 5 is better in the short run. Even with a three disc array, you're getting more storage for your money, and you can always restripe it onto a fourth disc.
It's not all porn, and some of it is high def, in h.264. And I don't even edit videos, I just watch 'em.
That is true. However, I would fill a terabyte easily, and right now, I'm guessing it's cheaper to buy three 500 gig drives than two 1 tb drives.
You highly doubt he's got SATA?
The one thing I will say is, either have another disk (even a USB thumb drive) to boot off of, or do some sort of RAID1 across them. You almost certainly want software RAID on Linux, and you don't want to try to teach a BIOS to boot off of your array.
Parent
Re:SCRUB your arrays! (Score:4, Informative)
Parent
Wait a sec... (Score:4, Funny)
Why not the "windows XP vs. Linux" bit? Do you want 100 responses or 1000?
Media Server? (Score:5, Funny)
Re:Media Server? (Score:5, Funny)
Only if you make all your network file shares pubic.
Parent
Linux, raid5, LVM on top, can use extra capacity (Score:5, Informative)
If you buy 1TB drives further down the road, here's what you do- With each disk, create a partition identical in size to the partitions on the smaller disks, then allocate the rest of the space to a second partition.
Join the first partition of the disk to the existing RAID set. Let it rebuild. Swap the next drive, etc. etc. Then once you've done this switcharoo to all the drives, create another raid set using the 2nd partition on your new disks--call it
Take that
Just be sure that any replacement drives you have to buy... you must partition them out similarly. I'd recommend pulling back on the partition sizes a bit, maybe 5%, to account for any size differences between the drives you bought right now and some replacement drives you may purchase later on which might be slightly lower in capacity (different drive manufacturers often have differing exact capacities).
Linux, RAID 5, md (Score:5, Informative)
Go Linux. The Linux MD driver allows you to control how you RAID- over disks or partitions. there are advantages. We will discuss.
First, don't get suckered into a hardware RAID card. They are *NOT* really a hardware card- they rely on a software driver to do calculations on your CPU for RAID5 ops. Software RAID is JUST AS FAST. Unless you blow the big bucks for a card with a real dedicated ASIC to do the work, you're fooling yourself.
Now, you want to go Linux. By using the md driver, you can stripe over PARTITIONS, and not the whole disk. By doing this, you can get MAXIMUM storage capacity out of your disks, even in upgrades.
Say you have 3 500GB disks. You create a 1TB array, with 1 disk as parity. On each of these disks is a single partition, each the size of the drive. Now, you want to upgrade? SURE! Add 3 more disks. Create three partitions of EQUAL size to the original, and tack it on to the first array. Then, with the additional space, you can create a WHOLE NEW array, and now you have two seperate RAID5's, each redundant, each fully using your space.
Another advantage with MD is flexibility. In my setup, I use 5x 250 drives right now. On each is a 245GB partition, and a 5GB partition. I use RAID1 over the 5's, and RAID5 over the rest. Why? Because each drive is now independently bootable! Plus, I can run the array off two disks, upgrade the file system on the other 3, and if there's a problem, I can always revert to the original file system. So much flexibility, it's not even funny.
I recommend using plain old SATA, in conjunction with SATA drives, and just stick with the MD device. For increased performance, watch your motherboard selection. You could grab a server oriented board, with dedicated PCI buses for slots, and split the drives over the cards. Or, you can get a multiproc rig going, and assign processor affinity to the IRQ's- one card calls proc 1 for interrupts, the other card calls proc 0. If you have multiple buses, then performance is maximized.
The last benefit? Portability. If your hardware suffers a failure, then your software RAID can move to any other system. Using ANY hardware RAID setup will require you to use the EXACT same card no matter what to recover data. Even the firmware will have to stay stable or else your data can be kissed goodbye.
Windows? Forget about it.
Good luck!
Re:Linux, RAID 5, md (Score:4, Interesting)
I have 6 320 GB disks. The /boot partition is RAID 1, mirrored across all 6 (yes, 6) devices, and grub is configured so that I can boot from any one of them. The rest of the partitions are RAID 6, with identical allocations on each disk.
There's a RAID HOWTO for Linux: it tells you everything you need to know about setting it up.
Parent
Re:Linux, RAID 5, md (Score:4, Informative)
Higher RAID-levels are not always THE ultimate solution and depending on your solution you might just have to go for a non-secure RAID level (RAID0) for large media storage with nightly snapshotting to your backup device. Usually it's not all that bad to lose a single day worth of data and if it is for these applications, use RAID10 or so. I do it as follows: get media on RAID0 (HD streams are large and fast on 10k drives) and then as soon as job is done, I copy it to the storage area which is RAID5 on cheap SATA storage and then a nightly copy to an offline backup station (HW-RAID5 with ATA100) of the data I want to keep.
Parent
Re:Linux, RAID 5, md (Score:4, Informative)
I recommend Gentoo to do this with. Other distro's dont include the latest mdadmtools required to manage and migrate RAID5 md devices. Ubuntu is catching up, I believe.
Here are some places to start:
http://gentoo-wiki.com/HOWTO_Gentoo_Install_on_So
http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2
http://linas.org/linux/Software-RAID/Software-RAI
http://linas.org/linux/raid.html [linas.org]
http://evms.sourceforge.net/ [sourceforge.net]
http://www.tldp.org/HOWTO/Software-RAID-HOWTO.htm
Parent
How the hell did this make the front page? (Score:4, Insightful)
This place has really gone downhill. I thought Firehose was supposed to stop stuff like this, not increase it!
Anyways, just to be slightly on topic: there's no one answer to this question. It depends on your budget, your motherboard, your OS, and, most importantly, your actual redundancy needs. This kind of thing is addressed by large articles/essays, not brief comments.
Planned obsolescence (Score:4, Insightful)
Here's the way I do it (for a home storage server, not a solution for business-critical stuff):
Examine current storage needs, and forecast about two years into the future.
Build new server with reliable midrange motherboard, and a midrange RAID card. These days you could do with a $100-$300 four-port SATA card, or two.
Add four hard disks in capacities calculated to last you for two years of predicted usage, in RAID 5 mode. Don't worry about brand unless you know for a fact that a particular drive model is a lemon.
Since manufacturer's warranties are about one year, and you may have difficulty finding an unused drive of the same type for replacement, buy two more identical drives. These will be your spares in the event of a drive failure.
When the two years are up, you should be using 80 to 90 percent of your total storage.
At this point, you build an entirely new server, using whatever technology has advanced to at that time.
Transfer all your files to the new server.
Sell your entire old storage server along with any unused spare drives. A completely prebuilt hot-to-trot RAID 5 system, with new matching spare disk, only two years old, will still be very useful to someone else and you can recoup maybe 30 to 40 percent of the cost of building a new server.
Lather, rinse, repeat until storage space is irrelevant or you die.
Go RAID 5 BUT with real hardware.... (Score:3, Insightful)
Again, doing it correct up front takes care of upgrade options down the line. It also gives you room to do monster sized volume if you ever need that much space (8 disk array). Most of these RAID solutions are also OS independent, so if you want dual boot, the volume would be recognized by Windows, Linux, Unix, BSD, etc., and you are also not dependent on using the exact same motherboard if you motherboard dies or wants to be upgraded (you would lose all your data if you use the built in RAID on the motherboard when changing to a new motherboard other then the exact same model).
These better cards also can be linked together (i.e. you always get a second card assuming your motherboard has a slot for it, and add more disks to the array that way as well).
One acronym - ZFS (Score:3, Interesting)
Your data should be perfectly safe, with raidz2 can lose up to 2 drives, without data loss.
Performance requirements (Score:5, Insightful)
The advantages of RAID 0 versus RAID 1 versus RAID 5 have already been covered in detail, here, and in many books and websites.
However, allow me to address the issue of how they relate to a media center:
Firstly, when you say "media center/media server", do you mean "I just want to build myself a kickass Tivo?", or do you mean "I want to serve video for everyone in my frat house, simultaneously?"
If the former, consider that Tivos ship with 5500 RPM drives for several reasons:
1) They're cheaper than faster drives
2) They run cooler than faster drives
3) They run quieter than faster drives
4) They use less power than faster drives
5) They're more than fast enough for streaming a single video to your TV while recording another
Long story short, if you're just building a "free" Tivo with a kickass drive array, performance is *not* an issue. Keep in mind that if you're building a set-top box of sorts, the low heat and low noise features are *very* big benefits. You probably want RAID 5, and/or JBOD.
If, however, you're planning on serving video to more than a handful of stations simultaneously, you may need to consider performance. This is a vote for RAID 0 and/or RAID 10.
Now, the second axis: How important to you is this data? Really?
I've got over 300 gigs of drive space on my Tivo. Most of it is the last two weeks of television reruns (Scrubs, 6 copies of last Thursday's Daily Show, etc.), movies I recorded but won't watch, etc. There are about 10 gigs (3%) of video on there that's been saved for a few months, and frankly, I couldn't tell you a single thing on there that I'd miss if my drives went belly up tomorrow. So: do you *really* need to save all those Seinfeld reruns on a highly-redundant storage array? How *much* of the stuff on the server do you really need to keep?
Assuming it's less than 50% (in the Tivo scenario, it probably is), consider using JBOD for most of your storage, and maintaining a single backup drive, or small backup drive array. Or just backing up the good stuff to DVD.
In summary: If you're just building a Tivo, you probably don't really need the performance, or redundancy that RAID offers.
Infrant X-RAID is the solution (Score:5, Interesting)
Buy an Infrant RAID with the two biggest drives you can afford. Let's say two 750GB drives or whatever's on sale that week. It starts out acting as RAID-1 with the drives mirroring. So you have 750GB of "safe" storage. Now you add another 750GB drive. Okay, now you have 1500GB of storage with one of the drives acting as parity drive (RAID-5). Add a fourth drive and how you have 2250GB of "safe" storage. Now you come back and just replace one of the original 750GB drives witha 1TB drive. Do you get extra capacity? No...not initially. But the drive is fully formatted and integrated as X-RAID. What this means is that eventually after you have piecemeal or onesie-twosie upgraded all four drives, suddenly the X-RAID resizes itself to match the capacity of the new drives with no transfer or downtime. So in theory if you wanted to upgrade your RAID, buy four 1TB drives, swap them out one at a time (letting each one rebuilt the array) and then at the end you'll have 3TB RAID isntead of the old 2250GB RAID and all the data intact.
http://www.infrant.com/products/products_details.
I have three ReadyNAS units and love them to death. They are a little fussy about drive temperatures (I guess that's a good things but, I may get like 40 emails during the course of the day about it and it's not like I'll drive home from work to turn up the A/C in my house). My only sadness is that Infrant doesn't have a higher capacity unit than four drives (oh please oh please, eight drives with a RAID-6 type protective hotspare in one nice rack-mountable unit would be my ultimate dream).
-JoeShmoe
.
Re:RAID (Score:4, Funny)
Actually, the failed hard disk will personnaly walk to you.
Parent
mdadm (Score:3, Informative)
If it is a Linux server, you're already using mdadm, which has a monitoring daemon with e-mail notification.
LVM is your friend (Score:3, Informative)
Not at all, these days one does have better options than rebuilding a blank array. Read up on LVM, it is powerful stuff.
Replace the drives in the array one at a time, allowing time for the array to rebuild. Then you can grow the volume to make use of the extra capacity. Yes it will require some planning and will probably take a week to slowly merge in the new set of drives, but it sure beats a bare metal restore b
Re: (Score:3, Interesting)
Yep. And if you boot something like Knoppix you can keep the OS on cdrom and storage on the raid device. Samba config goes on a usb key. I have two servers in a corporate environment running software raid 5 and booting knoppix. Updates are nearly impossible, but you can keep the updates on the usb key (tzdata) and untar right over the top of UNIONFS after boot. Either that or just download a fresh Knoppix version (I've gone through 3 versions now). The software raid in Linux i
Re:go for RAID-5 (Score:4, Informative)
I concur. You would be crazy not to have redundancy--without out it one disk failure will pull down a good chunk of your data.
As for growing the array. From what I understand (and I have not tested this) you can grow the size of the array if you replace all the disks (one at a time with a resync obviously). Also, as of linux 2.6.17, you can add a disk to the RAID and grow it that way.
I would caution against making your array very large (either in disks or in space). Consider the case of a 3 disk RAID array where each disk has a probability of failing in any given second of 10^-10 (you would do this analysis using the reconstruction time of your array as the time window). The probability for any two drives not failing is (1-10^-10)^2. The total number of 2 drive pairs in a 3 disk RAID is 3, thus the probability of the array not failing in any given second is (1-10^-10)^6=0.99999999940. Over a period of five years, the overall probability of no two drives failing is (1-10^-10)^(6*157680000)=0.909729. If you increase the array size to 10 disks, the overall probability of two drives failing is 0.241927 (the number of 2 drive combinations is 45 so you replace the 3 with 45).
Parent
Re: (Score:3, Informative)
Why? Linux software RAID (md) does a fine job with excellent performance, assuming you are not saturating the PCI bus (solution: use PCI Express or PCI-X instead). With sufficent bus bandwidth, software RAID outperforms the majority of soft RAID (rocketraid) and hardware RAID controllers.
You can do this with md wi
You can mix raid drive sizes, with planning. (Score:4, Interesting)
I had 4 300GB drives, and 2 200GB drives.
I broke them up into 100GB partitions, and layed out the RAID arrays:
A1 = [D1P1 D2P1 D3P1 D5P1]
A2 = [D1P2 D2P2 D4P1 D6P1]
A3 = [D1P3 D3P2 D4P2 D5P2]
A4 = [D2P3 D3P3 D4P3 D6P1]
Then I concatenated the arrays together, giving a little less than 1.2 TB of space from 1.6 TB of drives; if I had just RAID'd the 4 300 gig drives, and mirrored the 200's I would have only had 1.1 TB available, and the drive accesses would be imbalanced.
I could also grow the array, since it was built as concatenated, so later when I got 4 400GB drives I raided them then tacked them on for 2.4 TB total.
Parent
Re:Is Google broken today? (Score:5, Funny)
Not hard at all... [killsbugsdead.com]
Parent