HOWTO: 0.5TB RAID on a Budget 278
Compu486 writes "Inventgeek.com has a new how-to article
titled 'The
Poor Mans Raid Array.' The article details how to make a modular .5 terabyte
Raid 5 array for under $250 (USD), and it all runs on the Mandriva flavor of Linux." Drive prices being what they are, this seems cooler than it is practical. Update: 06/25 23:31 GMT by T : If that's not enough storage, Yeechang Lee writes "Let me show off the 2.8TB Linux-powered RAID 5 array I built for home use a few months ago. I provide lots of details on how I did it, what I used, and the results. The Usenet thread has good followup posts from others, too."
typical? (Score:5, Insightful)
Perfect for slashdot!
Re:typical? (Score:5, Informative)
If you want a high performance system, spend the money to get a small, top-of-the-line drive for your root partition (15k rpm SCSIs are nice, if you have a scsi card - you can get a 9 gig for 30$ including shipping, an 18 gig for 55$), and then put all of your space-consuming files (movies, music, etc) on your cheap bulk storage. Get enough spare ram to have good disk caching. And, of course, choose a good filesystem for small files - ReiserFS works well for me, but there are a lot of good options.
You only need major throughput if you're doing a lot of very long file reads that need to occur at top speed (i.e., not playing video or listening to music; more like what you need for running a large relational database, or being a fileserver on a crazy-fast network). To the "Raid 0" crowd: Does this really fit your disk's typical usage patterns?
Re:typical? (Score:2)
Re:typical? (Score:3, Informative)
Really, if you need that much storage, I would hope your hardware budget is a little bigger than what I allocate for my stuff at home.
Re:typical? (Score:3, Informative)
Raid 0: Not quite double the read and write throughput; somewhat higher latency, but not usually a relevant amount. No redundancy.
Raid 1: Not quite double the throughput; slightly lower latency, but usually not a relevant amount. Redundant.
Raid 5: Very fast read throughput; lowered write throughput; higher latency. Redundant.
Now *thats* redundant. (Score:3, Insightful)
That aside, a decent motherboard will come with a RAID IDE controller, so you could easily just grab a pair of 250 WD caviars. Or go the cheapo route and do maxtor.
Re:Now *thats* redundant. (Score:5, Informative)
http://www.acnc.com/04_01_05.html [acnc.com]
http://en.wikipedia.org/wiki/Redundant_array_of_i
Re:Now *thats* redundant. (Score:3, Funny)
Keep those drives cool! (Score:2)
Keep those drives cool! Mount a fan next to them, using plastic straps for flexibility so as not to vibrate the drive.
I like WD, but now Seagate has 5-year warranties.
Re:Now *thats* redundant. (Score:2)
Thankfully, all four failures occured under warranty. Also, failure in the three total failures was progressive: after each restart, I could copy some (decreasin
Re:Now *thats* redundant. (Score:2)
I've also had good luck with Samsung, though I believe their biggest drive is only 160GB.
Re:Now *thats* redundant. (Score:5, Informative)
We've been selling pretty much exclusively Samsung and Seagate since then. The other *huge* often unmentioned advantage is that they're both much quieter than WD and Maxtor equivalents - Samsung being a little quieter in my experience.
Really, outside of the Raptor line, I see no compelling reason to buy a WD drive. I can definitely agree with the sentiment that I have *never* heard anyone say they hated Seagate drives, especially if you talk to the SCSI freaks out there.
Re:Now *thats* redundant. (Score:4, Informative)
Re:Now *thats* redundant. (Score:2)
Getting it to boot is a bit of a bitch though. You need to use a ramdisk and experiment with LILO an awful lot. LILO also won't work with anything other RAID1 for obvious reasons
Re:Now *thats* redundant. (Score:2)
umm, if you're smart enough to set up raid, you will do backups won't you?
I learned my lesson... (Score:2)
Then the CONTROLLER failed. The drive itself was fine. But Perstor, in the meantime, had gone out of business. Bye bye data.
There's Some Debate on the "I" (Score:3, Informative)
Re:There's Some Debate on the "I" - Very True (Score:2)
Re:Now *thats* redundant. (Score:2)
--
Evan
Re:Now *thats* redundant. (Score:2)
Re:Now *thats* redundant. (Score:2)
at the same time, the MD driver is faster than the cheapo drivers. or so they said last time i had one
Howto (Score:5, Insightful)
Re:Howto (Score:2)
Foodstuff commanders take note!
/. on budget: (Score:3, Funny)
Cool? Naah, old (Score:3, Insightful)
It takes just TWO modern disks to get 1/2 terabyte of space, and not much more ot get them in raid5, plus you can have a compact box (the one in TFA is very boxy and ugly) and a lot less noise and power consumption.
Not impressive. Sorry.
Re:Cool? Naah, old (Score:2)
Re:Cool? Naah, old (Score:2, Interesting)
I know he got the drives on the cheap and all, but still... I've got nearly 1tb in my desktop machine without all the extra work.
N.
Re:Cool? Naah, old (Score:5, Insightful)
And a double failure is all it takes to take out a RAID5.
-Z
Re:Cool? Naah, old (Score:2)
Not a big deal. (Score:5, Funny)
That's a lot of storage.
That's balls-to-the-wall.
I'll take a picture of all the drives stacked up on one another on the desk (5 rows, 4 drives tall).
I take my porn seriously.
Re:Not a big deal. (Score:4, Funny)
I'm serious. And it shoudl be "Informative", you insensitive clods!
Re:Not a big deal. (Score:2)
He's telling the truth! I won't tell you how I know, but... he's using an insufficiently patched Windows XP.
Re:Not a big deal. (Score:2)
Re:Not a big deal. (Score:5, Funny)
With that much porn, I think the last picture
Re:Not a big deal. (Score:2, Funny)
Re:Not a big deal. (Score:3, Funny)
Worse, all of the "media catalogue" programs for Windows crash before they finish cataloguing my collection. I even tried it with iView Media Pro on my Mac and that crashed, too after about a terabyte.
This is a serious issue that, as time goes on, I believe Google will need to address. Maybe some
Re:Not a big deal. (Score:2)
What you need is a gmail-like system. All the files go into a single pool and apply labels to each. Then simply organize them in "searchs" like gmail and Outlook 2003. So the same file would be in "midgets" "bestiality" "strapon" and "poultry", but it would only be a single file!
And you could
Only reason it's 'budget' (Score:4, Insightful)
Re:Only reason it's 'budget' (Score:2)
What am I missing here? (Score:4, Interesting)
Why not just hang a four *large* drives in a workstation with MB that does RAID 1+0? Yeah, it'll cost more than 249, but it won't involve a 50 lbs box of drives..
Re:What am I missing here? (Score:4, Funny)
Yeah, really I'd prefer raid 5 too, but TFA was about a "poor mans" raid array.
More or less on-topic: take a look at this guy's really pointless raid 5 setup [8k.com]. Very cool.
my attempt at RAID... (Score:3, Informative)
This is a bit off-topic, but I want to share my most recent experience with linux-raid
A few months ago, I decided I'd put together a RAID5 system in a dedicated box, to be used as network storage. I put together a Duron 1.6 on an ECS (I know!) K7VTA3, 512mb RAM, a Promise IDE controller, and 4 200GB drives. I figured the kernel-based software raid would be fine for my purposes.
I installed linux to a normal partition, then set up the RAID array. Everything seemed fine. I set up samba/nfs shares and ftp. Files seemed to transfer just fine. But for some reason, if I transfered a large file over the network directly to the RAID, the md5sum would have changed, no matter how I transfered it. To make things even more strange, if I transferred to a non-RAID partition, then directly used mv or cp to place it on the RAID partition, it worked great. Strange.
I never quite figured it out what was wrong, and I scrapped the project, with the intention to try again with some more decent hardware. Any ideas as to what happened?
Re:my attempt at RAID... (Score:2, Informative)
Actually sis beats out nforce on pci bus and ide throughput hands down on the athlonxp platform.
Been there done this tons of times.
I currently have a tyan tiger mp board with a fasttrak 100 & 4 120GB maxtor drives in software raid 5, It's about 40% full or so and I've had zero problems with it. The whole system is underclocked EXCEPT the hard drives (wish I could underclock those).
Re:my attempt at RAID... (Score:2)
Re:my attempt at RAID... (Score:2, Interesting)
I too have this same exact problem and haven't been able to figure out what causes it. The array in question is a 1.6TB software RAID5 array with eight 250GB Maxtor SATA drives on Promise SATA150 TX4 controllers. A few months ago I noticed files that I'd coppied via samba being corrupted once they got to the array. As such I now checksum every file that I dump on it before copying from my windows box. I would say somwhere between 1/5 and 1/10 of the large (350MB+) files I copy to it
Ridiculous (Score:4, Insightful)
A 400 Gig drive (probably of equal or better reliability overall and a warranty) costs about $260 on newegg.
Reminds me of people using 486's as routers/firewalls when you can pick up a Linksys or D-Link for $20 or $30.
Thanks, but no thanks.
Re:Ridiculous (Score:4, Insightful)
Re:Ridiculous (Score:2)
Both of which differences would affect throughput, which is exactly what the OP was talking about...
Re:Ridiculous (Score:2)
Re:Ridiculous (Score:3, Insightful)
Not really. Most drives coming out now have a 1-year warranty (some have 3). Modern drives pack more data into a smaller space, so they are more likely to lose data than older drive. Small imperfections will be more noticeable, and will cause more and greater problems. They are not the quality level of the old seagate SCSI drives used in this setup. Those SCSI originally came with a 5-year warranty. If those SCSI drives are s
Re:Ridiculous (Score:2)
That's a common misconception. RAID 5 will give you greater data reliability in some cases (sudden drive failure, RAID 5 will keep you from going down), but won't help you in many others (eg. accidental rm -rf /, someone rootkitting you just so they can spam people in Brazil (happened to me), accidentally running an SQL UPDATE command without a WHERE clause (also happened to me), etc).
Bottom line, you need incremental backups for data reliability. D
Re:Ridiculous (Score:2)
See, for instance: / [mikerubel.org]
http://www.mikerubel.org/computers/rsync_snapshots
This document describes a method for generating automatic rotating "snapshot"-style backups on a Unix-based sy
Re:Ridiculous (Score:3, Informative)
No manufacturer is giving less than 3 years warranty. [newegg.com]
The 10,000 rpm WD Raptor and all Seagate drives come with 5 year warranty.
I think you wanted to refer to the not so recent attempt by some major players to cut warranty to 1 year. That didn't last long, I guess because their sales must have suffered.
Re:Ridiculous (Score:2)
A 400GB drive isn't redundant, unlike the setup in the article, so if it fails, you've lost 400GB of data. The drives are SCSI, which means they will be incredibly reliable, and a lot more so than a consumer-grade SATA drive. I would accessing large files on that thing would be incredibly fast.
Reminds me of people using 486's as routers/firewalls when you can pick up a Linksys or D-Link for $20 or
Low Tech Routers (Score:2)
Ive also found that those cheap 'home routers' that you get for 50 bucks or less are absolute garbage.
Missing the point (Score:2)
It has always been cheaper to buy a single big disk than it is to buy a raid, but do tell us, how do you expect to:
1. Replace the disk while still allowing access to it's contents?
2. Recover your data after your single disk has failed?
Raid has more advantages than just size, and although it's easy to point out that the storage size could be had more cheaply, that's not the same thing as saying that a hardware controller based RAID 5 system cou
Speaking of which... (Score:2)
That said, is there any similar RAID controller to that of the article (one of which I have lying somewhere) but for IDE PATA/SATA drives? You know, in order to set up a similar project but with 160-200 Gb SATA drives instead?
Regards,
Re:Speaking of which... (Score:2)
If you use Linux, be careful what chipset you are getting as you may struggle with drivers in some cases.
Re:Speaking of which... (Score:2)
Re:Speaking of which... (Score:4, Informative)
Re:Speaking of which... (Score:2)
That being said, 3ware makes some damn nice cards. Adaptec, too, though they're a bit more pricey.
eh? (Score:2, Informative)
Re:eh? (Score:2)
Re:eh? (Score:3, Informative)
Need more... (Score:2)
But shipping and handling as well as heat would make this too much hastle. Why not just get a left over PC, put in a pair of 250GB drives? Cooler, faster and about the same price or less. And if you ever needed to double or triple it many PCs will hold up to 3 drives and a CD-RON for 4 devices. Or if you really need alot, put 3 x 400GB = 1.2 TB. Use Linux for mirroring and Samba for NT sharing. Maybe even put a wireles
Why? (Score:2, Informative)
1) The drives are used. If you want to impress us, do it with new components with warranties (even refurb). Used makes it impractical and unreliable, even moreso because you didn't use hot swap.
2) It is only 500GB. This can be achieved in a RAID5 configuration with 3 NEW UNDER WARRANTY 250GB drives.
3) Heat. This negates the whole "cool" (both figurative and literal) label.
4) Power. Old drives suck up alot of power. Putting alot of them in
Re:Why? (Score:3, Insightful)
Oh, and where did I even mention the RAID controller? I was talking strictly about the array itself.
That setup runs hot, sucks back too much power, and cannot have failed drives swapped out live.
It is not cool, it is not news worthy, and it is not cutting edge.
Price issues... (Score:2, Insightful)
And what about us not so fortunate enough to stumble upon deals like this?
Re:Price issues... (Score:2)
Bah! (Score:3, Informative)
Damn the enviroment, burn those kws... (Score:4, Insightful)
$60 a barrel oil? What $60 a barrel oil? Must be nice not to have to pay your electricity bills...
Only .5 Terabytes? (Score:2)
RAID with 4xIDE disks is better (Score:2)
A more practical RAID is to put 4 large IDE disk drives in an old PC and run software RAID1, to give you two virtual disk drives.
That means that you can't use a CDROM drive, since all IDE ports are used, but you can do a network install using either a boot floppy disk or a USB key.
Re:RAID with 4xIDE disks is better (Score:2)
I've done stuff like this. The best thing to do is to get a 5.25" USB 2.0 external drive enclosure, and put the CD/DVD/CD-RW/whatever drive of your choice in it. The enclosure usually only runs about $25-$30. Many computers can boot from USB now, and modern Linux distros have no problems with USB drives. As a bonus, after you are done installing, you
Re:RAID with 4xIDE disks is better (Score:2)
My shorter HOWTO: (Score:5, Informative)
mdadm --create
View the status of the raidset construction by cat'ing
mdadm --assemble --scan
Re:My shorter HOWTO: (Score:2)
Even better, just send the $250 to me, and I'll send you a _1000_ MB RAID.
Re:My shorter HOWTO: (Score:3, Informative)
OK...let's do the math... 1. Buy 3 250GB EIDE or SATA HD's very cheaply. [pricewatch.com]
(looks up prices) $98.00 * 3 = $294.00.
Reminds me of a friend who keeps insisting that he can build a full-sized house for $10,000.00 if he only had the land.
Re:My shorter HOWTO: (Score:3, Interesting)
I recommend creating one primary partition which is slightly smaller than the fullsize of the harddisk
I've built a ~600GB RAID array for my home video jukebox, and I'd modify your recipe in one way: Rather than creating just two partitions on each drive, create many, then create many small RAID arrays and glue them together with LVM. The result is much more flexible. You can use different partition sets with different RAID levels for different purposes, and it also makes adding additional storage int
Re:My shorter HOWTO: (Score:2)
Re:My shorter HOWTO: (Score:2)
You've got to read the kernel from somewhere before you load it, so you need your bootloader to support your array too.
Um, yeah, the article is not that cool. (Score:3, Informative)
2. Get 4 250GB EIDE drives (cheap anymore!)
3. Get 4 $20.00 CompUSA lockable EIDE drive trays.
4. Get an SMP board + CPUs and slap 'em in there.
Ta-da. One power supply, four quiet drives, one case, software RAID-5 easily swappable with 2 dedicated fans per drive, looks professional, comparatively quiet, with the benefit of included scalable SMP workstation. And
There was a time when a SCSI array of many, many drives in a separate case at 10k RPM was something to lust after at home, but these days it just isn't. You can get close enough at home while saving space, using less power, and getting better overall performance.
Re:Um, yeah, the article is not that cool. (Score:2)
Power, Heat, Noise (Score:2)
This is news? (Score:2)
Motherboard with onboard 4 port sata raid
1G ram
AMD 3000+
ATI 9800pro
Put this all together several months ago for around 1100 bucks, nothing out of this world, No redundancy just straight raid-0 and 960 gigs or so of usable space (I'm just storing music and movies if the drives go south oh well).
My other pc that runs my 57" projection TV has 4x120 and 3x80 gig drives in one bigass volume set (yeah boo hiss) and the machine next to it has 4x160 gig on a 3ware ide raid controller to hold even more
all those used components off ebay... (Score:2)
Hah... You have been outsmarted... (Score:2)
But I
like to buy and build systems I can use for years and years without
having to bother with upgrading, and figure I've made a long-term (at
least 4-5 years, which is long term in the computer world) investment
that provides me with much more than just storage functionality. And
again, $1.46/GB is hard to beat.
Sure, call me in two years from now, when they come out with $5 laser-etched holographic 3d memory cubes, which store an unlimited amount of data in a space the size of a few cubic inches..
Yet another implementation (Score:2)
http://www.schaefer.nu/pics/nikita/ [schaefer.nu]
My 0.9TB *quiet* RAID 5 howto (Score:3, Insightful)
First off, Low-noise is my new religion (with 8 PC's in my office, it makes a huge difference), and secondly I don't belive in skimping... being frugal and practical yes, but cutting quality to save a buck (a la walmart)
So to achive that I acquired the following:
- Antec Sonata Lifestyle case.
- nForce 2 motherboard with out chipset cooling fan (just heat sink)
- ATI Radeon 9200se video card with out cooling fan (just heat sink)
- Mobile Athlon XP 2400+ CPU - 35 watts
- 22 db Socket A Heat sink/Cooling fan unit
- 22 db 12cm fan.
- Gigabit NIC
- 512mb RAM
- Combo optical drive
- Samsung 120gb drive (to hold OS, and work space)
- 3ware Escalade 7504-LP RAID controller
- 4x Maxtor 300gb 5400 RPM Drives (chosen for lower heat output over 7200 RPM) drives
- APC 1000va UPS
So put it all together and you get a system that has a total of only 4 fans in it including the one in the power supply. It is the quietist PC I have. The case has a nice rack to hold the 4 RAID drives with cushions to reduce vibration/noise and mount a 12cm fan draw air directly across them, as well as another at the back to produce decent airflow despite their lower cfm ratings.
It runs cool and very quiet. I can't hear *anything* out of that system if my ears are more than a foot away from it. I can transfer large files like
It wasn't $250, but it's good enough for me to do real production work on and sleep better at night.
So I may not have the fastest possible server, but it's still more than enough
You could replicate using 400gb drives for 1.2TB of storage by trading off for the slightly higher heat of 7200 RPM.
Take it farther... (Score:4, Interesting)
Put that up on
And then going on to mount those luns on another system (say a solaris, aix or another linux box). Instead, I was dissapointed to find out that you took a linux box and created enough software RAID to for a TB or more. If this was done with windows, it would be rejected... so why doing it with Linux make it front page news?
Re:Take it farther... (Score:2)
Logical Volume Manager (Score:5, Informative)
Unfortunately, aside from RAID'ing the volumes or something similar, I haven't been able to find any information on making the system redundant.
Read about it more on TLDP [tldp.org]. It's a very robust system that works well on both servers and desktops.
Re:Logical Volume Manager (Score:2, Interesting)
As mentioned, my 2.8TB setup uses LVM2 on RAID 5 (mdadm, not raidtools). I think anyone building one of these babies would be crazy to not use LVM; why limit your future expansion options?
RAIFs (Score:3, Informative)
Just a thought.
1.5 TB Array on the cheap.... (Score:3, Interesting)
2) Free -- I got a full tower case from my brother in law (no faceplate).
3) Free -- I had a few 120mm fans laying around which I have cooling the drives.
4) $1040 -- 8 Maxtor 250 GB PATA HDs. (8MB cache, 7200 RPM)
5) $215 -- 3Ware 7810 (8 port PATA hardware RAID 5 card).
6) $140 -- APC RS 1500 battery backup. (You don't want the array to suddenly lose power for any reason!)
Total Cost $1395.
What it got me: I have 1400 GB usable redundant storage with a hot-spare. If a drive fails at 1:00am the computer will automatically start the rebuild on the spare drive, and likewise if I'm not home. This was more important than the additional storage. I also know that I can get 40 minutes of power out of the APC if the power goes out. The machine is set up to shut itself down in the event that the battery runs low.
I didn't have to fight with any software configs. The driver is included in the Linux kernel source, and can be compiled into the kernel. I don't have to worry about figuring out SMART data. "tw-cli info c0" gives me easily readable output on all of the drives plugged into the RAID card. It's simple, does the job, is stable as all hell, and was fairly cheap. It would have cost nearly as much to have bought 4 PATA cards (ones not using the flawed silicon image controller) as it cost for the 3ware card off of eBay.
More information here [scatteredbits.net].
Re:Well, it's not without its problems (Score:2)
Mods on crack (Score:2)
Re:Useful? (Score:3, Insightful)
Spam, of course.
Seriously though, there is way more than 500 gigs of free content (including source code) available on the web. If you've got a DSL connection or faster, you can easily fill up half a terabyte.
Re:1Gb for under 1000 and it runs cool (Score:2)
Or did you mean to say TB?
Re:How I built a 2.8TB RAID 5 storage array (Score:2)
I have 2TB at my home as well (well 1.75TB usable), 8 250GB SATA drives, at about 150 each, I actually think I have better price/GB. Anyway, having a huge disk array isn't even geeky its almost becoming a necessity (music, movies, tv shows, home videos), every home needs at least 500GB, probably closer to 1TB, and since drives are so big anymore, its easy to slap 4 or 5 in a case, get a SATA raid card, and fly.