Turnkey Linux RAID Solutions? 110
Total-Gig-Age asks: "I want to buy an expandable RAID system for home storage of large media files (music, film, and photo). I'm absolutely unwilling to rely on optical discs (bit rot, not always online) and un-RAID-ed hard drives (unsafe: if it fails, you're screwed). The thing is, I don't have time to shop for and configure a RAID system myself, and I want a turnkey solution that will just work out of the box. I'm aware of Apple's XServe, but $6000 for 1 TB is just too expensive. What are my best options if I want to buy an open source system that I can maintain and upgrade if need be? Any recommendations on a full set of components, so that I don't have to spend a week shopping? Trustworthy online companies? Can I trust a local store to do it for me? Is it better to keep the server as a separate machine? Finally, how much should I expect to spend if I want something that doesn't suck (for 1TB say)? I can find plenty of info on how to set up RAID on the Internet, but I just want to be told what to buy so I can get on with other things, even though I could probably handle setting the whole thing up myself if I had to."
Re:Build it. (Score:2)
Re:Build it. (Score:1)
http://www.linuxdocs.org/HOWTOs/Software-RAID-HOW
Hardware forums if you need advice
http://forums.devhardware.com/
Even Toms Hardware offers help with Windows RAID. All you have to do is look for the help it's out there in forums and how to websites. I'm sure you can even find decent books on the subject.
Re:Build it. (Score:1)
Re:Newegg.com (Score:1, Offtopic)
Re:Newegg.com (Score:2)
Hell one time I accidently ordered too big of a heat sinks and bent a capacitor which fried the motherboard. They took it back for credit even after I told them the truth. (Of course don't expect
Re:Newegg.com (Score:2)
$6k/1TB too much? (Score:1)
is $6k just too much, or is 1TB just not enough disk space for $6k? in other words, if you could get 4TB for $6k, would would consider it?
Re:$6k/1TB too much? (Score:3, Insightful)
Re:$6k/1TB too much? (Score:1)
otoh, if he's ok building it himself, then $6k is a lot. but for something out of the box, perhaps not
Re:$6k/1TB too much? (Score:2)
8 port 3Ware 9500 controller : $500
400G Seagate sata disk: $350
400GB * 3 = 1.2TB
you we need 4 disks and one controller for 1.2TB of raid5
500 + 350*4 = $3400
add the motherboard and processor ($600), and you have a system for $4K
You know why it costs more, right? (Score:5, Insightful)
Quality comes at a price. Everybody learns this eventually. With a DIY solution, the price is your time. You can make something really great if you're willing to burn a weekend or two on it.
If you're buying something, you can have "moderately expensive, stable, and really limited", "really cheap, but likely to fall apart or catch fire", or "really expensive and really flexible".
The other thing that you run into with a sommercial system is the difference between home and business requirements. For a business with a machine room, dust, humidity, and temperature are easy to control. A noisy unit is fine. Under your desk, temperature and dust build-up will be a problem, and the thing'll sound like a jet engine.
OT: definition of sommercial (Score:2, Funny)
sommercial - n - An advertisement for soma.
Source: A Brave New World, by Aldous Huxley
Re:You know why it costs more, right? (Score:2)
Well, many of us might have a weekend or two. Probably fewer of us have $6K for an xserve just laying about.
Yes, of course, there is a cost associated with your time. However unless someone is paying you for every minute of your day, you have access to your own time.
Promise UltraTrak SX4000 (Score:5, Informative)
Each enclosure presents itself as a SCSI drive. You can chain enclosures together and use LVM to make them available to one (or more) filesystem(s). Hardware RAID takes place within each enclosure, but not across them.
Mine were set up with four 300GB drives which, after RAID5 overhead, gave me 900GB of usable space per enclosure (I had two enclosures). You could easily use 400GB drives and have 1.2TB per enclosure if you wanted. I would think that total cost for the enclosure with 400GB disks would be around $2500 US.
Re:Promise UltraTrak SX4000 (Score:2)
Re:Promise UltraTrak SX4000 (Score:2)
Re:Promise UltraTrak SX4000 (Score:3, Insightful)
Re:Promise UltraTrak SX4000 (Score:2)
What you're talking about saturates the resources of a single PC, and has to squeeze into the confines of one chassis. It is more or less only scalable to the 4x400GB mark and then falls flat.
Re:Promise UltraTrak SX4000 (Score:1, Funny)
Retard.
Re:Promise UltraTrak SX4000 (Score:3, Insightful)
that by adding PCI RAID IDE controllers, it scales
to the available case cooling and mount points.
I'd probably add some mounting brackets from
erector set pieces and a few fans (and maybe some
acoustic insulation to my closet!!) if I wanted to
scale up to, say 12x400. But that really is the
wall for my personal taste, 3.6TB of RAID 5.
If I ran out of erector set pieces, I'd just pick
up an old gutted tower server case for the job
from ebay or axe-man.
I'd peg the 3.6
Re:Promise UltraTrak SX4000 (Score:1, Funny)
posting
in
narrow
columns.
It's
fuckin
annoying.
Re:Promise UltraTrak SX4000 (Score:2)
Re:Promise UltraTrak SX4000 (Score:3, Interesting)
The controller decided to flag 1 drive in a raid5 as out of sync, (the drive was perfect) then in a few mins. bring it back online and decide that all other drives were out of sync and needed to be rebuilt based on the 1st drive it just de-synched. All the while happily reading the 1st drive's data incorrectly.
On both accounts (after much pressing) promise engineers said the firmware immolated itself. This was all during a few months of uptime on the firs
One more thing I'd consider unreliable (Score:2)
Against my spidey-sense I made a few arrays out of the 100 and 120 gb ide offerings from WD- Some did not like being on the controllers (promise and 3ware), some needed a huge ammount of extra power to boot and not recaliberate/reset or spin down(!).
In the end, all 20 are now dead after 2 years of use, the first 15 failed in the first 6 months (and yes all were oem).
Re: (Score:1, Redundant)
And when the house burns down??? (Score:5, Insightful)
For keeping your data safe, however, RAID is mostly useless - something you will come to realize when the house containing your RAID burns down or when the RAID is stolen by burglars or a human/software glitch manages to "dd if=/dev/zero of=/dev/yourdisk" your RAID. Power-surge, tornado, flood...the list of things against which RAID fails to protect is long.
If you really care about your data you _must_ perform regular backups and take them off site. I rsync my photos to my work machine and use a VXA tape drive for regular backups.
And I don't bother with RAID except on servers at work.
Re:And when the house burns down??? (Score:1)
Furthermore, RAID won't help you a bit with filesystem errors or if individual files get accidentally deleted or corrupted.
That's why we run tape backups every night.
Re:And when the house burns down??? (Score:2)
Re:And when the house burns down??? (Score:2)
Re:And when the house burns down??? (Score:2)
Not an option for everybody -- most work machines are behind firewalls. Fortunately, there are a lot of online backup companies [google.com].
The kind of solution every geek should have -- you really want regular restore points for your whole system. Alas, at $1K, not within everybody's budget. (My whole setup cost me about $1K.) There used to be a lot of cheap tape cartridge drives
Re:And when the house burns down??? (Score:2)
True, but it is an option for me (small company, I run all the machines, company owner is a friend, etc.) It doesn't have to be work. I have friends who backup to each others' machines.
Alas, at $1K, not within everybody's budget...
And that's why you have to get your equipment requests in early when the dot-com goes bust. :)
Seriously, a couple of big drives in hot-swap carriers is a pretty attractive backup option these days.
Re:And when the house burns down??? (Score:3)
3ware isn't cheap but you get what you pay for. High-quality, great performance, great support, and it "just works" with Linux.
I've had issues with several brands / chipsets of onboard SATA controllers, which is why I went with 3ware. Software raid is all fine and good if you can get reliable drivers! If not, then avoid it.
With 1T of storage, you are talking quite a fe
Re:And when the house burns down??? (Score:2)
Re:And when the house burns down??? (Score:2, Insightful)
Re:And when the house burns down??? (Score:2)
You can rsync/backup/incremental backup to those, and make a weekly drop at your bank's safety deposit box.
Depending on the # of drives/enclosures and your directory structure in your raid setup you may be able to put your
This gives you offsite backups. Sure a tape system that could backup everything each nite would be
Bing Bing Bing.... (Score:2)
Repeat after me...
RAID IS NOT A BACKUP.
Here's how I do my backup....
1. I have a computer with 2x200 GB drives mirrored.
2. I do an incremental backup of the data on these drives every other day to DVD-R discs. (I do not keep the discs for more than 6 months)
3. Every other month I do a full backup and take it off site. (Safety deposit box)
4. I rsync my critical data every four hours to an offsite server which is backed up to tape every night.
Most of the data that I back up is not c
Re:And when the house burns down??? (Score:2)
Software RAID works for me (Score:2)
Re:Software RAID works for me (Score:2)
Re:Software RAID works for me (Score:2)
When the HDD fails with hardware RAID, you rip the drive out, replace it, and it starts rebuilding.
Linux software RAID only works on a per-partition basis.
When the HDD fails with Linux software RAID, you have to repartition
$6,000 (Score:5, Interesting)
The thing is, I don't have time to shop for and configure a RAID system myself, and I want a turnkey solution that will just work out of the box
You're paying $6,000 because Apple does the work for you.
I suppose you could get a tower PC, and fill it with hard drives and setup RAID. Cheaper? Yea. Reliable? Yea. But there's more to it than that.
I googled for 'building 1TB server'
http://www.martinandalex.com/blog/archives/2005/0
Home 1TB RAID Server
CPU Athlon 3200+ $199 Frys 11x multiplier, should over clock to 2.6GHz easily
Memory 1GB Corsair 4400C25 $275 Very fast at DDR466
Motherboard ASUS K8N-E Deluxe $149.99 Frys, 6 SATA RAID chips on Motherboard, 3GB memory
Case SUPERMICRO Beige 4U Rackmount Chassis, Model "SC742T-550 Beige" $307.50 New Egg. Has 7 SATA backplane built in
CD drive NEC 3500A $67 newegg or zipzoomfly
system drive WD740GD $185 10000 rpm system drive
Data drive Maxtor DiamondMax 10 250GB $149.99 ($.59/gig vs $.68/gig for 300GB) 5 of these bad boyz
total: $1933.48 or less than $2 per gig for RAID. Half the cost of white boxes and 1/3 the cost of anything from the channel.
Here's another article, more information
http://www.ethics-gradient.net/myth/storage.html [ethics-gradient.net]
Re:$6,000 (Score:2)
Maybe a commercial NAS box? (Score:2)
Re:Maybe a commercial NAS box? (Score:1)
Re:Maybe a commercial NAS box? (Score:1)
Re:Maybe a commercial NAS box? (Score:1)
Looked at this already (Score:2)
1. httP;//www.3Ware.com [slashdot.org]
2. http://adaptec.com/ [adaptec.com]
The 3ware is quite popular, but the adaptec wins in my book because it will do several things the 3ware one will not:
a. On-line expansion: the ability to expand your array without backing up the data to another location, reconfiguring the array, and reloading the data.
b. Differently sized disks: if I have 2 dis
3Ware, 3Ware, 3Ware (Score:2)
Look to eBay for deals, I managed to get a 3Ware 7820 (I think, it is 8 Channel PATA) for a hundred bucks. Currently I have 2 120G drives attached to it for my main system drives, and I plan on getting 6 more 250-300G drives to build a second array on the device.
The 3Ware cards (mostly, they have one cheap RAID 1 or 0, 2 channel card) support RAID 0, 1,
Re:3Ware, 3Ware, 3Ware (Score:2)
2. Adaptec cards claim to do their raid computations in hardware, meaning the system CPU isn't required to do the computations. This is much better than software raid solutions for obvious performance reasons.
3. As I said, the differentiating factor is the ability to grow a setup by adding a disk, without the need to back up the data to another location. Can 3Ware do this? Last I checked (about 2 months ago) they could not.
Adaptec has a good reputation for quality part
Re:3Ware, 3Ware, 3Ware (Score:2)
Yeah, that's the bitch with the 3ware cards. My 8500 can't expand. I heard the newest cards, with their still-beta 3dmd2, can do expansion, but I can't verify that on my own.
Re:3Ware, 3Ware, 3Ware (Score:1)
Everyone forgets LSI (Score:1)
Re:Everyone forgets LSI (Score:2)
Between me, customer's software vendor who built/configured the thing, IBM, Microsoft and the local IBM maintenance reps - we couldn't turn off write-back cache. At all. Windows 2000 had "couldn't disable write-back cache" errors spewing all over the system log every 10 seconds until I applied a hotfix so that now it was only once per minute.
The IBM guys actually said "We're not that familiar with LSI RAID". He had a simulator in front of hi
Re:Looked at this already (Score:2)
eBay, FOOL! (Score:2)
Look for "Snapserver" or, my fave, the Iomega p410u. In a pinch you can netboot it with bigger hard drives, but by default you get >300GB of raid-5 with hot-swap IDE. and with gigabit ethernet, who cares.
Anyway, for mass storage, get to eBay
(note: yeah, yeah, eBay eats babies)
Form Factor (Score:2)
Re:Form Factor (Score:2)
One source is here: http://www.smalldog.com/product/43611 [smalldog.com]
This is not a stupid question... (Score:5, Insightful)
and other nonsense excuses for not answering the question.
I'm guessing people have spent a LOT of money on reliable storage solutions and tend to be irrationally dismissive of the possibility of inexpensive redundant storage.
The fact is, if you know Linux well, maintaining a Linux based RAID array for home use is perfectly reasonable and generally quite painless. I build an inexpensive 4 drive 480GB RAID array a few years ago that I've been delighted with since. I have survived a disk failure with minimal downtime and no data loss.
"And when the house burns down?"
I'm so tired of this stupid argument. Data loss due to fire will happen with or without RAID. The fact is, losing a disk is much more likely than having your house burn down by a very large margin (I'd take a rough guess that disk loss in a 8 disk system is about 10,000 times more likely than disk loss from fire). But even if they happened with the same frequency you'd still be reducing your exposure by 50% by eliminating data loss from disk failure with RAID.
I have yet to find an online company selling properly configured systems for a reasonable price.
I thought about building a standalone storage server recently and saved my design in a newegg wishlist [newegg.com]
For rack mount RAID systems I like the design cases they have at www.rackmountpro.com but I've never dealt with them personally so I can't say how well they work.
Re:This is not a stupid question... (Score:1)
Re:This is not a stupid question... (Score:2)
Well, you could write a Samba VFS module or something of the sort which automatically intercepts video media and re-encodes it to Ogg Theora on the fly or something I suppose...
(You know, I was joking when I started writing that, but the poster DID say it was for storing "large media files" - maybe he DOES intend to do some sort of server-side processing of the files...)
Re:This is not a stupid question... (Score:2)
You're taking that arguement too literally. The point is that there are many things that proper backups can protect you from that RAID cannot. For example, the
Re:This is not a stupid question... (Score:2)
Re:This is not a stupid question... (Score:2)
And that's what he wanted, or have I mis-understood 'turnkey'...?
The fact is, if you know Linux well, maintaining a Linux based RAID array for home use is perfectly reasonable and generally quite painless. I build an inexpensive 4 drive 480GB RAID array a few years ago that I've been delighted with since. I have survived a disk failure with minimal downtime and no data loss.
Whenever my employer (and I don't
Re:This is not a stupid question... (Score:2)
The fact is, if you know linux well, then you don't need to bother asking Slashdot.
If you -don't- know linux well, then the price that you pay will be your free time. Setting up your first stable (and properly backed-up) LVM system can be quite time-consuming. For some people, it's not fun, so spending an extra thousand dollars is a better deal than losing a few weekends.
LaCie F800 (Score:4, Informative)
Re:LaCie F800 (Score:2)
"total capacity for RAID 0, 50% of total capacity for RAID 0+1, 75% of total capacity for RAID 5, 50% of total capacity for RAID 5+spare"
So their 2TB setup is more like 1 or 1.5TB if you want any kind of data safety. It also must be using the bleeding-edge Hitachi 500GB drives, which haven't proven themselves yet--I don't think they're even shipping.
When is 1TB not 1TB? (Score:2)
Ask a better question.... (Score:2, Insightful)
1. How much money you want to spend, just that $6k is too much.
2. How fast you want it.
3. Whether or not it needs to be quiet or has to run in an unairconditioned garage.
4. Whether or not you want a "headless" server or a Linux box with vga and keyboard 5. Why you're willing to maintain and upgrade it but not build it.
In short, you're probably not a geek else you'd be digging into this and figuring it out yourself. Or at least asking better questions.
Re:Ask a better question.... (Score:2)
Oh, and your comment of "In short, you're probably not a geek" just makes me think; grow up and move out of your mothers basement becasue I can't WAIT to see you get a new asshole ripped in the real world.
Honestly, its numnuts like you who really piss me off.
To paraphrase; there are no stupid questions, just stupid answers. You know what, that
Re:Ask a better question.... (Score:2)
Re:Ask a better question.... (Score:1)
Re:Ask a better question.... (Score:2)
And your reply obviously goes to show my initial comments concerning your unecessary flaming. Grow up.
(oh, I live in a £750,000 home... thats about 1.5M of your dollars, thats because I wasn't fixing minicomputers when you were "pissing diapers", I was busy).
Honestly, what a prick
eRacks, other alternatives (Score:2)
mmmm (Score:4, Funny)
too late... (Score:2)
Well, clearly what you should have done was to pay me < $1,000 for the sweet sweet dot-bomb (sort of) leftover that I donated to charity last week. Too late, so sorry--RedHat installe
Promise UltraTrak + Hitachi's new beasts (Score:2)
4x Hitachi 7K500 500GB drives (should be out RSN) - $1200-1500
A good U160 SCSI card for one of your computers - $200.
For about $2500-3000 (half the price of an Xserve RAID), you can have 1TB of RAID0+1.
Tape drive. (Score:2)
Similar Problem (Score:2)
I only want to use the disk on one machine so NAS is overkill.
I only need moderate speed so SCSI is out.
I need >1gb of logical drive space.
I want hot swap.
I don't need to run another OS, just a dvice to manage the array and connect to my desktop.
Cost is likel to be around $2k. I might get lucky on ebay but it's unlikely.
SAN? (Score:4, Interesting)
I'm looking for a roughly similar "solution" - adequate performance with redundancy and lots of storage space. A "SAN" ("Storage Area Network" - one of the current buzzwords going around these days) might be useful.
Either iSCSI (if you want economical and standards-based) or Fiber-Channel (if you're wealthy and the speed of writes to the hard drive array is critical) based boxes of hard drives seems to be an option, and from the point of view of the server (or whatever computer is using them) they are just another hard drive. Or so the materials I've read say. (Think of them as an external RAID box...)
iSCSI seems to be limited to 1Gb speeds (unless you can get your hands on 10Gb ethernet cards and switch, which I gather are now available), which to me seems perfectly adequate for most file-server type uses. It looks to my still-new-to-the-area eye that you can also do a lot of potentially useful tricks because of the standard IP-based nature of the data transfer (such as being able to mount a "hard drive" directly over the internet or a LAN, if you have some reason to need to do so). Fiber-channel is faster (2Gb seems to be typical, 4Gb is apparently getting fairly established, and 8Gb is available if you're made of money) but requires specialized and fairly expensive hardware ($500+ for each fiber-channel interface card at the LOW end, as I recall, plus several thousand for the fiber-channel equivalent of a "switch".)
On the subject of iSCSI, there seem to be active projects with both "target [sourceforge.net]" (iSCSI device server) drivers and "client [sourceforge.net]" (iSCSI device mounting) drivers for Linux on Sourceforge...
Corrections welcome, of course...
Incidentally, that's not to be confused with "NAS" ("Network Attached Storage") which as far as I can tell is a buzzword used by people due to the fact that "file server" doesn't sound "cool" any more...your "NAS" might be using a "SAN" to store the drives that it is serving...
In any case, this may be me trying to "hijack" this Ask Slashdot, but what do people here think of the "SAN" concept and its implementations?
SAN Implementations (Score:2)
As long as all of the FC devices you use have fabric support, all you need a SAN filesystem and perhaps a metadata server and you're good to go.
One sample setup would be this:
Brocade FC switch
Xserve RAID (just the RAID box, not a server)
Mac w/ FC card
PC w/ FC card
The Mac, PC, and RAID box are each plugged into the FC switch. Now either machine can access the RAID directly through this FC SAN. There is no need for
More SAN fun (Score:2)
1) SPEED! No need to access the data through a server. You can connect directly over fibrechannel.
2) Server redundancy. You might want to keep your data on a hardware RAID-5 box, such as the Apple Xserve RAID. You could then have two servers connected to the RAID. One server would be doing all of the work while the other would be a hot backup. In the event that the primary server dies, the secondary server could mount the RAID and take over. (T
Raidcore, RAIDCORE , R A I D C O R E! (Score:2)
Everyone seem to think that raidcore is not in the game, but i am telling you that it ROCKS.
The standard line is that it is software raid. This is not the case. You can boot and run dos off of a raidcore with no drivers. The only reason i say this is to underscore that there is no REQUIREMENT to do software raid with this board.
They DO have a driver which allows a software ASSIST to increase speed of the XOR calculation if you have the spare c
Coraid (Score:1)
Congratulations! (Score:1)
Congratulations! Your application to PHB candidate school has been accepted. A rewarding career awaits you.
Just kidding, of course. This kind of SAN device should be as ubiquitous as a TV. Unfortunately the general consumer doesn't know (s)he needs one, and the market isn't scrambling to provide them. Yet, hopefully.
$0.02,
ptd
A couple options (Score:2)
linux with 3ware (Score:2)
My only complaints with my 3ware card:
1) no live expansion
2) haven't had much luck running 3dm/2 with Fedora Core 3 and my 8500-8 card.
Other then that, it's been great. The entire family uses the server for various needs. And knowing that all of our digital pictures are that much safer is nice.
Learn from others who've done this already (Score:3)
home raid with firewire external enclosures (Score:2)
I just ran out of space on my old raid array (two 200GB drives in a gentoo linux machine with software raid 1 mirroring.) I also was inspired by the slashdot story about someone who did a iPod shuffle raid using a usb hub... and since I just ordered a mac mini, I wanted to do something similar, but with actual hard drives. The screenshots of the OS X raid configuration in the disk tool sold me on the idea.
I checked out Tom's hardware for hard drive enclosures that could power and connect some cheap
I'm surprised people haven't mentioned... (Score:2)