1.8TB Of Disk Space In A (Semi-)Normal PC 455
zdzichu writes "A friend of mine is building a personal server. He bought 17 of the cheapest IDE drives available and used Linux' LVM to get them together. The result? Almost two terabytes of disk space in regular x86 PC. The
most juicy part - photos are here.
For an operating system, he first tried the enterprise-ready PLD Linux Distribution, later he reinstalled Slackware Linux." Update: 03/01 20:24 GMT by T : I'm sure that should be "drives" and not "drivers" :)
Amazing (Score:2, Funny)
Technology has come so far that we now have disk space on drivers! Simply amazing.
maybe they're cheaper, over there. (Score:3, Funny)
On the other hand, I think it was a typo -- so I fixed / updated.
timothy
MPAA (Score:5, Funny)
Man, check out that URL... (Score:5, Funny)
That sounds like one mean perl script. First post?
I fondly recall.. (Score:2)
Re:I fondly recall.. (Score:3, Funny)
Re:I fondly recall.. (Score:3, Interesting)
Why is this news anyway? I, personally, have built (and sold) several 1TB+ "PCs" over the last few years. 1.8TB can be done with a half dozen drives these days. (for the cost of *2* large SCSI drives, even.) Heh, I could fit that in a 25$ mini-tower case.
link already dead (Score:5, Funny)
Re:link already dead (Score:5, Informative)
Re:link already dead (Score:2)
It's Top Secret... (Score:2)
Hillary Flammond: "Nic? What does that mean?"
Nic Rivers: "Oh, nothing. My dad thought of it while he was shaving."
(Ok, slightly bastardized...)
-FF
Won't last long (Score:2, Interesting)
Re:Won't last long (Score:4, Funny)
can someone smell burning? (Score:4, Funny)
Slashdotted (Score:5, Funny)
I'm not sure I'd use the word friend after this. I hope he's not paying for his bandwidth!
Re:Slashdotted (Score:5, Funny)
Must not be a very good friend... (Score:2, Interesting)
Re:Must not be a very good friend... (Score:3, Funny)
Before it goes into production, he wanted to do a stress test. And what better way to do one of those than to get linked from the front page of Slashdot?
Re:Must not be a very good friend... (Score:2, Informative)
Controllers? (Score:3, Interesting)
Damn, /.'ed already...
Anyone that actually saw the site know how he hooked up all those drives? I'm guessing motherboard IDE, motherboard RAID, and three PCI IDE cards. Wow, talk about IRQ hell.
Re:Controllers? (Score:2)
Re:Controllers? (Score:2)
Oh, I'm not saying it couldn't be done, I was just flashing back to the days when juggling IRQ's via dipswitches or jumpers was an art. Nothing like interupting your music playback everytime you moved your mouse!
I guess, though, you also have to take into account everything else a system would have. A standard, onboard IDE controller uses 2 IRQ's. Onboard and add-in RAID uses 1, NIC, USB, firewire, sound, legacy ports, all use 'em. Thank god for IRQ sharing.
Re:Controllers? (Score:5, Informative)
Second, 3Ware, and a couple other companies, make 12-drive ATA RAID cards. So one of those, plus onboard ATA would reach 16 drives. Or, a second ATA RAID controller would allow an additional 4, 8, or 12 hard drives without resorting to the onboard ATA. For a max of 24 drives without using onboard ATA. (In my personal server, I have 8 10GB drives on an ATA RAID card... They're in dual 0+5, for a whopping 60GB of space, but it's fast, and reliable. Someday I'd love to upgrade them all to Maxtor 300GB drives, but I'd need a new RAID card in the process. [and a large fortune.])
He'll need the space . . . (Score:5, Funny)
Does anyone else find this stuff boring? (Score:2, Insightful)
I am much more interested in what interesting things people do with computers, not how tricked out their computers are.
Re:Does anyone else find this stuff boring? (Score:5, Informative)
Re:Does anyone else find this stuff boring? (Score:3, Funny)
Uhmm... Let me see. hmmm... uhmm... no, he's interested, so is she, and over there, yep, and everybody in Europe, China, rest of Asia, Africa, check, yep, all interested... Oh, wait, South America... Yep, all interested. US, rest of North America... hmm, Surely someone is Canada not... nope, they're all interested, too. I know! That guy who reads /. from Antartica. Nope, he's interested.
Yep, you're the only one.
This isn't just a resource article (Score:3, Insightful)
It's where they put together a powerful system...cheaply. Using those little rails looks like an interesting solution. And I'm always interested in ways to get more for less...
Re:Does anyone else find this stuff boring? (Score:2, Insightful)
Yes, but I think it would be even more dull, if Slashdot would report what geeks are doing with their computers. There would only be news about geeks watching pr0n all day long.
Re:Does anyone else find this stuff boring? (Score:2)
Maxtor makes 250 gig HDs (Score:3, Informative)
Man... (Score:5, Funny)
Re:Man... (Score:2, Funny)
I looks like he just siliconed them to some metal brackets. Ghetto mod for sure.
My opinion... (Score:2)
I know this'll pull out the SCSI bigots, but the only reason SCSI is good these days is cause they're tested for longer times (disk media is better quality).
Re:My opinion... (Score:2, Interesting)
I can't begin to think how you'd come back up after losing a drive in a concatenated R^HAID. Whoops, no R if it's not redundant eh?
I'm actually quite glad I'm not sitting on 1.8TB of data, and I don't intend to in the near future.
If he does mirror the drives, I wonder if his mobo will be the bottleneck..?
Re:My opinion... (Score:2)
I wonder how much this would cost scsi wise. (shudder)
Re:My opinion... (Score:5, Informative)
I wonder how much this would cost scsi wise.
Figure a 72G 10K rpm SCSI disk at $500 times 20 + 4 spares = 12000 for the disks. Then figure that a raid controller runs $500 - $2000 and add a large hot plug chassis and you're looking at $15k. However, You now have hardware supported RAID at up to 400MB/s sustained and all of those drives are covered by a 5 year warranty. The 4 spares are just insurance against a supply problem down the road. Of course, you need to buy you disks from different lots (5 per dealer perhaps) to minimize the effects of a bad lot. Yeah, SCSI is expensive, but you get better reliability.
Mirror... (Score:4, Informative)
Mirror of images... (Score:2)
In closing, I'd just like to say, it will never cease to amaze me what some men will do for their pr0n. Err, I mean, sharing a lot Linux distros in p2p networks? ;)
Am I the only person... (Score:5, Funny)
Re:Am I the only person... (Score:3, Funny)
Large Disk Arrays (Score:5, Interesting)
With 8 250Gb Maxtor drives, he could have 1.75Tb per array.
I'm curious. What did he use to allow him to put so many IDE drives in the same machine? Off the top of my head, I believe he can use PCI cards that have 2 IDE controllers on each, allowing 4 drives.. Did he have 4 of those, plus the onboard IDE controllers? The pictures are going really slow to load..
I have a server now, that has 8 120Gb IDE drives, with a Promise internal RAID card, which works ok.. It freaks out under load though, so I don't recommend that. We don't use it for a web server any more. It's just a backup machine now, with 840Gb storage.
Re:Large Disk Arrays (Score:3, Insightful)
Ugh! Don't do RAID0 with 8 drives! With RAID0, losing a single drive means the whole array is all but worthless. (Hard to get data off with one in eight chunks missing...) I think the longevity of a single drive is a normal distribution with a mean at its MTBF. If I remember my statistics, that means the combined MTBF is just the MTBF of a single one divided by eight. Don't divide your reliability by eight!
The RAID5 is a much better solution, since it can handle a single drive failure with no problems. The odds of two drives failing at the same time are really low. So as long as you are prompt about replacing failed drives, you can't go wrong.
Re:Large Disk Arrays (Score:3, Interesting)
Except that performance blows.
And it gets worse when (not if, but WHEN) a single disk fails. Parity recalculation is EXPENSIVE.
RAID0+1, or RAID 10. Mirror that stuff, don't look back.
And egads, seriously, 2 drives per IDE controller? Performance has to be in the toilet already.
Just buy a damned XServe RAID [apple.com] and be done with it.
Are linux drivers ready? (Score:5, Interesting)
In kilobyte blocks, 2^32 blocks only allows for 4TB of data to be referenced. ext2 still has options to set for 1024 byte blocksize, and supports up to 4096 - which would be a 16TB barrier.
Re:Are linux drivers ready? (Score:3, Informative)
We have run into this barrier at work several times. With large ATA arrays, it's getting almost trivial to amass 2TB+, so I sure hope this gets fixed post-haste.
MIRROR HERE: http://crazyserver.150m.com (Score:5, Informative)
Enjoy!
PS: Sorry for the banner ads, it's a free server.
Precarious setup? (Score:3, Interesting)
He has 1 normal PC case, 2 homemade stands for the drives, and one more homemade stand for additional power supplies.
The stands with the drives look like they could topple with a moments notice! Why did he put them at the top...?
I think it would be better to mount as many power supplies and drives in 2 additional cases, with the shells removed. Might be a problem with IDE cable length; maybe you could do 2 next to each side the the master computer.
The setup. [iwaynet.net]
Re:Precarious setup? (Score:3, Informative)
He has 1 normal PC case, 2 homemade stands for the drives, and one more homemade stand for additional power supplies.
The stands with the drives look like they could topple with a moments notice! Why did he put them at the top...?
I agree. It would have been better engineered with 2 power supplies at the bottom of each tower, providing a more solid base for the disks. I'd of made the towers far shorter as well - perhaps even turned them sideways.
I think it would be better to mount as many power supplies and drives in 2 additional cases, with the shells removed.
That would be expensive, and would also make heat an issue. The setup he has allows for passive cooling - even a case with the shell removed would trap more heat. Heat can lower MTBF - not something to do with IDE drives. A proper external disk case would make even more sense, as most come with fans and cooling.
Might be a problem with IDE cable length; maybe you could do 2 next to each side the the master computer.
Now you know one of the reaons why SCSI is king for servers - it's meant to be used both internally and extrenally. I've used 10' long, high quality SCSI cables to attach external disks to servers in my time without issue. As well, you can have 14 disks per SCSI controller - not possible with IDE.
It's a nice hack, even if it had design issues, though.
Soko
Re:Precarious setup? (Score:2)
It also doesn't look safe as I don't see any data redunancy being done on Linux. With over a dozen drives I'd say the risk of data loss is pretty high.
I'm not sure if that many power supplies are needed either, but I don't know what recent drives take. The hard drives I have in my system draw maybe ten to fifteen watts, so even with a 100% safety margin, a 200W power supply should supply nearly seven drives. The best thing I can think is startup draw, having so many drives start up at once might be bad, but I figure that the safety margin should handle it.
I don't see a backup method in that picture either. No redundancy and no backups. It looks like an interesting proof of concept project but not one that I would entrust hard to replace data to.
Wow. That's a lot of porn. (Score:2)
Have to wonder how cheap 17 100GB drives could be? I think of a relatively cheap 100gb drive as running around $90.00 (US). Which would make it very much on the pricey side for your average user.
Seems like you could just buy a DVD-RW and keep all yer porn on handy little disks, while having enough $$$ left over to go on a major bender, or upgrade the REST of your computer.
Kudos for the sheer weight of it though. (Both literal and figurative.)
Just my 6.32070 Drachmae worth
fsck (Score:2)
Re:fsck (Score:2)
Why is this interesting? (Score:2)
You can get a hardware IDE RAID controller from 3Ware [3ware.com] right now that supports serial ATA (the model 8500 [3ware.com]) in 4, 8, and 12 channel varieties or parallel ATA in the same capacities (the 7500 [3ware.com] series), and install commodity disk drives. The hardest part about this is getting a chassis with sufficient power and cooling capacity to handle all the drives.
It looks like running 12 Western Digital "Drivezilla" [wdc.com] 200GB drives ought to give you somewhere around 2.0TB of storage (taking into account the bullshit mathematics of hard drives). At Pricewatch prices, I see about $3,500.00 tied up in the drives and the controller.
Whoopy shit.
mirror (decreased image size) (Score:4, Informative)
cheapest drives eh.... (Score:2)
Hope that crisis counselor [slashdot.org] over at that data recovery place is ready for one heck of a call in about 12 months.
Not that I'm a SCSI fanboy, but if he's buying the cheapest IDE drives I hope he's also planning to invest some money in one of these [storagetek.com] or something. I'm pretty sure that regardless of what he's putting on there, he'd be disappointed to lose that much data. I mean, imagine the amount of time you would have to invest to collect that much warez and porn.
Hehe... Fear the shell :-) (Score:5, Funny)
20 gig is fine for me (Score:2, Flamebait)
Before you answer porn consider how much money this array cost and how much money it would cost to actually *pay* for the xxx dvd's.
Most of the kazaa is crap anyway and transfer rates are terrible. It seems there a few mpg's that people like and the other %95 are wasting space and are terrible. For the few good movies and lots of garbage that you actually pay for with the expensive hard drives, you could save money and not be a criminal by just buying the good dvd's. Sure we all hate the RIAA/MPAA but pornographers have not been assholes as of yet and do not deserve to be ripped off. After all the stigma sucks for pornstars and they at least deserved to be paid for their horrible jobs.
I need to focus on school so I purposely took out the ethernet card( internet addiction) and deleted most of my porn. I have so much space free it is silly. Even when I was on kazaa I had close to 10 gigs free. I use another 20 gig drive for Linux/FreeBSD and that is the only reason why someone would want a large hard drive. Its easier to buy too small ones to make dual boot life easier.
Porn and exploitation (Score:3, Informative)
Well, you have to suck to get stigmatized. The real hit on mainstream porn has been amateur porn from overseas. Round up a few starving Belarussian girls pay them what is in their eyes a king's ransom, then take the digicam back to your iMovie-laden iMac and 1,2,3 you are a porn magnate.
If you ever thought Jenna Jameson was getting exploited (which is a tough sell), you haven't seen shit until you see this stuff. Obviously frightened women getting grovel shagged by overweight dudes from Valley Stream who kick 'em back to the cold with 50 bucks and a case of genital warts.
Wait, all you Libertarians, no they have no choice. You gotta keep the lights on somehow. Wait, all you Free Marketeers, go back and actually read Adam Smith. He warns against shit like this, particularly white slavery.
Re:Porn and exploitation (Score:3, Interesting)
I'd imagine, though, that the bulk of women who are tapped for this are already in the market as prostitues (it makes sense -- there are usually established ways for outsiders to find prostitutes, and I would guess the only differences between the two are that the action is being recorded when it's for porn).
This isn't to say that it makes it OK, etc, but if, say, all of them are willing prostitutes (of reasonable age/education, not economically coerced or otherwised forced), that does narrow the scope of the exploitation by a bit.
Re:Porn and exploitation (Score:3, Insightful)
Exploitation based on money/sex/power relationships
and exploitation of the image/profits the owner makes off of it relative to the exploited's compensation/etc.
Re:20 gig is fine for me (Score:2, Informative)
Eh? Who said anything about porn? Maybe this guy wants to mirror linux/BSD isos or other software, docs, other websites, etc. You can never have too many mirrors, after all. Or maybe it's just a fun project. It would be nice if not everyone jumped to the conclusion that this guy is setting this all up for some grand warez site or what have you. At least I hope so, anyway.
Yep, let's do that (Score:3, Informative)
2571 * $20 (at least, here) = $51428
What? Unfair comparison? Well you're comparing with an extreme machine. Maybe kazaa sucks, I don't use it. But at my uni there's no problem getting more movies than you'll ever see, mostly in quality DVD rips by ripgroups in a matter of an hour or less per rip. Not that it makes it right, but if you want me to do a pure financial estimate less moral costs, rips win hands down. That's not a troll or a flame, that's a fact. Even if you factor in the chance of getting caught and fine, it still wins hands down. And no, having a client running in the background of my machine isn't really costing me much time, it's a fire and forget thing, check back later.
Kjella
Re:20 gig is fine for me (Score:5, Funny)
Yeah, I can't imagine any possible down side to working in porn. [danheller.com]
don't use LVM for this (Score:3, Insightful)
If you need a really big file system spanning a lot of drives, use some form of RAID. Using LVM for spanninng volumes is mostly a band-aid, if you have run out of space and desparately need some additional space right now.
TechTV did this in XP last year... (Score:2, Informative)
Like 8 IDE drives, 1TB+ on XP.
This isnt news
its all in the noise... (Score:2, Interesting)
I love it when 17967MB can be considered rounding error. That's more space than I have on my whole computer!
First time I heard of a terabyte of storage (Score:4, Interesting)
Re:First time I heard of a terabyte of storage (Score:3, Insightful)
Five or ten years ago, it certainly seemed that way and I would have agreed with you. But, the reality is that as storage space has increased and gotten less expensive, the software and file formats have grown to match and consume the space. Programs get more and more bloated everyday because storage and memory are plentiful and cheap so, programmers no longer make an effort to keep their code small. The same holds true for the file formats. 10 years ago, a one page word processing document required 2 to 5K. Today wordprocessing documents regularly go to a couple of hundred K and a few "choice" documents can be over a meg.
Sadly, instead of fitting our entire lives on a massive and inexpensive disk, we will need a terrabyte sized disk just to hold our favorite office suite.
Write-up? (Score:2)
Does anyone know where the writeup is about how this was done precisely, e.g. with what kind of case, drives, and at what cost... (For example, how does it compare to a super-redundant xraid solution from Apple?)
Thanks.
P.S. I enjoyed the bottom of the last picture:
Filesystem Size Used Avail
Woah.. any relation to this guy? (Score:2)
This makes me remember this post [iu.edu] on linux-kernel where Milan Roubal ask for help with breaking the 10 IDE devices "barrier":
(emphasis added)
You could do 6TB with IDE ... (Score:2)
Maxtor makes 250GB ATA disks as well.
With 3 controller cards, 6TB (before formatting) is possible. With LVM, you could manage a single volume nearing 3TB in size. Without it, you could have three 2TB volumes, striped, in hardware. Or three 1TB volumes striped and mirrored.
That's a lot of stinkin' space!
Cost? $300 for 3 controllers, about $7000 for drives. Plus, oh, a few power supplies. Figure $8,000, $8500 with a box to hook it up to.
$8500 for 3TB, fault tolerant, is cheap.
-----
Bandwidth? (Score:3, Informative)
Is he preparing for the next install of MS Office? (Score:2, Funny)
The Screen Snickers (Score:2)
You either get it or you don't. If you don't, watch more TechTV :)
Wow, I *just* did the same thing! (Score:2, Informative)
(4) 120Gb Maxtor UDMA/133 drives from the local "megamart" computer store (Best Buy in the NW) for $89/each (after mail-in rebate, of course). Cost (after rebates) $372.00 (Had to pay the state sales tax, sigh, Washington sucks sometimes!)
(1) Promise SX4000, 4-Channel hardware RAID-5 controller that can handle UDMA/133 drives. Cost = $145.00 from you favorite PriceWatch merchant. Free shipping, no tax.
Slap it all together, format, viola - 360Gb of redundant space for a total of $517.00
My big concern was long-term backup - I opted to go with a DVD-R/+R Sony drive. Drive ran $350 at the local office supermart (Plus that d*mn sales tax = $381.10
100 4x capable DVD-R discs were $1.61ea via an online source. 4.7Gb/ea, a total backup capacity of 470Gb. Cost = $161.00, not tax, free shipping.
Drives: $372.00
Controller: $145.00
DVD-R/+R: $381.00 (Could have gone with the cheap one for $199, but wanted the dual-capability)
100 DVD-R discs: $161.00
Total cost = $1059.00
Total capacity = 360Gb (RAID-5)
Backup time = 15m per disc, ~20h for 360Gb (swapping discs sucks, but sure beats paying tape backup prices)
What is the space used for? Try DV video editing sometime and tell me how far you get with a 40Gb drive in your machine.
Why? (Score:2)
packaging lots of ATA drives in one box (Score:4, Informative)
AMS DK-035A [amselectronics.com] (ignore the Ultra SCSI reference on that page, the A suffix is for ATA), available for $159 from Central Computer [centralcomputer.com]
I just set up an eight drive RAID using one of those, and one 3-drive-in-2-bay version, the DK-023A [amselectronics.com] ($119 from Central Computer). That way eight removable trays fit in my 5-bay 4U rack mount case.
I used a 3ware Escalade 7500-8 RAID card [3ware.com] rather than Linux software RAID, but there's no reason why it wouldn't have worked with software RAID. I just couldn't find a "dumb" eight-port ATA-133 card. (And I like the 3ware cards.)
I initially tried to use Serial ATA, using the Promise SATA150-TX4 4-port Serial ATA controller [promise.com] and some Highpoint RocketHead 100 Serial ATA adapters for the drives. The Highpoint web site claims that the RocketHead 100 is not available for sale as a separate product, but lots of retailers including Central Computer seem to have them. The cabling was *much* nicer, but to make the SATA150 work with Linux a binary-only driver was required, so I decided not to use it until there's a driver available in source form.
I thought about using the 3ware Escalade 8500 [3ware.com], which is the Serial ATA version of the 7500, but there's quite a price premium, so I decided to stick with parallel ATA for now. Maybe next year I'll set up a bigger RAID using Serial ATA.
AH HA! (Score:3, Funny)
Then there's that bloody goop in the food processor
Finally there's the Windows box peeking out from behind the dresser!
Come on people do I have to spell it out for you?
Don't you see what's going on?
Oh the humanity!
Big deal (Score:3, Interesting)
I have a raid motherboard....so...
2 primary IDE chains x 2 250GB drives
2 raid chains x 2 250GB drives
WOW.... 2TB.....whoopdeedoo...that was hard
Terabytes gets you chicks! (Score:3, Funny)
drive letters? (Score:4, Interesting)
Re:2TB? That's a lot of PR0N! (Score:2, Funny)
Re:Great, now for the.. (Score:2)
Re:Great, now for the.. (Score:2, Insightful)
There is no way 90 minutes of 720x480 should take up that kind of space.
An hour in DV format is about 13G
An hour in USB MPEG-2 is about 4G
Even if you use something like HuffYUV it would only be around 30G, something like that.
Have you done a lot of video before? This just doesn't seem right.
Is your source material clean enough that lossless really helps? What kind of software are you using to sample?
Re:Great, now for the.. (Score:2)
If his numbers are right, then he's not even hitting the full uncompressed rate, which would be around 110GB/hour.
That being said, it's rare the casual situation where you need to work much higher than DV res. Unless he's doing broadcast, I'd suggest cutting the data rate a little. At the very least, learn about off/on-lineing projects.
-Brett
Re:Great, now for the.. (Score:2)
Get real. (Score:2, Insightful)
First.. you have no idea how I may or may not use disk space. When you have such space, you find ways to use it.
IDE drives? Yet more bullshit about "IDE drives sucks". Guess what genius, IDE is just an interface.. it says nothign about the durability of the hardware. Yes, it's true that most manufacturers make scsi drives with better parts, simply due to the target market, but not all.
And what do you think raid is for.
Nice troll though.
What's that go to do with it? (Score:2)
only about whether or not ide drives are unreliable.
Re:Great, now for the.. (Score:2)
Didja use ogg, or at least VBR MP3?
CBR MP3s drive me bananas. I mean, the only reason people rip at 320 or whatever is for those instants where it really does matter...and VBR handles that without throwing away tons of data on bits where it doesn't matter.
Re:YOU SIR ARE AN ALL AMERICAN HERO NOT UNLIKE GI (Score:2, Informative)
Re:Why 17? (Score:2, Funny)
Re:Why 17? (Score:3, Interesting)
Re:The standard answer (Score:2, Funny)
warez
Re:The standard answer (Score:3, Funny)
One word:
pr0n!!!!
Re:The standard answer (Score:5, Funny)
A highly available, highly redundant data warehouse for storing customer information, product inventory, supplier status, and outstanding orders in a lightning fast database format with a user-friendly front-end, adding to worker productivity while decreasing maintenance downtime, thereby lowering total cost of ownership and increasing company profit.
Nah, I changed my mind. Porn.
-FF
Re:use? (Score:2)
Re: You may want to look at FreeBSD (Score:2)
Linux is not the only free os in town.
Linux's 2.4x vm has had alot of controversy. I heard stories about even Redhats advanced server 7 corrupts regularly if you stress the i/o disk subsystem.
Linux is great but not perfect. Neither is FreeBSD. However FreeBSD's strenght is its TCP/IP stack and its vm and i/o subsysm. Yahoo, ftp.cdrom.com, netcraft.com, samba.org, and even apache.org use FreeBSD because they transfer alot of trafic and use alot of i/o. The only thing FreeBSD is weak in is multiprocessor support. Solaris can handle large loads because it divides the work evenly among all the different processors in the server. For a single processor system FreeBSD can take quite good loads and remain stable. Its just more mature. FreeBSD 5.0 includes nvidia support, java, and better threading. It also comes with a great book.
I am mentioning this because it seems you have invested alot of money into your system and hardware is only one part of a platform. An operating sytem is just as important and a good os that is optimized on what you want to do is essential. FreeBSD was designed to be a server operating system so good volume and raid managment is essential. Also did I mention that the ports rock!
Re:Experience with large raid setup and linux (Score:5, Informative)
I don't think it's Linux, bud - my suspicion is that the controllers themselves can't deal with a maxed out PCI bus. They are normally bus-master cards, so it's possible that one is grabbing the PCI bus and holding it for so long that the one of the others is giving up, instantly corrupting your array. Promise likely didn't take into account such setups where there's more than one or two RAID contollers per machine.
Currently I am thinking about changing the raid 5 arrays into just plain volume groups without stripping. This would allow me to lose some of the transfer rate and avoid stressing the pci bus.
I doubt it - you still are tryng to squeeze way too much data through the northbridge chip on your motherboard. You may be able to do something with PCI bus timings, but you stand a very good chance of hosing the whole setup that way. You simply need more motherboard bandwidth if you want to support that much disk space - sorry.
What you need is a dual (or triple) peer PCI bus motherboard, so you an have 2 controllers per northbridge channel. Look into SuperMicro and one of their ServerWorks GC LE based boards.
Soko
Re:2TB for what? (Score:2)
Who says I read Slashdot?