Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Hardware

Terabyte File Server for $5,000 133

Posted by michael from the more-data-than-you-can-shake-a-stick-at dept.
pluto378 writes: "SDSC has a report on their attempt to build a Terabyte fileserver for less than $5,000."
This discussion has been archived. No new comments can be posted.

Terabyte File Server for $5,000 More

Terabyte File Server for $5,000

Comments Filter:
  • okay then.

    "Gee, I wonder how much pr0n it could hold. Hahaha!

    Gee, I wonder how many MP3's it could hold. Hahaha! (some stoopid reference to Napster would be in order)

    Gee, I wonder how many different distros of Linux I could install? Hahaha!

    Could you imagine a Beowulf cluster of these? Hahah!"

  • A way to do this (Score:1)

    by Anonymous Coward
    Reasonable PC (cheap, std): $500 SCSI controller: $50 10 100 GB hard drives (IDE, see pricewatch): $2500 Alcita IDEPlex 2 board (scsi IDE): $600 Chassis for disks: $100 Total is $3750, holds a terabyte. Put Linux on the box so it will serve SMB and NFS and you have it. An IDEplex will let you hang 8 IDE busses on a single SCSI ID, so you can hang up to 56 IDE drives on a narrow SCSI bus with one of them. That would allow you to expand to 5.6 terabytes using 100GB IDE disks even on a narrow SCSI bus.

  • Not to rain on their parade (Score:4)

    by Anonymous Coward on Friday July 20, 2001 @04:03AM (#73106)
    But for those wanting to build a 1 TB servers, cheaper options already exist with commodity components. For example:

    18(*) WD 60 GB hard drives ($2250)
    3 Hotrod ATA/100 controllers ($180)
    Powered enclosure ($200)
    Custom extra-long IDE cables ($150)

    Get a motherboard with 4 IDE channels (most "raid versions" have this) and plug 12 drives into the Hotrods, 4 into the motherboard's raid channels, and 2 into the secondary ide channel. The boot hard drive goes on the primary ide channel.

    The total cost of this server is still under $3500 after supplying the rest of the computer. As long as you don't need 24/7 uptime or massive throughput, this is enough for 1 TB of directly available storage.

    (*) This is a bit more than 1 TB but you have to account for space lost during formatting.
  • The space is an anti-lamer measure. Slashdot will put spaces in ANY long text string to break it up. Back when Slashdot was still new some lamers discovered that they could make a string 10000 characters long and force the "width" of the Slashdot window out forever. It was extremly annoying because you had to constanty scroll left and right to read the real posts in the article. These days you can't do that anymore, but you have to deal with spaces in links if the poster doesn't know how to make them real links (Hint, look up the Anchor tag, particularly the HREF option).

    Down that path lies madness. On the other hand, the road to hell is paved with melting snowballs.

  • Terabyte? (Score:1)

    by kir ( 583 )

    Holy crap. $5000?? I could even afford that. Can you say... pr0n server!!!!


    Word!

    --
    Kir

  • Computer, hell... the car I want always costs $5000! I should stop spending my hard earned money on misc. hardware and beer.


    Word!

    --
    Kir

  • ...but not Office XP.


    Word!

    --
    Kir
  • Yeah... last place I worked had 8 tape robots, with a total of *3800Tb* of storage space- yeah, 3.8 *peta* bytes. So, using your math, that array could play music for FIFTY NINE years. Hell, I might even double the bitrate, i think 30 years of music would still work out just fine!
  • If you aren't using Gigabit, how in the hell are you getting 20-25MB/sec reads? Last time I checked, Fast Ethernet maxed out at ~10MB/sec.
  • Bah. That's IDE Raid. I can do a TB of fibre channel for that much. Bunch of Seagate Cheetahs, ICP Vortex controller, and cases from Transduction.

  • Re:Any real use for this? (Score:5)

    by Imabug ( 2259 ) on Friday July 20, 2001 @05:01AM (#73114) Homepage Journal
    There are plenty of uses. I work in a radiology department where all our images are acquired digitally instead of on film. We acquire an estimated 4-5 TB worth of images each year. Our current archives (optical disk) will only hold about 1 months worth of data online before patient images get taken offline and put on a shelf. The ability to have a cheap TB online archive would mean significantly faster image retrievals when radiologists want to compare a patient's images with a previous study. Especially important when hospitals are cutting budgets back in a big way, but workloads keep going up.
  • How manny of those "Hotrod" controlers can you put in a a single motherboard? I ask because It looks like a Cool idea to have the controlers do hardwear raid and present the OS with a single drive 180GB or 300GB drive each. (Using 60GB or 100GB drives).

    The OS could then be asked to make a software RAID of those. I.e. with 5 controlers you could have 1.2 TB at a sane cost.

    Perhaps not under $5,000 but still sane.

  • If you aren't using Gigabit, how in the hell are you getting 20-25MB/sec reads? Last time I checked, Fast Ethernet maxed out at ~10MB/sec.

    I was just going to ask the same thing. Maybe he meant that the system itself was able to access the disks at 25 MB/sec?

  • At $5,000, I'd imagine just building a second one makes the most sense. Unless you want to drop $80,000 for the hardware and software to put all that data on tape. :-)
  • $3242.50 as of this morning. Still too close for comfort. :)
  • RAID-5 in hardware kills you, unless you go to very very expensive hardware. Expensive as in more than $5k for the RAID controller alone.
  • Well any company that can afford that kind of storage and the software associated (11TBs plus the mirroring software would be 6mil easy) should also have someone smart enough in the architecture department who would understand do fiber multiplexing off to yet another box of disks for your hotbackup. Obviously they wanted a hotbackup if they needed disk backups and the mirroring wasn't enough.

    DLT would of course choke a horrid death on this. Although I have seen some pretty interesting DLT setups. The best was using another 4 port 100baseT card off a sun with each port doing a specific set of mount points to a DLT with 4 drives. That thing could backup a ton of data in one evening.
  • Ok, but they forgot to buy their backup system. A big server's no good if you can't get your information back after a catastrophe (e.g. is it sitting under a sprinkler head?).

    Good grief, what do you back up a terabyte onto these days?

    Jon Acheson
  • Sweet, finally enough disk space so I can compile mozilla staticly with optimizations!
  • I am in the process of building a fileserver for my little visual effects company. This article was extremely helpful and informative, although my solutions will be somewhat different from theirs.

    In particular, they claim to get 50 MB/sec transfer rates from their disk array. But, you can see that they also specify just a 10 MB/sec 100BaseTX ethernet; so 80% of that bandwidth is completely wasted.

    I was curious what people's solutions to this are. Does one just get multiple 100BaseTX ethernet connections to a switch, or are people going Gigabit ethernet from the fileserver to the switch?

    I plan to go the second route. There are many ethernet switches out there with one or two gigabit ports such as this 3COM switch [3com.com]. This should give 100MB/sec to the switch, and then the switch can distribute this bandwidth to all the client machines at 10MB/sec.

    At least this is a reasonable stopgap until Gigabit Ethernet is more ubiquitous.

    Any comments?

    thad

  • Commodity disk is $3 / gigabyte.
    A quarter of that in 2003.
  • Offsite mirroring? Most of the big storage systems like EMC or Xiotech can be mirrored to another unit somewhere else, and most can do hot copies of existing data sets, which can then be dump to tape at some drone's leisure.

    It's not an inexpensive solution, and it presumes you have a facility someplace offsite to mirror to. Storing 1TB is easy, managing 1TB isn't.
  • For the time/space functionality of offsites, why not just have three storage units: primary, offsite mirror, and the backup/copy one. When the backup/copy is done, pull out the disks. Send them to the vault, and insert another set of disks.

    We've been looking at re-doing our storage, and the ability to do backups transparently to disk is quite compelling.
  • I helped build one of these a few months ago. It had 16 of those 80 gig maxtors. The worst part is getting all those cables arranged... what a headache. Now i'm posting this message on another one I didn't build but is very similar.
    $ df
    Filesystem 1k-blocks Used Available Use% Mounted on
    /dev/sda1 80636044 2412332 74127544 4% /
    /dev/sda3 466820072 8225276 434881652 2% /home
    /dev/sdb1 20161172 6528 19130504 1% /tmp
    /dev/sdb2 531336984 6593064 497753516 2% /var

    I spent the last few days building a third one. It was all good until i tested some disks on it. The escalades weren't detected the disks all of the time. Yesterday was wasted attempting different combinations of disks and positions. I have no idea whats wrong. Worst of all the setup is practically identical to the machine i'm posting from which never had a problem.

    I've just been told that this machine i'm posting from cost us around $10K.

    I have to go put the rack together meanwhile send me some pr0n please i have the space.
  • ...and there's a space embedded in yours. Slashdot does this on purpose forsome perverse reason that nobody including the guys at Slashdot knows. C'mon guys... this misfeature has been annoying us for years. Please get rid of it!

  • I never doubted that there was a legitimate reason, but I hafta think there's a better way to handle it. Of course, since I don't have a suggestion, maybe I should just shut up.
  • The NetApp Filler's NFS does NOT work flawlessly, at least not with AIX. I had one in on a demo, and it works fine writing with many small files, but as you increase file size, there is a problem that pops up. I had the same problem using Linux as an NFS file server for AIX.

    As you increase file size the filler takes longer and longer to return write acks. The more ram cache you have on the filler, the worse it gets. The exact symptoms have under linux, but on linux I can get to a command propmt, and type 'sync'. As soon the prompt returns, AIX is off and running again. This does not happen if I use Solaris. I did not have the time to try any xBSDs. NetApp was confused, IBM was confused.

  • Backups are inherently cheap, as well (Score:3)

    by Plasmic ( 26063 ) on Friday July 20, 2001 @04:08AM (#73131)
    After careful consideration, I have determined that every terabyte of data you want to backup will cost roughly $5,000.

    Even if you don't do incremental backups, it's still a bargain. Hell, get some more of these teraservers and do software RAID between them; this is phenomenally cheap space, folks. For the price that companies are paying for this kind of space currently, you could buy tens of terabytes of space and make it octuple-redundant and still knock a heap of cash off of your capital budget.

  • Re:Backups are inherently cheap, as well (Score:3)

    by Plasmic ( 26063 ) on Friday July 20, 2001 @04:48AM (#73132)
    Tar up your terabyte, copy it across the Internet (or a private circuit, if you're really smart) to your off-site spare teraservers that cost $5k a piece. That's still an order of magnitude more cost-effective than shelling out the dough for Digital's disaster-tolerant FDDI remote clustering technology.

    The bandwidth costs are irrelevant since they apply to any backup/clustering technology, and it's quite obvious that it's cheaper to buy lots of $5k terabytes and spread 'em out across the country than any of the big guys' "commercial enterprise-class solutions."

    When in doubt, have your company buy 10 more terabytes just to be on the safe side. It's only $50k! Most manager-level positions at Fortune 500s can sign off on a purchase that size.

    This stuff is great! Give terabytes as stocking-stuffers to your kids at Christmas.

  • Re:SDSC != Netapp (Score:4)

    by Plasmic ( 26063 ) on Friday July 20, 2001 @04:17AM (#73133)
    I guarantee you that N, the cost for however many layers of redundancy it would take to make you comfortable with using these as production servers, where N = $5,000 * your preferred number of layers, is cheaper than your Netapp filers.

    A quick search on google yields this:
    $20k for a terabyte of rackmountable RAID5 [208.56.182.73]

    Believe it, folks: Terabytes really are as cheap as the Slashdot headline makes it seem like they are.
  • $3 /GB. I wonder when we'll start to measure the cost per TB. That would be $3000 /TB (roughly)

    ----------------------------------------------
  • 8 x 80gb is 160 gb not 1,000gb = Tb. Sorry. ;)
  • I just checked pricewatch - actualy the 80GB drive vs 100GB drive are better for price/size ratio eg:
    # * size = cost/mb
    10 & 100 = $2710, 1000GB
    14 & 80 = $2405, 1040GB (might need to add another controller)
    NIC: $20 D-Link DFE-530TX+ (Linux works great)
    $58 for 20GB Fujitsu HD
    $210 for Yeong-Yang Cube server case
    (as someone mentioned)
    $105 for 3 * $35 for 512Mb ram

    Worst part is this you NEED 2 - 3ware Excalade 8600 8 port contoller cards $345 each (8 drives per card) OUCH!!!!!
    (I don't see why these boards are priced more than a GeForce 3 - $299)

    The raid you are refering to only support mirroring, and only 4 drives per card.(4 on board- 4 card)

    (Yeong-Yang cube case)
    15bay-6bay(V)+2bay(V)+7bay(H)
    - ATX cube server case w/300w ps

    IMHO: I don't like the intel etherexpress - too many weird boards out there that don't work right (100b chipset)
  • Take a look at SnapFS for Linux. You can find it on Freshmeat.

    It does snapshots on top of most filesystems!
  • Oh, they've got that. That [slashdot.org] is a 4GB quad-CPU machine running IIS over Windows NT over quad fast ethernet serving static pages...
  • If you are lusting after a Daewoo... well.... I'm sorry.

    Real cars cost real money.
    --
  • $130 for 3x256? Nah. Pricewatch had 512MB sticks for $40 apiece.

    Now I feel silly that I dropped twice that on my half gig... sigh.

    -grendel drago
  • Is a pedabyte file server a file server for pedophiles?

  • SDSC != Netapp (Score:5)

    by chrysalis ( 50680 ) on Friday July 20, 2001 @03:57AM (#73142) Homepage
    1 Tb for $5000 is nice. Size is one thing, but the underlying software is another thing.
    Netapp filers are expensive, but excellent because the filesystem (designed by former SGI employees, who designed XFS) rocks. It's fast, it's damn reliable, and the "snapshot" feature kicks ass. Also, their NFS implementation works flawlessly (and this is a *rare* thing) .
    So SDSC has a nice project on the hardware part, but I wouldn't trust them for production servers.

    -- Pure FTP server [pureftpd.org] - Upgrade your FTP server to something simple and secure.

  • D-I-Y Fileserver (Score:4)

    by raynet ( 51803 ) on Friday July 20, 2001 @04:33AM (#73143) Homepage
    Hmm.. A Terabyte fileserver under 5000 bucks, that's easy (unless you are a SCSI freak):

    - 180$ for Motherboard ABIT KT7A-RAID
    - 138$ for 1GHz Thunderbird
    - 130$ for 3 * 256Mt Dimm PC-133
    - 71$ for GeForce 2 mx (Overkill for a fileserver)
    - 83$ for 10GB MAXTOR (for booting :)
    - 38$ for A CD-Rom drive (might be needed when installing OS)
    - 51$ for Inter EtherExpress PRO100
    - 58$ for HTP370 IDE Raid controller
    - 290$ for BIIIG Case
    - 2900$ for 10 * MAXTOR 100GB
    - 145$ for 10 * IDE Rack (It is nice to have a cooler for each HDD)

    Total: 4085$ for Big Ass Fileserver

    This leaves us 900$ (plus discount) for extra stuff like gigabit ethernet, monitor, keyboard and mouse etc.

  • In a non-free world (Score:5)

    by alexjohns ( 53323 ) <[moc.liamg] [ta] [cirumla]> on Friday July 20, 2001 @07:07AM (#73144) Journal
    Wayne and Garth, computer geeks, in an MS dominated world. (In some twisted alternate reality.)

    Wayne: "Hi, everyone. Today, we're gonna make a cheap 1TB file server"

    Garth: "That's right. We're going to be using the new Windows XP OS."

    Wayne: "Yes, Apple's stuff is too proprietary. MS is the open alternative."

    Both: "MS rules. Apple sucks. MS rules. Apple sucks."

    Wayne: "Alright, that was cool. Anyway, we've got an Intel chip {ed note: No AMD, either}, some RAM, case, power supply. We've just got one drive hooked up to see if this thing will boot."

    Garth: (Turns on power) "Alright Wayne, I'm installing XP. It will take a while. Commercials?"

    Wayne: "Yeah, we'll be right back."

    {Commercials}

    Wayne: "Alright, we're back. Had to call MS to register, but we're up. Everything looks good. It's a little slow, so we're going to add some memory while we add drives."

    {Time passes}

    Garth: "OK. Well, XP says that we need to re-register. Commercial?"

    Wayne: "Alright."

    {Commercial}

    Wayne: "Well, the nice lady at MS chastised us for changing our system, but she let us off with a warning." {W&G both laugh} "Anyway, we've got a new registration code and we're well on our way to 1 TeraByte of storage!"

    Both: "Woohoo! MS Rules. MS Rules."

    Garth: "Alright, so we're up to 256 MB of RAM, 200GB of hard disk space. We'll add a new controller, add some more memory since this thing still seems a little slow and we'll be right back."

    {Time does its thing}

    Wayne: "OK, we're back. Had to call MS again. They were a little peeved this time." {Shoots Garth a knowing look and both kind of chuckle} "Anyway, they were gracious enough to let us have another registration code. Thanks, Bill." {Laughs}

    Garth: "Yeah, and someone called and wanted to know why we need 1TB of storage."

    Wayne: "Yeah, like that's not obvious. Between us and all our friends, we've been to every major rock concert in a 500 mile radius of Chicage for about 10 years now."

    Garth: {Whispering, looking furtively around} "And, even though nobody knows..."

    Both: {Yelling in glee} "We've taped every show."

    Wayne: "Thousands and thousands of hours of rock. We're going to rip 'em all - Hey, Garth, 'Rip em all', is that a Metallica album?"

    Garth: {chuckles} "Good one, Wayne. Good one."

    Wayne: "Thanks. Anyway, we're going to rip 'em all, catalog 'em, rate 'em, and listen to 'em until we get sick of 'em."

    Garth: "OK Wayne. Things look good. It's still a little slow. Maybe a faster chip and some more RAM. Perhaps we should've gone SCSI. It should be alright for just serving MP3's, though."

    Wayne: "OK. We'll be right back."

    {Time, again}

    Garth: "We're back. Wayne's getting some water. He got a little hoarse begging MS for another activation code. For a minute, it looked like we weren't going to get it, but Wayne talked 'em out of one. Here he is now."

    Wayne: "Thanks, Garth. It's alright, just some red tape. No problems. MS rules!"

    Garth: {Sounding not so sure} "Yeah, MS rules!"

    Wayne: "OK, Garth what do we have?"

    Garth: "We got a faster chip, we're up to 512MB of RAM, and..." {drumroll} "...we've got 1 TeraByte of storage!!"

    Wayne: "Alright, Garth. Party on. Are we ready to serve up some MP3's?"

    Garth: "No. I think we're going to need to up the RAM again, this thing is still slow. Also," {Laughs ruefully} "We don't have a network card yet. No sound card, either."

    Wayne: {laughs too} "Alright then, Garth. Down one more time and this will be it. Right?"

    Garth: "Sure thing, Wayne."

    {The commercial break seems to last forever.}

    Wayne: "Sorry we took so long. We ummm... ran into a slight snag. It appears that MS won't give us any more access codes for 6 months."

    Garth: "Bummer, dude!"
  • Actually, yes, sometimes a budget server with massive disk space is just what one needs. I currently maintain a mirror site (various flavors of Linux, BSD, etc...). We can't really afford to utilize our most powerful server for this task (though it isn't too shabby of a box) but have equipped it with a 3Ware IDE RAID controller and 240-gigs of data space (it has a 9-gig scsi boot disk). (FWIW, one of the 2 120-gig RAIDs just ran out of space last night; going to move to an 8-port 3Ware =)

    Anyway, my point is that the data is totally expendable. All of it can be restored simply by running the same scripts that keep the data updated with rsync.

    Daniel

    ---

  • SuperMicro's 370DLE (ServerWorks ServerSet III LE Chipset) appears to be the low price point for these, with two 64 bit, 66MhZ PCI slots and a 4-slot 32-bit PCI bus, starting around $270 street price. Requires ECC registered SDRAM but supports up to 4GB of it, and up to dual Intel P3/933s.

  • Re:Not to rain on their parade (Score:5)

    by adjuster ( 61096 ) on Friday July 20, 2001 @04:31AM (#73147) Homepage Journal

    Get a motherboard with 4 IDE channels (most "raid versions" have this) and plug 12 drives into the Hotrods, 4 into the motherboard's raid channels, and 2 into the secondary ide channel. The boot hard drive goes on the primary ide channel.

    Ouch! Spend a few hundred more, and get an Escalade Storage Switch [3ware.com]. They perform very well and aren't wildly expensive (you should be able to have an 8-channel 32-bit, 33Mhz version for under $500.00). You also have your motherboard IDE channels free for things like DVD-ROM drives... heh heh... Lots of DVD-ROM drives... Heh heh...

    Ahh, yes-- and there are Linux drivers available for the Escalade controllers. If you're looking for wild amounts of performance, they do have a 66Mhz, 64-bit PCI version available, too. Wowza.

    Promise [promise.com] has their SuperTrak controller, which looks very interesting, but based on some messages I saw flying around on the Kernel List, apparently it's not as straightforward as just compiling in I2O support to use it under Linux. Grrr...

    Check out this review [storagereview.com] and this review [neoseeker.com] if you want to see how the Escalade stacked up to other "high end" IDE RAID controllers.

  • 100GB harddrives, $285 each on pricewatch. Attach them to a 3Ware IDE RAID card, wrap a dual PIII around it, and you've got a decent file server $5000.

  • .. (Score:1)

    by vbrtrmn ( 62760 )
    Admins report that yet another technology site has been taken out by the Slashdot DDoS attack. Authorities are still baffeled as to why a site promoting News for Nerds. Stuff that matters. would want to take down normally low traffic sites. Attacks seem to be focused on: open source projects, home hardware hacking, video game sites (especially ATARI), etc.

    Mr. Taco, if you read this, please cache these people's sites, before you cost them hundreds or thousands of dollars.

    --
    microsoft, it's what's for dinner

    bq--3b7y4vyll6xi5x2rnrj7q.com
  • I always thought it was $3,000...
  • I put together one of these babies (2/3 TB, $2500)a few months ago! 3ware is cool. They had a bug in their RAID 5 code, but the support/handling of the issue was exemplary.

    We put 8 80 GB maxtors and an Escalade 6800 in an old Gateway P166 tower with 32 MB, an extra power supply, 3 extra fans, and 2 NICs, running Linux 2.4 kernel running recent NFS code.

    The only trouble was when I tried to format the drives with Partition Magic; it couldn't handle the size and corrupted my partition table! It's now running great. A drive died once, but came back online; the rebuild ran flawlessly. PS whoever posted about using PowerFile should have checked the prices first.

    I have a hard copy of http://www.raidzone.com [raidzone.com] taped to the side of the box.

    It's used to send and retrieve large files over the 'net; I haven't even bothered to benchmark it, as the 'net will be the bottleneck.

  • Oh, and one more thing...because you are working with optical media you have a far less likely chance of data failure than with a bunch of expensive hard drives. Really all you have to worry about is fire and you can easily make a complete backup of the entire system by just buying another spindle of CDs/DVDs and a nice standalone burner.

    What I would use this system for is creating a single media center for my entire family. Everyone would give me their records/tapes/CDs, I would convert them to MP3s and burn them to the collection. They would give me their video tapes/DVDs, I would encode them as DivX or something and burn them to the collection

    Then any time they wanted to watch or listen to any record, tape, CD, video, or DVD it would be as close as the nearest computers (or entertainment center by making a nice TV-out PC). The originals could all go into storage in a nice, safe, dry place.

    - JoeShmoe
  • Well, if you took your own thumb out of your ass and used it to thumb through a CD burner manual you would find that any modern CD-R/RW drive can record data instantly in UDF packet format. So to the users it would look just like a normal hard drive. Since DVD-R/RW/RAM uses UDF by default this is even easier.

    Not to mention, the intent was to migrate an entire collection of data that would be offline storage to online storage without wasting hard drive space. If the data is a bunch of audio or video files it is pointless to have read/write access since the data doesn't change. If you need to "delete" it you can take the CD/DVD out of the unit.

    - JoeShmoe
  • I said CDs OR DVDs...the carousel could mix and match any type of disc you want just like the home theater versions (you can put CDs or DVDs in your Sony SmartFile and it will play either one). And as the above posted said, there are disc formats the same size that can holds GB of information.

    But you are still wrong that there is any sort of filesize limitation. The filesize limitation is a function of the Linux thinserver, not the disc hardware. After all, any modern hard drive can only store 1/6 of its data per platter but you don't have to worry about that, do you?

    If you wanted to save a file larger than 700MB then the thinserver would just have to know to split it into multiple CDs but present the file as a single directory entry (file.mpg instead of file0.mpg file1.mpg and file2.mpg).

    - JoeShmoe

  • I have a better idea... (Score:4)

    by JoeShmoe ( 90109 ) <askjoeshmoe@hotmail.com> on Friday July 20, 2001 @04:30AM (#73155)
    Combine:

    A carousel that holds 200-300 CDs or DVDs (just like they have for home theaters for around $800)

    Sort of like the stuff PowerFile [dvdchanger.com] makes. (http://www.dvdchanger.com)

    Add to it room for at least two slot-load drives (although four would be even better, one in each corner) so you can access at least two CDs at any given time.

    Throw in a Linux thinserver (like the stuff Linksys or any number of companies use) to manange the contents of each CD/DVD.

    Result? Over 1TB of storage for around $1000. The only catch is that it is not meant for more than a handful of users at a time.

    But considering what an equivalent RAID would cost it doesn't seem like a bad idea. You could put every file you've ever downloaded into one box and each CD/DVD could show up as a separate directory on one master volume. Imagine that. Near-instant access to TB of info.

    If you use DVD-ROM drives, it's 1TB of read-only storage but if you use DVD-R/RW/RAM drives then it is 1TV of read/write storage! Wow, you could open your own Avalon!

    Please please someone start a project to help build something like this! I desperately want one but the crappy PowerFile version is junk because it relies on crappy Windows/Mac software and ties up a whole computers just to access the damn thing. Plus having to manually mount/unmount the discs like individual drives instead of having just one volume.

    - JoeShmoe

  • On the bright size, this machine just MIGHT have enough disk space to allow us to install the next release of Windows (after XP).

    Nope....apparently M$ is predicting a 1 petabyte footprint for the next version of Windows.

    On a sidenote, M$ has just announced it's new IIS freebies [bbspot.com]

    ----------------------------
  • Word up! I bought a 1981 Mercedes 300D in 1991 for $5300. I've thrown more than that into keeping it running but it's a tough, easily repairable car with over 330,000 miles.

    Imagine if you could still use an 8088-based IBM PC.
  • > I mean, for 5000$ you can get machine to store your entire pr0n-collection!

    Speak for yourself.
  • Hey. Can people stop using lowercase b when they mean uppercase? 3800TB is a completely different beast than 3800Tb.

    Thank you, I'm done.

  • NT (Score:1)

    by jaxon6 ( 104115 )
    wow, i guess it's finally gonna be NT's moment to shine, with NT standing for Needs a Terabyte. that microsoft, it sure does know it's stuff. who woulda known that 5 years ago when NT was released that $5,000 machines would have a terabyte of disk space, ideal for NT. microsoft is a visionary in that sense.


  • it was funny, not flamebait.
  • ...was the sound of the bat connecting with your head!
  • The Escalade 7800 (8-port, 8-channel, UDMA 100, 64-bit) starts at about $350 on pricewatch. Two of these, and a dozen 100 GB drives in two RAID 5 arrays, and you've got your terabyte. The drives start at $271, so drives and controllers together are still under $4k. Still need housing and cables, of course.
  • Used IBM drives for reliability. Using Anything else seemed a bit of a risk. Hardware raid 5 in the chassis connected to a Fireport 40 on the server. ReiserFS is finally stable enough in 2.4.5 and 2.4.6 to make this useable. We do streaming video so this should hold a semester or two depending on the number of classes we serve. NAU Webcast [nau.edu]
  • Combine:

    A carousel that holds 200-300 CDs or DVDs (just like they have for home theaters for around $800)

    Sort of like the stuff PowerFile makes. (http://www.dvdchanger.com)

    Add to it room for at least two slot-load drives (although four would be even better, one in each corner) so you can access at least two CDs at any given time.

    Throw in a Linux thinserver (like the stuff Linksys or any number of companies use) to manange the contents of each CD/DVD.

    Result? Over 1TB of storage for around $1000. The only catch is that it is not meant for more than a handful of users at a time.

    But this would not work for files >~ 700 MB. I am sure that the SDSC simulations generate individual files that are a few GB in size (this is very common with simulations).

  • Optical jukeboxes like the above describes use 5 GB rw drives. (2.5 GB per side)

    Actually, the original poster refers to using ordinary CDs, which would have a smaller filesize limit. In any case, 5 GB is still a constraint the first time you try to create a file larger than this. This is a requirement for some of us.

  • Can't help but notice, though, that they didn't also test against SCSI chains using hardware RAID 5M Folks, always remember that you get what you pay for.
  • "The computer you want always costs $5000".

    I believe this to be true. My current machine, and the one before it, both cost that much. Laptops are just too darn expensive!!!

    Mind you that's $5000 Cdn, so that's what? $2500 US? ;)
  • Where are you getting this??? I have a Dell 2U server with their own hardware RAID controller - that does RAID 5 just fine - and the whole system cost less then $5000.
  • That's what a stripe is for!

    ===
  • I have another problem to bring up, regarding the cheap disk backup situation. Hard drives are probably not a good choice for long term archiving, because they're built to have mechanical moving parts, so the oil and bearings will coagulate.

    There's a company called Solid Data which does file servers based off of RAM chips for main storage, with an optional hard drive periodic snapshot mirror. They learned the hard way that they had to periodically and automatically spin up the drives, because after about a year of solid operation, they had a failure on one server and it wasn't able to snapshot the data out before the data was lost, because the drives couldn't spin up.

    The hard drives are a cheap high performance backup, but they apparently don't do well on a long term shelf. Does anyone here have more info on that phenomenon?

    This type of suggestion ( http://slashdot.org/comments.pl?sid=01/07/19/15542 16&cid=48 [slashdot.org]) is my favorite so far. I've been waiting for that for a long time! I don't like tapes!

    ===

  • Depending on the network it was attached to, or the processing load it was under, you might never know the difference. The throughput of the whole system is restricted by the slowest link.
  • 2 years of non-repeating music is a nice idea, however I dare you to find 2 years of non-repeating music worth listening to. Oh sure, the first month would be great, its the next 23 months that would be a bitch. Somewhere in the middle of the second month you know your going to be hearing is the words to all of Barry Manilow's music, then Christopher Cross, Dionne Warwick Paul Anka, etc... and worse yet, you hit the random button and your forced to listen to this shit while you track skip your way to a decent song. Yeah, it sounds cool but guess what, it isn't.
  • And when there's a fire, or an earthquake, or a lightning strike, you'll feel really dumb.

    Backups involve more than just having a redundant copy of the data in case of a drive (or a few drives) failing. Offsite storage and archiving are just as important - and while you could do this with the "build a second array for backup" approach, it's not really practical to move these heavy boxes around on a daily or weekly basis.

  • I always thought it was $3,000...

    Could be -- my subscription ran out a quite a while ago.. Why slog through John Dvorak's column when I can just hit 'reload' on Slashdot??

    :)

  • Computer, hell... the car I want always costs $5000! I should stop spending my hard earned money on misc. hardware and beer.

    Man, $5000?! What kind of car are you trying to buy, a Gremlin?? My four year old Civic cost me $10k!

  • Mind you that's $5000 Cdn, so that's what? $2500 US? ;)

    Somewhere close to that -- last time I was up in Canada, $1 US got you $1.45 Canadian. Playing the slot machines at Niagra Falls was GREAT because $20 US got you $29 Canadian. Sweeeeeet. Cashing out your "big winnings" sucked, though.

    :)

    Geez, it must be Friday - I've got my mindless rambling bit permanently OR'd on.

  • Machrone's Law? (Score:5)

    by sid_vicious ( 157798 ) on Friday July 20, 2001 @03:48AM (#73178) Homepage Journal
    Did this remind anyone else of Machrone's Law (which held up pretty nicely through the 80's and into the early 90's):

    "The computer you want always costs $5000".

  • I've made a copy of the file (minus images) since it got slashdotted within one minute.

    http://whatever.ii.net/mirrors/terafile.html [ii.net]
  • Nah. Just slow. Welcome to the world of dial-up :)

    It's still up.
  • http://www.zdnet.com/zdnn/content/pcwk/1439/pcwk00 03.html Machrone's Law is mentioned at the top of this article...
  • http://www.zdnet.com/zdnn/content/pcwk/1439/pcwk00 03.html

    There was a space imbedded in the last link given...
  • With a 1TB array you'll need lots of disks, no matter what type of disks you use. All those disks share the same PCI bus (I don't think you can get something with multiple PCI busses for <$5000). So the PCI bus will most likely limit the bandwidth.

  • How about... (Score:4)

    by Mik!tAAt ( 217976 ) on Friday July 20, 2001 @03:54AM (#73184) Homepage
    That's easy, but how about building a webserver that can survive the Slashdot Effect for $5000?
  • My bad :) -- 20/25 MB/sec locally. NFS was hitting less than that, but not much (100Meg Full Duplex.)
    The Qu+xum has spoken. Nyaaah!

  • Re:Speed? (Score:5)

    by DrQu+xum ( 218745 ) on Friday July 20, 2001 @05:00AM (#73186) Homepage Journal
    Well, here's how we did our 525GB server for $4200 last December (would've been 600GB, but we decided to go RAID5.)

    Key parts, both hardware & software:
    • PIII-933
    • 256MB PC133 SDRAM
    • 4 Promise ATA/100 controllers
    • 8 IBM 75GB Deskstars
    • 3c905CTX (We don't have Gigabit yet)
    • Linux 2.4.x kernel
    • ReiserFS
    • Big F*cking Case


    We went this way because of the nature of the files to be stored (mean size=120MB, many over 200MB), and their purpose (download once, read a few times, delete.)

    NFS read times are around 20-25 MB/sec, just fine for us.

    A few points:
    1. Use only ONE drive per bus -- remember, this is IDE.
    2. Get a Big F*cking Case (tm) with at least a 350W power supply -- you're going to get some serious heat from 9 drives (the 9th is a 45GB drive, our boot device), so you'll need some pretty decent cooling going on.
    3. There is a hardware hack (check Tom's Hardware) that'll make the Promise ATA/100 controller into a IDE RAID controller -- try at your own risk. We didn't feel like messing with it.
    If I get some pictures up, I'll post the URL later.
    The Qu+xum has spoken. Nyaaah!

  • unfortunately, we wont have our own libraries, because all information will be via encrypted subscription services :(

  • As of 10:00 (~3.25 hours after the first post I see [slashdot.org] [6:53] mentioning their getting slashdotted.) The site is still slashdotted. There's no google cache, but here's some info to hold you over (chances are it refers to the same thing Michael does):
    The heart of the storage component of the Active Web infrastructure is a terabyte server which will come online in Fall 2000. Because of the cost of commercial terabyte storage systems and the need for OS-level monitoring and customization, it was decided to build the departmental terabyte server from commodity components. The terabyte server itself is a dual-processor Dell PowerEdge 2400 running BSD/OS 4.2, with a AMI MegaRAID Enterprise 1600 hardware RAID adapter (four channel, Ultra160), and a link to the gigabit switch. The disk subsystem consists of two RAID enclosures of nine, 73GB Ultra160 disks each. Tape backup support for the terabyte server is provided by a generic PC connected via a two channel Ultra160 adapter to a dual drive Hewlett-Packard SureStore 2/40 tape library with a storage capacity of 3.2TB. Terabyte file systems will be exported to departmental UNIX systems using Kerberos authentication, and to Windows machines via Samba."
    This is from a ucsd "Active Web Equipment Infrastructure Plan" page. [ucsd.edu]. Found by google-ing "terabyte web server SDSC" [google.com] (since I couldn't find a google cache on michael's URL, but he says SDSC did it).
    If anyone managed to snatch a copy of the original before we went down, that, of course would be ideal. Mirror, mirror, anyone?


    Normal people, ignore below.
    -----
    "These are for the goat-weary:" (Though I don't know why people do this -- any decent web browser displays somewhere the target of a link before you click it. These people can viewsource and copy the format of my post for doing this, and stop cluttering your message body with plaintext URL's!)
    http://slashdot.org/comments.pl?sid=01/07/19/15542 16&cid=14
    http://www-cse.ucsd.edu/Department/ActiveWeb/Backg round/Equipment%20Plan.html
    http://www.google.com/search?q=terabyte+web+server +SDSC

    ~
  • Is a pedabyte file server a file server for pedophiles?
    No. It's disk space for the spelling challenged.

    We need all the disk space we can get for our spell checkers now that there are so many new words in the english language [cnn.com].


    --

  • But I wanted a Pedabyte file server... (Score:3)

    by hillct ( 230132 ) on Friday July 20, 2001 @03:54AM (#73190) Homepage Journal
    OK, so $5000 is a good price point but a terabyte really isn't enough space these days. There's more porn out there in the workd than that...

    On the bright size, this machine just MIGHT have enough disk space to allow us to install thenext release of Windows (after XP).

    Oh, wait, maybe it won't matter because by that point Windows will be a remotely hosted subscription service.


    --
  • 18(*) WD 60 GB hard drives ($2250)
    3 Hotrod ATA/100 controllers ($180)
    Powered enclosure ($200)
    Custom extra-long IDE cables ($150)
    A geek who can put it all together (priceless)
    ----------
  • They were planning on having a presence in two data centers, with mirroring between the two. This was a Fortune 10 company - they could afford it.

  • pr0n? Sheesh. mp3! (Score:3)

    by sacremon ( 244448 ) on Friday July 20, 2001 @03:55AM (#73197)
    A Terabyte of mp3's comes out to be around two years worth of music, if you encode at 128kbps. Fill up the server, play till it ends, then replace with the one Petabyte model that will be out by then.

  • Re:Backups ? (Score:3)

    by sacremon ( 244448 ) on Friday July 20, 2001 @03:58AM (#73198)
    A potential customer of my company's (we do web hosting, amongst other things) wanted to be able to backup to tape a completely filled 11.7TB EMC unit. Nightly. A consulting firm that they hired figured it could be done using the proper type of tape (not DLT - far too slow), but it would require a pallet of tapes each night. This evidently didn't faze the customer. I don't know what they finally did, and they didn't host with us.

  • Re:Backups ? (Score:3)

    by sacremon ( 244448 ) on Friday July 20, 2001 @10:04AM (#73199)
    Actually, they wanted tape backup, which would then be stored securely offsite.

    It was sort of funny dealing with them on the tour of the facility. They wanted to know how long our cooling would last if someone threw a satchel charge over our fence and took out the cooling towers. They also didn't want to host in two centers on the same coast, regardless of distance, in case a hurricane came along and took both out.

  • Re:This poses an interesting question... (Score:3)

    by dasunt ( 249686 ) on Friday July 20, 2001 @11:53AM (#73201)

    I have 5 gigs of music myself, non-repeating (it isn't too hard to find, actually). However, here's the real hard drive killer: video.

    I'm into anime (yahoo), and my full/almost full collections of Ranma (TV Seasons 1 - 5 + OAV), Tenchi, O!MG, Lodoss, Evangelion, Lain, and the like are currently killing my 40 gig hard drive. Some of the full length movies (fan-dubbed) can run to half a gig alone. That will quickly kill alot of hard drives.

    OTOH, I'm seriously considering 3 40/80 gig hard drives and IDE RAID 5 for my next system. :)

  • Re:SDSC != Netapp (Score:3)

    by daniel_isaacs ( 249732 ) on Friday July 20, 2001 @06:42AM (#73202) Homepage
    It's hardly the same. A TB on a Netapp might cost 300,000, but it's worth every penny.

    Sure, you can get just as much disk space for an order of magnitude less. But you won't get their reliability, or their feature set. We use them exclusively for our storage needs. We have about 8TB of space on them right now. At anytime we can retrieve a recently deleted file or directory, do hot bakups, make new containers on the fly, hot swap out bad drives, add a new shelf of drives, chew gum and walk.

    In a production environment, where the data on those servers is the lifeblood of your company/organization, some DIY IDE RAID setup will not withstand the demands, or come close to yielding the results of a NetApp (or EMC, or Xiotech, or Compaq, etc..)

    These IDE RAID setups are fine for your mp3's or your personal or small workgroup fileservers. Just don't bet the farm on them.

  • Speed? (Score:3)

    by moz25 ( 262020 ) on Friday July 20, 2001 @03:51AM (#73205) Homepage
    Does anyone know how the speed of this system would compare to one using SCSI drives instead of IDE drives?

  • Re:Terabyte? (Score:3)

    by Marcus Brody ( 320463 ) on Friday July 20, 2001 @03:50AM (#73209) Homepage
    Hey, I might even be able to run Office 2000 now....
  • So if you cold hold 2 years of non repeating music on a $5000 file server, would you actually do it? Think about it, how much time would you have to invest to actually go find 2 years worth of MP3s that aren't the same. I have about 10GB myself and I still haven't gone through all of them to remove the duplicates. Anyway, forget about this making better radio stations or whatever, even if they wanted to broadcast using mp3 or some other compression format, they wouldn't use more than 50 songs, just listen to any station out there...
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  • What you could do with a 1 Terabyte server (Score:3)

    by Dexter77 ( 442723 ) on Friday July 20, 2001 @04:34AM (#73213)
    Let's see what 1 Terabyte could hold:
    16700 Full length MP3 albums
    1430 Divx dvdrip movies with surround sound
    700 Divx dvdrip movies with AC3 sound
    200 DVD movies

    A Guy with a T3 and 1 Terabyte fileserver could replace a fullsize videostore or even a library. Then again he could even have a radio station that would broadcast non-stop music for 2 years and none of the songs would be played twice.
    Funny how the world turns around when technology advances. Few years from now and that server costs only 1000$. Then we all have our own videostores and libraries.. I wonder what happens to those public ones, that have been so popular till now.

  • Backups ? (Score:3)

    by john_updyke ( 453831 ) on Friday July 20, 2001 @03:53AM (#73218)
    I'd really like to read the article but it's slashdotted or traindotted but what about backups ? At least here in Hawaii the DLT drives cost a bundle and what's the use for a terabyte of data that could go away any day ?

    It's like proclaiming /dev/null as my trillion-terabyte disk array.

  • Slashdotted? (Score:5)

    by pj7 ( 469369 ) on Friday July 20, 2001 @03:56AM (#73229)
    Less than 10 comments at 8:00 am EST and the sight is unreachable. Look out Code Red Worm, The Slashdot Effect is not to be trifled with!

Slashdot Top Deals

He has not acquired a fortune; the fortune has acquired him. -- Bion

Close