Ask Slashdot: Linux and Fibre Channel Storage Systems 149
"The connection to the Clariion storage sub-system will be made using point-to-point Fibre-Channel via two Emulex LP8000 64-bit PCI host bus adapters over copper connections/cables. The FC interfaces and a Gb NIC will be plugged into a triple-peer 64-bit PCI bus. The whole thing should be able to sustain approximately 120 simultaneous MPEG feeds at 1.2Mbps/feed on our 100baseT LAN."
Eventually, we will be clustering several servers in order to increase the number of simultaneous feeds, and also plan to expand video-on-demand service (educational only, primarily for foreign language curriculcum) to the dorms and other locations on campus, possibly the Internet at large if permission is granted from the video title's author (sorry guys, no free StarWars d/l's). Our initial storage capacity will be 180 GB (approx. 144 GB w/ RAID 3 striping). But we expect to scale up to 2 TB in the next two years.
Another advantage to using Linux would be its clustering support (under Caldera for example). We will need that capability soon, and Microsoft still seems to be dragging behind, with only a handful of 3rd party solutions available."
Somehow I don't think either are the solution (Score:1)
Obviously part of the question is, which of the above three OS's can work with the hardware. We know NT can, it sounds like Linux might, and I would assume that Solaris can.
Right now with a just 144Gb, any of the OS's can cut it, but I've serious reservations about either NT or Linux being able to scale up into the Terabyte range and be able to still perform well.
Oh, and yes, I do have experience with LARGE fileservers. I play daily with servers that range from 300Gb to 1.6Tb.
Emulex Driver (Score:1)
-John Justice
justice@quantumres.com
Quantum Research Service
why (Score:3)
CLARiiON RAID for Linux Home Page (Score:4)
Please moderate this up. They used the Emulex LP6000 not the 8000, but depending on how different the card is the driver may work or may be easily tweaked to work.
I wish you all the luck and love Linux but did you look into getting a Sun instead?
Re:several things (Score:4)
One thing to consider: NT is not good at handling large filesystems. According to many people, once you get past 100 gigs of storage NT becomes massively unstable (gee, and it isn't for less than that?!). Linux is not ideal here either, especially for video on demand, where you will run into the 2gb file size limit.
The "safe" bet here, that would avoid the massive instability of NT when dealing with large data sets and the incapability of Linux, would be to go to Sun. You say you are in an educational environment. Sun has approximately 50% discounts in that environment for many of their products. In one case at CUNY, Sun quoted a system for the exact same price that an equivalent high end Linux system would have costed from VA Research or Penguin. Sun also has very good support in the educational environment, because they want to "seed" the field.
Go distributed (Score:5)
What happens when something... blips.
What happens when you need to do system upgrades?
What happens when you need to do maintenance?
Building one big fancy server is -- in my opinion -- a really bad approach to solving this problem. Sure, I'd love to have a box like the one you describe. It sounds cool. But having the coolest box on the block does not necessarily build the best solution to the problem.
Suppose instead that you bought a small farm of rack-mount PCs. Equip each with a nice 10K RPM SCSI hard drive and a 100BaseTX network interface. If running Linux or *BSD, such boxen could get by with CPU and memory that is modest by today's standards. NT, well, will require more to do the same job. Such boxes can be put together for around $2k a pop. My guess is that you'll get much more aggregate horsepower building it out of a cluster of such boxes than the same amount of dollars would buy you in a big fancy server.
Then you simply make your software able to cause the video to stream off the box that has it. You might also add the cability for the boxen to automagically clone frequently used streams from their home machine to others and load-balance.
If a server goes down, you just restore its data off of backups onto some of the other ones and shuffle where things are homed.
If you need more capacity, to a pretty high degree, you just buy more inexpensive boxes.
This would of course require some clever software to make it all work. Your application sounds like one where some custom software is involved anyway, and it won't be that hard to do the stuff needed to pull off a clustered solution.
Trust me (Score:2)
Sun is a great company, and will be around for a long time.
I'd suggest the Enterprise Server 4000, as you can fit 3 or 4 processor/ram boards in it, and still have enough slots available to talk to 8 or more storage arrays. You can also get your quad-fast-ethernet card, and it works like a champ.
Solaris is designed to work with lots of processors, lots of ram, and lots of disk space. You'll have much more success than you would clicking on stuff in Windows NT.
We have Enterprise Servers here running several hundred TB's of data for all sorts of things... oracle databases, simulation data files, testing data files... it works great. And you don't have to sit at the thing in order to administrate it.. just use your favorite method to export your display back to your local X machine (or use the text utilities).
Re:Sick of this kind of crap.. (Score:1)
You're language doesn't worry me, I just didn't understand what you wrote.
Is Linux really the best solution here? (Score:2)
Nice try, MS shill (Score:2)
bull****.
My brother is NT-certified. He installs high-end NT hardware to monitor custom-built RTU's (real-time control units) that control things like oil pipelines and utility transmission lines. They do not take chances with these things, they use Compaq servers with NT pre-installed (thus avoiding the biggest problem with NT stability, buggy device drivers).
Yet every week a server crashes. For an oil pipeline, thousands of dollars worth of oil can flow through the pipeline, unmonitored, unbilled, while NT reboots and rebuilds its filesystem. Sometimes they even have to fly someone out to an oil rig in the Gulf of Mexico to get NT back running.
Point: If reliability is $$$, NT ain't it.
Meanwhile, our Linux servers just run. And run. And run. It was a pain to get some of the services going (NIS in particular was painful), but once they're going, they just run.
Workstations are even easier. I just installed Red Hat 6.0 straight off the disk, told it to use NIS for passwords, and voila. Then NFS-mount
-E
Would NIC binding make a difference here? (Score:1)
Just curious...
Re:CLARiiON RAID for Linux Home Page (Score:2)
Actually, the LP series cards maintain the same API, much the same way that a family of processors maintains a consistent instruction set. The code will seemlessly port (hopefully).
Re:Trust me (Score:1)
o I'm not sure if clariion will support multiple hosts attaching to one of their controller ports.
o If you're using dynamic multipathing w/in veritas' volume manager then you may have to be extra careful w/ your hardware and software setup. I don't know if veritas has confirmed that this works on pci-based fibre interfaces yet for their product.
o If you don't buy a sun A5100 or so array then you have to buy veritas' volume manager to achive storage sharing. However this is worth it b/c it will save you the cost of buying another pair of controllers on your clariion system.
Man, I love big storage.
-Peter
Re:Sick of this kind of crap.. (Score:1)
-Peter
Don't use Ext2 here (Score:1)
Fibre channel video streaming from multiple server (Score:2)
A plug for the company that pays my salary:
MountainGate [mountaingate.com] sells a filesystem called " CentraVision [mountaingate.com]" which sounds like it might be similar to what you're looking for. It is designed for streaming video and shared fibre channel RAID boxes; there are clients for Irix 6.2 through 6.5 and Windows NT. Partitions can be customized for specific types of media, and performance is top of the line. It is a distributed file system, so there is a lock manager but no server - clients load data directly from the drives. Thus it is scaleable.
This system gets you two things: a filesystem capable of handling large amounts of data with grace and speed, and a copy-free way to share the same data between more than one video server.
If you're not going to use more than one video server, why not look into SCSI-3 instead of fibre channel? The whole point of fibre channel is that multiple workstations can have direct access to drives. If you don't need that, SCSI-3 gives you 160mb/sec transfer rates (as opposed to fibre's 100mb/sec), and SCSI equipment is generally cheaper and easier to find than fibre.
There is no linux port of CentraVision yet. I'm writing this in hopes that you'll phone up the marketing department at 800 556 0222 and ask for a Linux port, thus making it easier for us linux-friendly engineering types to get management to approve the project... :-)
(Just don't ask them about the MacOS port. That's my department, and I'm getting tired of them asking when it's going to be ready!)
-Mars
Interphase FC Card (Score:1)
UNH has developed an independent driver that will be posted on their website, and I will be providing an Interphase supported driver soon.
The Interphase supported driver/hw is being used at the Univ. of Minn. GFS research project. Interested/serious beta site users can send email to mark@iphase.com.
Re:Interphase FC Card (Score:1)
Check out Pluto Technologies (Score:1)
http://www.plutotech.com/
Re:Sick of this kind of crap.. (Score:1)
With Linux on the other hand, you have developers that have the potential to create something better than Microsoft could ever dream up in a million years (even with their billions of dollars), however those developers don't have the millions of dollars to build and test Linux for many of the high-end solutions needed by companies/research groups. When a new technology emerges unless that company wants to oppose Microsoft, they have to allow Microsoft to have access to the hardware to build their drivers, but most of them won't don't open up because they don't want their competitors to get their proprietary information from open source code in the kernel or drivers.
Most Linux solutions are created on the fly with real live systems. They don't have the option "Oh lets spend 3 months build and test these drivers/systems".
The next few years are going to be trying times for both Linux and NT. They both have the potential to win. Linux I know and hope will put up a fight more dreadful than anything Microsoft has every seen. Then they will come to realize money is not everything.
Re:several things (Score:1)
I still hate NT, but I haven't found any NT OS problems with huge filesystems. Backups and checkdisks and defrags, that's another matter.
Re:Go distributed (Score:1)
2G filesize limit (Score:1)
We need to make a FAQ for Linux's limitations.
Linux's 2G file limit is caused by the VFS on 32-bit Linuxae. I think it doesn't apply to Alpha and UltraSparc, though I'd have to verify.
Matti Aarnio's Large File Summit [tmt.tele.fi] patch takes care of this limitation on x86, PPC and MIPS.
Maybe a Network Appliance Netfiler instead? (Score:4)
We've been talking with Network Appliance. Their NetFilers look pretty good. They claim to be very fast, and are highly scalable. They natively speak both SMB (NT's file system) and NFS (for Unix). ALL they do is storage. You can't program them; they're highly specialized rackmount machines that tie into your NT domains (and presumably your NIS domains, though I didn't ask that) pretty much seamlessly. You can organize your available space as Unix, NT, or shared.
It has one very, very cool capability in the filesystem. You can take a snapshot on a given day/time, which basically means they copy all the inodes at that time. If anything changes subsequently, the original drive data is preserved, and only the new data is written. The inodes in the fresher filesystem are updated to point at the new data, where the old data is still there, being pointed to by the old inodes. It gives you an online revision history, which would be exceptionally nice right after one of those "*doh, I just did rm -rf
Another real advantage we see to it is that because it's a specialized device, if something else on the network needs to be taken down for maintenance (say, the backup server), the network storage is unaffected. The only thing that's at all likely to take your storage down is, well, storage problems. And it's fully hot swappable and running fibre channel internally. It looks really good.
I think you're talking around $100K - $120K or so for a midrange box with 150GB of storage. I believe they presently scale to 1.5TB, and are building ways of scaling further -- they claim much further. Tons of options for high speed networking on it, too.
It's just a thought -- instead of doing what you are doing, which is using PC hardware, switching to a specialized device, with its custom, highly reliable hardware, might be a very appealing solution -- as long as all you're doing is streaming the data off the server. If you need the server to fiddle with the bits before they arrive, you'd have to have another server in the way, which is probably defeating most of the purpose in a specialized unit anyway.
My $0.02. I don't have actual experience with these yet. If anyone does, please chime in.
Re:Sick of this kind of crap.. (Score:1)
>> people seeing how much "linux sucks"..." Why?
>> If Linux sucks, why do you not want people to
>> see that, and share your attitude?
Actually, I think that is the only halfway sensible comment he made. While I agree with you about not covering up, it is also not a good idea to promote Linux for something it is not good at.
Now, i'm not an expert on Linux in highend hardware like that, or Linux and video but if Linux is not the way to go for this situation (which seems to be the overall tone of the responses I've read) then we shouldn't advocate Linux for this situation. Why set ourselves up for failure?
Or maybe I have no idea what i'm talking about... which very well may be the case 8 )
Steve
Re:Is Linux really the best solution here? (Score:1)
Why? Surely Linux can't be expected to fill every OS niche out there. It seems to me that peole what Linux to be able to do everything. It cannot. There are some solutions to certain problems that don't involve Linux. I'm a Linux user myself, but I'm not blind to the fact that it isn't perfect. Then again, what is?
Fibre Channel Experiences + Suggestions (Score:1)
I agree whole-heartedly with some of the people here that a different hardware vendor would be favorable. SGI machines generally have much higher bandwidth bus architectures, thereby are more suited for your particular application. They may be expensive, but (I'm no expert here) 1 SGI machine may be worth 2 or more PC's doing the same streaming.
If you must use Linux with PC hardware for the streaming, I'd say go with old-school SCSI. The stability of the SCSI drivers is excellent. There are very large SCSI drives available now, so the 127 device chains of Fibre Channel become a minor selling point.
-Brendan
Re:Yup, Linux fibre channel drivers exist (Score:1)
When EMC came to my company to pitch your products about 4-5 months ago, I asked about Linux support. "What's Linux?" was the wrong answer, and yet, they gave that as an answer. At the time I told them they should seriously be looking into Linux as that was part of the decision of bring EMC into our organization.
I'm hoping things have changed at EMC....
Re:why (Score:2)
> A hundred grand could build a whole LAB of computers for students to use.
This sounds like exactly that.
Re:WARNING (Score:2)
> Linux only supports 1 GIG... but I have heard some people say that it supports 2.
Erroneous. NT and Linux both have the same split memory model, 2 gigs for user, 2 gigs for kernel. A patched kernel on NT allowes a 3/1 split, as does a patch for Linux from SGI. It's quite disingenuous of Microsoft to push this line, and as it's more easily disproven than performance numbers, it's likely to bite them in the ass if they continue to.
How bout' cards? (Score:1)
perhaps sun? (Score:1)
I love Linux, but at the same time, I have to be realistic about its limitations. Linux (and every other operating system) isn't the best tool for every job...
Standard Disclaimer: I do _not_ speak for my employer.
Re:... Or SGI (Score:1)
hee hee.
-s
Standard Disclaimer: I do _not_ speak for my employer.
Re:Maybe a Network Appliance Netfiler instead? (Score:1)
I have some experience with high end NetApps, and I think you should give them serious consideration for this kind of application.
My recommendation is to go with a cluster / failover pair of F760s, which can have 1TB+ of storage between the two of them.
In normal operation, each filer of the pair serves 1/2 the files. If one dies, the other does file serving for both by taking possesion of the other's disks. This pretty much happens seamlessly.
As a test, we started copying a large directory tree to one filer, and then turned it off mid-way through. After about a minute, the other filer of the pair assumed control, and the copy continued without interruption. After the copy finished, I did a file-by-file comparison of the original and the copy (GNU diff rocks) and they were exacly the same. Very cool.
In addition to cluster/failover, the NetApps have quite a number of other high-availibility features: multiple fans, good RAID subsystem, snapshots and checkpoints, journaling filesystem, etc.
I love Linux (set up the 4th box for my home this weekend), but it's not ready for this kind of task. It's much better to go with an established vendor, instead of spending a lot of time trying to build it yourself.
Please feel free to contact me if someone has more questions about Netapps. There are also a lot of good technology papers on their web site http://www.netapp.com [netapp.com].
James Graves
Re:Is Linux really the best solution here? (Score:1)
Avoid DPT (Score:1)
However, DPT Fibre Channel has been an unmitigated disaster. Unless you purchase DPT hard drives, you are basically on your own. The latest Seagate 18 GB Drives (ST318203) do not work at all ("oh, we haven't tried those yet"). Neither do the older Seagates (ST118202), despite tech supports assurances to the contrary ("We know they work, they've been tested"). Don't even get me started on the inconsistencies of what tech support claims ("we never told you that would/wouldn't work", usually as part of a "it's the drives, not the controller" refrain, and in direct contradiction to earlier statements, with the offensive implication that I am somehow lying about what I was told earlier.). Despite Seagate's extreme helpfulness, well above and beyond the call of duty, it looks grim. Unless you are feeling particularly masochistic, or enjoy having your project used to debug their hardware, I strongly suggest avoiding DPT for your project.
In fairness, there is a (slim) possibility that the controller is defective -- a replacement should be in later today or tommorow. I will follow up with a note mentioning the success or lack thereof. If we are not successful getting the replacement DPT controller to work, we will probably return all of our DPT equipment and abondon fibre channel as too bloody a technology (still, even after two years) and look into a Mylex Ultra2-wide RAID solution instead.
Re:Avoid DPT (Score:1)
UPDATE: DPT OPERATIONAL, ABIT MOTHERBOARD AT FAULT (Score:1)
In light of the recent, extensive help the folks at DPT have given me, as well as the folks at Seagate, I retract most of my negative comments about their service, and all of the doubts I raised with respect to their product. It seems I had the misfortune of getting one grouchy tech on the day the question was posted to
In summary, the DPT Millenium Fibre Channel (dual port) controller is working fine with the Tyan DLUAN1836 Motherboard. It had problems with the ABIT BH6 Motherboard. Seagate disks work flawlessly, as long as they have the latest firmware (ST118202 v 0006 or later and ST318203 v 0002 or later).
Re:we are talking VIDEO so its SGI chaps (Score:1)
I am working with a major telecom company who has invested about $1 million in SGI hardware and configuration services, and I have to say that I am unimpressed, at least so far, with SGI's MediaServer thingie. Go with SUN.
Re:Maybe a Network Appliance Netfiler instead? (Score:2)
That said, I wouldn't necessarily recommend them for this application. NFS does *NOT* scale for most applications. It's great for distributing low-to-medium workloads, but I'd question the ability of the box to perform for such a large data set. (mpg streams) One of the major benefits of NetApps is the internal caching, which works very well for read-intensive applications. In some benchmarks, and occasionally in real life, I've seen data return from a netapp faster than from local 7200 rpm disks. However, with multi gigabyte streams, the cache will be worthless.
My only personal experience with FC-AL has been with Sun E4500's and MTI Raids.. similar to EMC, I'd imagine. They work well, and using Veritas products you can do high-availablilty and easy data relocation.. something to think about long term. (i.e. the cost for a mid-range quad proc AXmp, ~$35-45k could be offset in the future when you need to scale..)
I guess to summarize: for workloads like web serving, Netapps work great. For email, locking presents issues -- its not bad with Solaris, but its a joke in linux. For the initial application of large mpeg streams, I'd go with an FC-AL raid array, using dual loops on dual fail-over hosts. If availability isnt an issue, then 1 host, 1 loop. (And if performance is much more important than cost, use switched Fibre Channel)
-lynch
Sun Microsystems (Score:1)
Linux and big multi-processor machines (Score:1)
Oh, and BTW, I realize I've said some things that may get me flamed (I mentioned the Mindcraft benchmarks!) but hey guys, I'm a Linux fanatic as much or more than most of you, but you gotta be realistic.
we are talking VIDEO so its SGI chaps (Score:1)
you want it fast
!!!! you dont want NT unless your hardware can get round the prob
SGI is where its @ people
all the rest are like patchs and fudges to make linux do this or research for the future which is a good thing.
but SGI do this for a liveing
THATS THE DIFFERANCE
a poor student @ bournemouth uni in the UK (a deltic so please dont moan about spelling but the content)
we are talking VIDEO so its SGI chaps > get it (Score:1)
SGI make you pay and well they should if you want supercomputers get a SGI
ok their software is top notch you are just no familair with it thats all a SGI user can walk up to it and expect it to be the same
you have to learn a bit
john jones
a poor student @ bournemouth uni in the UK (a deltic so please dont moan about spelling but the content)
Re:WARNING (Score:1)
What??? Care to explain that...what the current per hour rate at M$ for support?
Re:Maybe a Network Appliance Netfiler instead? (Score:1)
Can't really name names because its not my company - but I belive that they have about 1.2M subscribers, which should narrow it down some *grin*.
Their solution is almost entirely Linux based using Network Appliance stuff for all of thier filling and some other stuff.
Anyway to cut to the chase - my friend swears by these NetApp and uses them with nothing but Linux.
Anyone needing to know more mail me and I'll forward the messages to my friend.
Tom
Re:64bit pci (Score:1)
This is a good enterprise class machine. Quad Xeons and up to 4Gb RAM. 2 Ultra2 SCSI and 1 fast SCSI on the motherboard. Built in hot swap Ultra SCSI. No ISA slots (no big loss there
Ext2 and large filesystems (Score:1)
Also, when dealing with large Ext2 file systems, they take forever to fsck, mainly due to the number of inodes. You can help overcome this by reducing the number of inodes. For example, if you know that the average file is 10MB, then maybe only have one inode per MB (instead of one per 4K). You should also use sparse superblocks (which requires a 2.2 kernel).
Re:Distributed servers, consolidated storage (Score:1)
Also, redundancy is the key to any high-end storage system. While the individual system will have redundancy (mirroring, raid, etc.), you also want remote mirroring to a separate storage system, possibly in another city for disaster recovery.
My obviously-biased viewpoint is that while you may have had some trouble with your EMC setup, in general, EMC is more reliable, faster, and more proven than solutions from any other company.
Yup, Linux fibre channel drivers exist (Score:2)
Note, I work for EMC, which also makes large multi-terabyte storage systems.
GFS: A filesystem for fibre channel storage (Score:2)
http://www.globalfilesystem.com/ [globalfilesystem.com]
Distributed servers, consolidated storage (Score:2)
The point is that you can't get the performance and reliability out of a small storage system that you can out of the enterprise storage systems. Of course, in some cases, you can build a system based on replication of the data, which for static data may work, but often as not, when all the costs are factored in, you're better off with a consolidated system.
The fun of Ask Slashdot (Score:2)
So if you're interested in fibre channel, multi-terabyte storage systems, media servers, and such, there's probably going to be a lot of interesting stuff here.
And as to your point, Linux may be a good solution if they can figure out the server architecture. They're already talking about using a cluster of servers, so the CPU power isn't a big issue. I wouldn't assume that not knowing about the level of fibre channel support indicates that someone is clueless about Linux in general.
Video Server - Hand-built or Off-the-Shelf (Score:3)
Bandwidth? Machine costs? or your time?
It really puzzles me how people wish to skimp on the hardware then set themselves up for later hassles and risk of (expensive) failure.
Lets take a look at a baseline video server from SGI, Origin 200 with MediaBase [sgi.com] with say 100 seat configuration, web management tools + 200 Gbyte FibreChannel and network bits would cost about $50K minimum upfront (guesstimate here based on educational discounts and extrapolation of bits and pieces we've purchased over the years) + 10% maintenance/year. Extra for their FailOver system.
OK, now the hand-rolled Linux version. You need to look for
a) streaming software (Darwin?) for multiprocessor
b) decent high-end file system (port SGI XFS?)
c) tuning the sucker for the best SCSI and network parameters
d) video library management software (none as yet, perhaps someone port SGI OpenVault?)
e) system management to monitor the whole thing
Lets assume you've got a collection of genius hackers at 1 man-years worth at each task, working for nothing except glory, you can probably get it done for $20K and 5 man-years worth of pizzas and coke.
Cheap at that price.
Rule #2 - If you don't know what you're doing, make sure you get damn good advice from people who've done it before.
Rule #3 - You pay peanuts, you get monkeys.
LL
Re:Maybe a Network Appliance Netfiler instead? (Score:1)
We use Unix connectivity only (via NFS) and our PC's connect via samba to the NFS mounts. SMB as an option is supposed to be pretty robust but we dont need it (or want it - too many virii exploring shares nowadays). NIS isn't used here (thanks be) but it shouldn't matter. We copy the necessary files (like passwd and hosts) to the filer via one mount point inaccessable to mere mortals and that's it. All NFS connectivity is controlled with automouter maps.
NetApps are just too sweet. Expensive, yes, but sweet.
Hardware Limitation (Score:1)
You can rule out both Linux and NT right off the bat because PC hardware is not intended to do what you describe. It isn't that it can't do what you describe, it's just that it isn't going to do it reliably in any sort of cost effective way. Like previous posters I would suggest SGI, Sun, or maybe even IBM, Hitachi, HP, or a few others. Now you're into different OS because of the different hardware required.
Remember that PC's were/are designed as light to medium duty machines.
What you're doing sound like something that's a lot of fun to do if you have a spare $quarter-million to tinker with. You'll be advancing the state of the art and I know you'll have a great time doing it. I would love to join you, it sounds like so much fun.
But what you're doing is also totally unnecessary. Maybe you need this thing for some big television or movie production but in that case you need the reliability of hardware designed to do this. But I have to agree with previous posts that your described intent doesn't justify the huge expense.....you could get by with lower cost solutions.
If you really are doing this just to advance the state of the art then the choice is simple: GO LINUX. You won't find Microsoft interested in helping you tweak NT to your specific needs on this project unless they get a major piece of the action. Nothing beats Linux for advancing the state of the art....and having fun while you're doing it.
Re:Have you considered an s390? (Score:1)
I couldn't have said it better.
Re:Try EMC :) (Score:2)
I understand DG carries a bit of a reputation with it, but CLARiiON arrays are generally considered top of the line for the market they are in. CLARiiON also gives some features EMC boxes just don't. They may not be as high end (8 GB cache might not be available), but they compete very well in their market. If CLARiiON has a fault, it's more with their PR departments than their product itself. [This URL shows how NEC recently broke some record for best performance on a specified benchmark using CLARiiON boxes for storage: Press release [clariion.com].]
Also, the previous AC who said CLARiiON was porting their drivers and management software to Linux, pending a port of a third party tool; I wasn't aware such an operation was officially underway. A search of CLARiiON's web page shows no reference to the word "Linux" anywhere.
-SDog
Re:Linux and big multi-processor machines (Score:1)
Message on our company Intranet:
"You have a sticker in your private area"
Re:Maybe a Network Appliance Netfiler instead? (Score:1)
Also, I'm currently in the process of evaluating the NetApp for use with Linux. So far, they are by far the best solution. And their implementation of NFS does indeed scale. You can use multiple clustered NetApps and even do RAID accross them.
A.
--
Adam Sherman
my 2 cents (Score:1)
My views on this is that the more discussion on what Linux CAN NOT DO is very important. All it will do is open the door for more support, more efficiency and more enjoyment for those that choose to use it. Maybe some pages should/could/or already are setup to note -
-supported/unsupported hardwares.
-what Linux can/can't do and it's efficiency on that note.
I'm sure a central area like this could be of nothing but benefit for the ever so funky LinuxOS
and the people that develop and use it.
if there is already something like this, slap me if you please and then leave a URL.
cheers
ORiON
Re:Linux and big multi-processor machines (Score:1)
No, Grand High Poobah, maybe, or perhaps a Kernel Overlord, but not a 'mere' hacker
...I think my tongue is stuck in my cheek....
Is Linux Ready for this? (Score:1)
While I think it would be really cool if you could pull it off, the potential of disaster here is huge. Your primary objective here should be to get this system to work, and work well. Look carefully at the technologies you have available. If NT or Sun is the best way to go then, swallow your pride, and go do it that way. Give Linux another few years to evolve and be 100% ready for this kind of task. By then they will want a new system anyway.
But if Linux is not ready for the challenge, and you use it anyway, you will make the OS _AND_ the project look bad, and many of the not-so-technical tech people will wary to try either again very soon. And, needles to say, this would give Microsoft more ammunition.
Sorry I can't give you much than this.
FYI: I use Linux as my primary (only) OS at home and (before I graduated) was the administrator of my High School's Linux web server. I love the OS but we need to be carefull not to say it can do things it can't.
Cost cutting? (Score:3)
Re:why (Score:1)
Re:Go distributed (Score:2)
On the other hand, there is a bunch of research about how to build video servers, and some of it does use clusters of relatively cheap boxes. Forgive me for not having a URL to it.
Re:WARNING (Score:1)
Re:Try EMC :) (Score:1)
So what are the significant differences? EMC arrays scale to more disk capacity, more cache, more ports. EMC arrays support load balancing between concurrently-active array controllers better, though still not as well as I believe they could/should. (BTW, be very careful before you challenge that claim, since I wrote some of the original software to do this on Clariion, and within the organization that did so on EMC) EMC support is very highly regarded, to the extent that we're wary of even someone as good as HP "diluting our trademark". Think about that. Of course, EMC equipment also costs a bundle. As for performance, that's mostly a trackless swamp that I won't get into except to note that you can't talk performance without talking scale. In my experience, talking now about both storage systems and hosts, optimal low- or mid-range performance and high-range scalability are rarely found in combination. More often, some companies focus on winning the benchmark contests at the low or middle ranges - and they succeed - but don't even have products in the high range. That's not a criticism; it's a perfectly good and moral business strategy. In the end, the only valid benchmark is your workload.
I'm not going to make any recommendations for either EMC, Clariion, or any of our numerous competitors in either the disk-array or NAS spaces, though. Just offering some food for thought.
Re:Sick of this kind of crap.. (Score:1)
Aside from conspiracy theories, why do people post pure flamebait like this? D.A, do you actually feel this strongly that Linux sucks? Why? And, if you do, why do you bother to come here and post comments like this? Why do you hate people that don't know as much as you (assuming you have any idea what you're talking about)? The unlearned are to be educated, not hated.
Also, an attitude D.A expresses is:
"I'd rather him run NT and not risk more people seeing how much "linux sucks"..."
Why? If Linux sucks, why do you not want people to see that, and share your attitude? Any attempt to sheild an audience from the reality of a product is really missing a major point of many developers: to give people choice over their software. Choice is not really choice, if you do not have real data to base your decision on! If Linux sucks, let them see it. If not, let them see it. There is no reason not to.
One more thing: you don't have to post flamebait. printf("rtfm: %s\n", location_of_document);
Thomas
Re:Sick of this kind of crap.. (Score:1)
>is also not a good idea to promote Linux for
>something it is not good at.
True... I hadn't construed his comment that way. I suppose consumers might see that Linux is not good for streaming video, and decide that it must be good for nothing, and that would be bad...
Interesting issue. I thought (and I bet he did too!) that it was just flamebait!
feh.
Thomas
Re:Go distributed (Score:1)
... Or SGI (Score:1)
The O200 starts at $10-12K, so you could use FailSafe to have redundant services available.
Re:Maybe a Network Appliance Netfiler instead? (Score:1)
Re:Sick of this kind of crap.. (Score:1)
Hmm, using that logic, everyone should still be buying just IBM solutions. IBM is still a much larger company and spends much more than Microsoft. :)
Everything is relative and specific to each case. For us, I looked into Exchange to support our 25,000 user mailboxes. The client access licenses alone -- even with our huge educational microsoft select pricing -- is over $100,000. The hardware required to run an Exchange server that size is out of this world. Hell, you're not even supposed to have 25,000 users in a single NT domain no matter how big the box is.
Contrast that with a simple unix/linux/*ix box running sendmail for peanuts.
All of us should try and keep bigotry and prejudices behind us. There is no ideal solution that will fit every need.
If you are able to make it to So Cal (Score:1)
www.siggraph.org to find out more.
I was there last year at the show in orlando. There were over 12 vendors demonstrating media serving technologies.
Perhaps RAID5 is not the way to go? (Score:2)
I'd bet that with the dedicated hardware, a netapp with a GigE interface would spool data to the net as fast or faster than a {Sun|NT|Linux} box would, regardless of caching. The dedicated hardware, and an OS that fits on a floppy would just plain do a better job.
However, you are still running RAID[45]. These have adequate read performance, but cannot exceed the speed of a single mirror. Since the server is to be primarily read dominated, RAID0+1 (striping and mirroring) may be a better choice. It still provides data integrity, and much better speed in this application. Although you need more disks to achieve a certain size volume, the controller hardware is cheaper, and generally more reliable.
With any decent stripe/mirror implementation, each mirror can satisfy read requests independently. Having 4 mirrors is like having 4 independent copies of the data. Need more bandwidth to the files, add another mirror....
Bottom line: If you want RAID4/5, and NFS or CIFS, the netapp is they way to go. It will be faster than any general purpose box with comparable equipment. If you want RAID0+1 or any other protocols, go somewhere else.
However, the original note specified GigE as the network connection with ~144Mbit active load. Just about anything can supply that load without a problem. Unless you are planning to up the network bandwidth to 500+mbit, this discussion is headed in the wrong direction.
Eric Brandwine
Re:64bit pci x 5 (Score:1)
Yours for the low, low cost of an internal organ of your choice.
http://www.intel.com/design/s ervers/ac450nx/index.htm [intel.com]
Re:why (Score:1)
Consider that distance education students using this system will pay the same fees but will incur little by way of incrememental costs (especially when it comes to rebroadcasting courses, ad infinitum). You may hate to see your money thrown away on seemingly trivial projects, but remember that a distance student will eventually be subsidizing on-campus ones.
My advice... (Score:1)
First, get a quote from Hitachi Data Systems [hds.com] for their 7700e open systems platform. You will be amazed at it's price, redundancy, and scalability. It supports up to 16GB of cache, and scales to 6.5 TB. I saw a quote for one of these that included support and the management software for about $250K (Not including much disk). It has a large initial footprint, but scales by adding disk cabinets, so it isn't THAT big for the top end of scaling.
Second, NT and Linux are both bad ideas for this kind of large file, massive network connectivity environment. (I love Linux, but 31bits of addressing is very limiting on large servers.)
NT is bad because:
If you find that your hands are bound with using NT, then look for alternative FileSystems. Veritas might be a good place to start looking. I don't know if they have NT FS, but one of their account executives could point you to someone who does.
You could use HP/SGI/SUN/DG. I know that HP is expensive, but it's management tools are nicer than SUN's. SUN has nice/easy hardware, but if you don't have in depth UNIX SA experience, you will have problems making the box admin friendly. I haven't worked with SGI and DG enough to comment, but I know that they would meet the requirements.
Lastly, you might be much better off looking into the client server communication protocols involved and seeing if it might be possible to use many hosts behind a CISCO Director or a BIG-IP box, (load balancing, redundancy through the IP layer).
WARNING (Score:3)
Windows NT currently supports 4 GIGs of RAM.
Linux only supports 1 GIG... but I have heard some people say that it supports 2.
For a high end video server, RAM is just as important as HD space or CPU power. NT may just be the better choice in this situation.
BTW, using NT for this solution won't increase your overall costs as much as you may think.
Often, the upkeep for Linux can be so much more expensive than NT (if you build your machine correctly) that it more than covers that initial costs of the OS.
Re:64bit pci (Score:1)
Costs? (Score:1)
Hmm...
Mong.
* Paul Madley
Re:Go distributed (Score:1)
This sounds like the most likley senario. some quick calculations. at full buss speed (80 Meg/sec) = 640 Megabits/sec. that would serve 533 1.2 megabit/sec video feeds. in reality, lets say that you have the five servers all on there own 100 megabit/sec subnet, that would serve about 450 cliants. also, this would provide fault tolerance. you would have RAID for the storage and 5 servers. Linux would do this easily.
Re:Sick of this kind of crap.. (Score:1)
um, I dont think so. from my expearance, this mechanisim has a sucess rate of about 25% in NT. This is one thing that I love about Linux, I realy can kill a rouge application ( Netscape comes to mind ) without re-booting the machine. There are several times when an NT server just froze cold, no BSOD, no error messages, nothing. now in some cases, it was a hardware problem like bad RAM, but atleast with linux, it gave me some idea of what went bad. with Linux, I have had 2 panics in the past year on about 3 machines. in one case, I accidently disconnected the IDE cable from the disk that held the swap partision, the second, I had a bad root disk on a slackware install....
Re:Linux and big multi-processor machines (Score:2)
If you really want to do this properly... (Score:2)
The chances of getting that sort of help sound reasonable from the scale to which you are looking, and they will also probably give you a nice support package and good response as their pilot.
If you want to put in a system that lasts, go pro, otherwise it'll be you that won't last from spending too much time fixing/tweaking/everything else on the system.
just my 2p worth...
Re:Get one big fancy server (Score:1)
I also think it will be difficult to come up with a simple Redundant Array of PCs (RAP?) solution where any single system can fail and the others will take over seamlessly. However, if you achieve this in a few lines of perl please post the source.
My 2 cents - buy that big fancy server. Pay extra for features like redundant power supplies, fans, and RAID 5. Get a rack mount. Add capacity by adding RAID enclosures.
Re:Go distributed (Score:2)
Re:Sick of this kind of crap.. (Score:2)
linux fibre channel drivers (Score:1)
FCAL Solutions and the PCosaur (Score:2)
sun boxes and Veritas cluster server. Its gawdaful expensive, but when the customer dosent care about $, it goes. If you have time to tweak though... and limited cash.. then PC may be the way, and linux has the reliability. Just remember, a PCI bus and a fast processor alone do not a bandwidth slinging server make, no matter what intel tries to tell you..!
Dedicated Video Server (Score:1)
How about some web sites?
http://www.tvbroadcast.com
http://www.videography.com
http://www.tvtechnology.com
Scott
Why not use Quicktime instead? (Score:1)
.ASF - Uh, yeah right...
.RM - Real may be the standard but it is a server hog and the cost for the server software... Yikes!
.AVI - We're talking streaming here...
For streaming seriously look at the Quicktime format. The server software is only $500 or so and runs like a dream. Everyone can use it and there are plenty of encoding solutions out there to make QT files to stream out.
Our current solution where I work for doing what you propose, is filming everything in DV, encode it in a render farm consisting of 4 G3 computers with BlueICE cards, all fiber channeled to 360GB of drive space.
If you're looking for quality streaming video at a great price, seriously look into getting a couple of G3s. You don't need to spend thousands of dollars of multi-processor NT/Linux solutions when a non-expensive non-MS platform already exists doing exactly what you need.
Right tool for the Job (Score:1)
Why not use Multi-cast video streaming with channels that repeat every 10 minutes or so ( kind of like pay per view with 4 channels that break into half hour blocks ). This way you have a fixed number of connections that scales with the number of titles, not the number of users.
Thus you have clients request a title, and on the next 10 minute block, all users that have requested that title ( or more specifically a 10 minute section of a title ) will be multi-casted together. The clients can be set up with varying sized caches ( based on their free disk space ), so if they need to pause or replay a section, BW doesn't have to be eaten ( and while they are pausing, they are still downloading future portions of the movie along with anyone else. ) I'm not sure how well a mainstream player such as RealPlayer, media player or quicktime would work ( not familiar enough with them ), but if this is an educational project, a custom built system shouldn't be too hard to tackle. ( though a mainstream ( no pun intended ) product would be easier to maintain ).
There are many variations that could be played with. The big problem that I see with brute force is that if you have 5 guys in a hall downloading the same 1.2Meg/s file, the BW is going to be too saturated for the rest of the hall downloading porn ( in jest ). Perhaps there is a switching configuration which would allow this media to be completely independant; I'm not experienced enough in that area, but it definately doesn't work on a generic shared network.
The cool thing about this sort of system is that you can easily divide an conquer. If your 150 connects are all independant titles, you can cluster them ( one, two or three titles per server ). Here you could easily take advantage of cheap, stable Linux and x86 and build 50 or so $3-400 boxes for $15,000 - $20,000.
You could stick with simple UDMA IDE drives ( $150 a piece ). You could even load directly off a DVD player ( $90 a piece ). Down time would be minimized and localized. ( tape restore to a new drive if one fails, and you only lose the title(s) local to that disk. Or not at all if using DVD ).
Titles that are more popular could have enough RAM installed to cache most of the system ( aside from the mentioned 1-2G limit on Linux ).
Because broadcasts are in blocks, they can be loaded as a continguous chunks ( 1.2Mbps * 600s = 72MBytes per feed ), and can easily be cached in memory. This removes the need for _any_ high performance disk hardware ( including SCSI ), since it would only take 9 - 36 seconds to load a chunk into memory.
A 2 hour video clip would only have 12 segments, and thus your 150 titles would require a maximum of 1,508 channels. If properly switched and distributed, each machine would only need 12 - 36 channels or 14.4Mbps - 43.2Mbps, which could be handled by a 100BaseT NIC rather comfortably. Thus all generic parts seem to suffice.
The only penalty I can see is that a user would have to wait a maximum of 9.9 minutes before their feed begins. I think this is a fair trade-off, given it's one step closer to interactive TV.
Re:64bit pci (Score:1)
12 PCI slots, 4 of which are 64bit, etc,etc.
Re:several things (Score:1)
Re:Sick of this kind of crap.. (Score:1)
Poorly documented? Hrm, get a copy of the MSDN cds, go out and buy a book on the Win32APIs or goto msdn.microsoft.com and read, or ask questions.
It's the most complete API I've used.
Re:Sick of this kind of crap.. (Score:1)
An NT solution *could* and most likely will get into 6 figures with your arrangement, but this is oppsed to the 7 figures of traditional Unix solutions. If you get it from someone like IBM you can get 99.99% type assurance etc etc.
I don't think that everyone should run around (and they have been for years) yelling Linux is better! Linux is faster! Linux is better designed! Linux is based on 30years of good computing (unlike NT which didn't learn from Unix) - oh but Linux isn't Unix - it's better.
NT is good for many solutions, as is Unix. Linux however is still immature, and I don't like the programming model. Too unset.
And I've lost more than a *wee* bit of data with Linux - which is why I wouldn't use Linux for a data or file server. Try letting linux play with the harddisk and then turning the machine off and on a couple of times. Eventually, Linux won't be able to boot, and you'll have to do a manual fsck, which sometimes doesn't work.
Windows NT/2000 doesn't mind it at all.
And Microsoft has BILLIONS, but they don't spend ALL of that on Windows. Most of it goes into theory and experiemental research. I'm glad someone is spending that kind of money on that sort of stuff these days.
Microsoft has billions. Microsoft spends all those billions developing Windows. That doesn't make sense.
As for that problem with the PPC, bad luck, but it looks like a problem with Pocket Streets rather than WinCE. Software crashes usually aren't OS related. Windows terminates applications that start to do bad things.
Ok, so i wasn't that brief
Re: terraserver.microsoft.com (Score:1)
100baseT isn't really 100mbit (Score:1)