Open Source Router on Par With Cisco, Users Say

Jane Walker writes "On a mission to avoid paying top dollar for Cisco routers, two users say Vyatta's Open Flexible Router is a viable alternative to the proprietary norm. Find out about the pluses and minor hassles involved in deploying this alternative." This probably won't surprise the users of (much lower end) networking gear like the famously hackable Linksys WRT54G, which — like a number of internally similar routers — can be reconfigured with one of several open-source firmwares to do things impossible with the hardware as delivered.

Difference between hardware and software....

It is not surprising that low-end software routers can offer most things a proper Cisco router can. However when you need hgher speeds, a software router can not cut it. It is then when hardware routers show their strenght. A 100Mbps line usually does not require a hardware router. A 10Gbps line does.

Re:Difference between hardware and software....

Bleah. This is tripe. Most Cisco routers have cheap, slow Intel processors in them.

Until you get up into the gigabit speeds, regular PC hardware is just as good or better. The only thing you have to watch for in the multi-hundred-megabit routing loads is that you don't have a lot of access control lists - which is also an issue you will run into with any router you might choose. Spending some time sizing the buffers and other kernel parameters is also important, because a stock Linux kernel is not set up to be a network core router.

I've got over 2,000 L2TP connections going into a single 2.4Ghz Intel box running Linux. Performance is significantly better than the Cisco 7204 that it replaced, and it's a lot cheaper and more flexible to support.

Now, in the multi-gigabit routing tasks, do yourself a favour and get a L7 switch with custom ASICs. Extreme, Foundry and others will be happy to sell you one. Cisco's stuff is crap, right up until you get their million dollar badasses which they bought from another party (go figure).

Re:Difference between hardware and software....

Certainly have heard of CEF. And have witnessed first-hand how bad it usually works in a big Internet environment.

Let's see...

--

IP CEF with switching (Table Version 271518), flags=0x0
    1030 routes, 0 reresolve, 0 unresolved (0 old, 0 new), peak 3
    1033 leaves, 27 nodes, 152040 bytes, 269271 inserts, 268238 invalidations
    0 load sharing elements, 0 bytes, 0 references
    universal per-destination load sharing algorithm, id 26B36E8A
    2(0) CEF resets, 1425 revisions of existing leaves
    Resolution Timer: Exponential (currently 1s, peak 1s)
    2250 in-place/0 aborted modifications
    refcounts: 9206 leaf, 7168 node

Adjacency Table has 888 adjacencies
    2 incomplete adjacencies

--

It does speed things greatly. Load on the 2.4Ghz Linux box that replaced it is 0.07 right now, with 1800 L2TP connections.

Re:Difference between hardware and software....

Ummm....no. In anything more complicated than what a switch can do, you are using software to process packets.

Yes, Cisco (and others) have routers that use ASICs to handle immediate in/out "routing" in hardware, but as soon as you start putting any kind of ACL, any kind of port/IP translation, or anything else that requires any intelligence on the router, you bring in software, and all of the processing overhead that goes with it.

So....if you are going to do anything *useful* with a router would you rather have a 50-200MHz Cisco box running a bloated IOS (do you *really* use X.25, for example???), or a server-class x86 motherboard running a 1GHz processor with a kernel optimized for routing and software optimized for the protocols you actually use?

We use http://www.imagestream.com/ [imagestream.com]ImageStream Linux-based routers where I work, and they absolutely run circles around the 2600, 3000, and as5000 -series routers that we have. Their support is absolutely phenomenal. When we have a problem with an ImageStream router, we frequently talk with their programmer, and he works with us until we have a patch installed on the box that fixes the problem. If there's a software bug in your Cisco router, it's "yeah, that will be fixed in the next IOS release"...which unless you paid out the <bodily orifice of your choice> for SmartNet you have to *buy*, even though their product was broken when you bought it.

You can use overpriced Cisco iron if you want; I'll stick with the Linux-based routers, thanks.

Link to Vyatta

Perhaps a link to the actual product would be in order?
Vyatta Open Flexible Router [vyatta.com]

Avoiding "License Transfer" Fees

Isn't this a way to avoid paying for the licensed software on Cisco equipment when it's sold second hand? (Not trolling or anything, I think it's ridiculous for Cisco to demand payment for software that's already been paid for once.)

if it is only "Standard PC Hardware"

It isn't comparable with Cisco.

But then again for SMB - you don't need 100 MBit routing - many of your internal clients are slamming into your sub 10 Mbit internet connection anyway (that is probably further BW limited by the cable/phone company). Now for true enterprise - you really do need switching/routing at the ASIC level - real switching fabrics (not a glorified PCI bus) in the hardware etc. to handle the multiple GBit links, multiple OC12/OC48 connections to the world, etc.

This is where Cisco shines and I don't see "software only solutions" coming anywhere close

Why change from DSL? Reliability...

Strange - why would you expect companies to step down from decent DSL speeds to T1 rates.

When you need reliability, you have to give up on DSL/cable, because no DSL or cable provider is going to give you service guarantees. If a DSL/cable line doesn't provide it's advertised 2Mb/s download throughput, that's too bad; you might be able to negotiate your bill down. And if it goes down, it's going to be you reporting it to your ISP, not the other way around...

But a T1 circuit (generally) has both through throughput and uptime guarantees written into the contract. And automated monitoring of its performance, and fast notification that something's wrong, 24 hours a day. I've had DSL circuits be out for days; the longest a T1 circuit was down was 8 hours, and there were severe financial penalties proscribed for that event.

That's not to say a T1 circuit is perfect; we use a bonded pair of them to feed one site. One went down, due to an incident with a trencher. Verizon promptly fixed it... by moving the circuit to another pair that tested good in the cable. Guess which pair got used... If you guessed the pair that the second circuit lived on, you'd be right, and it went down. This went on for a day, alternating which circuit was up and down, until one of our people met the Verizon tech at the repair site. "You do know that there are TWO T1 circuits here, don't you?" "Oooops..."

Re:if it is only "Standard PC Hardware"

If I had one dollar for every time I give this answer, I'd be frelling rich:

99% of businesses use sub 10Mb connection to the Internet and yet they are told the Cisco is the only way to connect them professionally. Moreover, the sub-$10k Cisco gear is a crap when it comes to performace, on par with good PCIe PC running on multiple Gbit eth interfaces.

That about sums it up.

Robert

Re:if it is only "Standard PC Hardware"

Gosh, someone who knows what they're talking about ;)

If your internet link is DSL, you do not need a real router :)

I should point out that this topic comes up every couple of years on NANOG, ummmmmm... here's a reasonable selection from the last decade [google.com]. These people have forgotten more about routing than most of us here will ever know. And until generic PCs come with multi-gig backplanes, it ain't happening anywhere except the low end. And at the low end, you're better off either leaving it to your ISP or using a few whitebox "desktop" switches/routers. They're cheap, cheerful, work, and you don't need to know the difference between "sh ip bgp run" and "sh bgp ip run"...

I LOVE DD-WRT

I rent a housemate cable internet, which we had terrible problems with before. The problem is a bad cable causing a load of bad packets to 'clog' the router. It is the only cable long enough I have though, but the DD-WRT firmware worked a treat. It does allow some cool features, such as increasing the number of IP connections from 512 (the default) to 4,096 which is ideal for p2p. You can also boost wireless power from the 28mW default to 250mW+. Anyway, my problem with it clogging up was solved by setting up a cron job within the router so that it reboots at 5am each day. Not ideal, but the solution works until he gets off his ass and finaly buys a wireless card.

ASICs

This seems to be an entirely software router that just runs on a standard x86 machine.

Isn't half the point of buying a dedicated-hardware router that you get ASICs and whatnot that do the job faster than software?

English, please!

Can we have that article again, this time in English, please?

TWO consultants agree?

Two unknown consultants decide that Cisco sucks?

If we were to judge solutions based solely on the word of two-or-more IT consultants, we would have "enterprise solutions" with MS-ACCESS backends, with a "robust" monthly backup to .TXT files on floppy.

Seriously, the holes in this article are big enough to park a datacenter full of Cisco hardware in.

* As long as we're not switching half the U.S.

Summary: Works great, supposed problem sounds like it was a driver issue more than an application issue.

Reads like a well-placed article-vertisement.

The "as long as we're not switching half the US" comment are the one's I grow tired of. It's a well-wrapped insult.

I'm not saying Linux is the best tool for routing half the nation, but the comment points out some things that do prevent more linux adoption.

1. "free" is not as good as something I paid for
2. Don't fsck with the status quo.

I admin a company 100% cisco routers/firewalls and I know for a fact Linux can do what gets done.

I'm not going to tell the boss to "just" switch or evangelize too much because of the social/economic implications of doing so may impact my future. I like my employer, they like me, so when we need another router, it's a cisco. I am personally disappointed by this, but I think it explains why innovation takes -so- long to come to the data center. (at least in the U.S.)

Let's not forget that cisco can fire most of their software devs and use a linux-based router project if it ever got close to competing with some Cisco products. Does that qualify as innovation? I'd say no. It's not cheaper or better.

Advertorial

It is great that someone is out there tyring to put some preassure on Cisco however this company is not it. I think the average Linux installation with NAT is a bigger threat that this project.

PC hardware is a joke, slow backplanes, limitation on how many interfaces you can plug in. On the techspecs the number of interfaces types they use is well very very limited. Then reliability of PCs a joke compared to a Cisco box.

Where is this product used?
- Is this a bloated replacement for the US$20 taiwan PPPoE router you can buy? The taiwanese will beat it on TCO hands down on power consumption.
- Is this for the edge of the network to service downstream customers? Why part away from the thousands of installations which live, have predictable and very proven track record of something like a Cisco 7200VXR..

The article (Advertorial) is nicely skewed as making Cisco seem expensive. Go on ebay and look for Cisco routers with FE ports, you can find them for a few hundred dollars. Or try to compare this with 3550 which will provide 24 ports with Layer 3 functionality for way below the US$2000.

I am tired of Cisco killing products off when they feel like it.
They could go into the market of breathing new life into a product that is being cancelled by other vendors. Firebox II anyone?

Vyatta still need a strategy. period.

No huge suprise

In or around 1999 I had a 1000 device network routing through a 133Mhz PC running Linux. The 133Mhz system practically thought is was sittle idle as it shuffled packets between three 100 megabit networks.

I'm not suprised at all that these Open Source solutions are on par with Cisco for many users. My only real concern would be support. At least back then (I have not dealt with them recently), Cisco had great support and would "own" network problem resolution in a way that made it worth paying their price.

Re:No huge suprise

There is nothing like calling Cisco TAC at 04:00. You get an Aussie TAC engineer that knows 1) you're network is seriously fscked up or you wouldn't be calling him at 04:00, 2) you've already removed what little hair you have from your head and your scalp is bleeding. The nightshift engineer is highly experienced in working under said conditions and is more than capable or resolving the problem. I've been in that position twice in the last month. All I can say is I want to move to Austrailia, mate.

I wish the SmartNet prices were a little more reasonable. They should cut the prices dramatically for the lower-end 8x5x4-day replacement support so that more people can afford it. This would be a solid recurring business for Cisco whereas only a small percentage of Cisco customers bother buying support nowadays.

in other news

a small truck can replace a semi truck.... if you are moving small amounts of items.

Support, Support, Support

Ok, I haven't looked at the performance numbers, but as a network administrator of a medium sized corporate network I could care less. Whether it be Cisco, Juniper, Nortel or 3Com the difference is in the support. When my wan interface or network interface dies at 2am I don't think anyone from the OSS community is going to have a parts depot within 4 hours to fix the problem. I also don't see 24x7 tech support phone numbers manned by volunteers anytime soon. Vendors don't make the money on the hardware, they make it on services and support. I love OSS, but Linux and OSS are not the magic pill for everything.

Huh, Samba file sharing?

"The feature set was comparable to your standard Cisco router," Knox said. "They were offering translating, gateway capability, Samba file sharing, VLAN trunking to 11q ... it really looked like a corporate-level router," he said.

Since when do "corporate-level routers" offer samba file sharing? This seems like the LAST thing I would ever want to put on a router. The only thing I could possibly see Samba being useful for is downloading log/config files. But on a router that is kinda scary, SCP seems much more secure and just as useful.

Open source routing is definitely an option now though. Over 3 years ago the web hosting company I worked for swithced out their Cisco routers that couldn't handle the slighest DDoS attack for a couple AMD based Linux boxes that could easily handle wirespeed DDoS attacks with ease. Not to mention they were a fraction of the cost.

This is a good thing?

It was my understanding that Cisco was the best network hardware manufacturer out there, but after having 3 linksys routers die on me in the last 5 years (one a week out of warranty coverage). I won't be going anywhere near linksys again.

My buddy just lost one as well.

So... a Cisco router as good as a Cisco router?

Hate to break it to ya, but Linksys is owned by Cisco.

Since when ... ?

Since when do we listen to "users?"

You're buying the hardware

I use Debian at home for a general purpose router and firewall and it is very flexible. There have been times when I've been tempted to deploy it as a small/medium business router in lieu of cisco but it's not just about the software, it's about the hardware as well. For a reliable system you need reliable parts... which are more expensive... preferable a cpu with a low thermal dissipation but still fast enough to handle the load, which is going to cost you money and either a RAID system or (ideally) a flash based storage system, which is going to cost money. You can build a system that will beat Cisco's cost/feature set easily. Building a system that can compete on cost/mtbf ... not so easy... and generally just not worth the effort. The article referenced a "still servicable pc" ... which roughly translate into "a machine that we picked up from behind the receptionists desk and cleaned all the dust bunnies out of.... *shudders*

Where should Vyatta go? What direction?

I think most points have been covered now.

I think the way we can help Vyatta is by giving them suggestions as to where they should go in the market.

What direction should they take in your opinion?

Best WRT54G/s Firmware I've found

I actually just flashed my GS v1 to DD-WRT yesterday. I;ve tried many other firmwares...Talis/Freeman, Alchemy, HyperWRT. They all had the same problem when in client mode I would loose the wireless connection to the AP and would have to re-boot to get it to reconnect. DD-WRT is the only one that works without a hitch, although I noticed HyperWRT had faster thruput (when it worked). It also doesn't get clogged up like everyother firmware when using Bittorrent or P2p

Looks Cute.

This package looks great, and I've got a couple of things to comment on that have been being said. Firstly, everybody seems to be talking about this like its some kind of CATOS/IOS ala Cisco replacement. They don't seem to be billing it like that at all, as far as I can tell. Of course, the dedicated "meant to do that" hardware solution from Cisco is going to be legions better than any software you can stuff on a PC.

Lets take the discussion where it probably should have gone, to the guys contracted to set up a network for a local law firm office, with 20 employees. Maybe a veterenerian's office that probably doesn't do that much business (dollar wise). It's the folks like this that might still need what a Cisco has to offer in features, but doesn't need what Cisco offers in capacity, and definatly not what they offer in cost. Considering how (it looks to be) well documented, I don't think that the contractor mentioned will have any problems supporting it. It's our bread and butter.

What DSL modem to use?

When using a PC as a router, what DSL modems do people use?
I am trying to find some ADSL2+ modems to connect to our Cisco routers.
(in the past we have used Cisco ADSL WIC, but it has become clear that a consumer-grade Alcatel modem outperforms those, and even worse: there is NO ADSL2+ WIC...)

The modems have to support PPPoA and provide a transparent "bridge mode" where incoming traffic is delivered on the ethernet port with the Internet IP address as destination. This would be the same mode you would want for a PC-based router between LAN and Internet via DSL.

The problem is that it becomes difficult to find a "dumb" modem like that, especially with a reasonable build quality.
Everyone has NAT routers with 4-port switch, Wireless access point, VOIP gateway, printer port and what not, but I just need a dumb modem with no frills that increase the failure chance or that interfere with transparent operation.

For example, the more recent Alcatel/Thomson models appear to offer a transparent mode, but it has proven to be unreliable. I think the NAT engine is in the path in a 1:1 mapping mode. After some days of operation it appears to drop packets of longstanding connections while still servicing new connections.
We never had those problems with our old Alcatel 510, but that is not ADSL2+.

Any idea where to go for a reliable, transparent, ADSL2+ MODEM??

Smoothwall?

We maxed out on our small soho router at a small company I worked at. I then turned to smoothwall (www.smoothwall.org) and loaded
it on a 700 Mhz pentium 3 system. Worked wonders. Lots of features and addons/extensions from the community. They also have a commercial version with a gang load of features http://www.smoothwall.net/products/corporatefirewa ll4/?featurecomparison [smoothwall.net]. I'm surprised it wasn't mentioned on this topic before. I'm not sure how vyatta compares though.

Let's get a few things straight here...

Vyatta is not just open-source routing software, they are a company that supports said software. Yes, if you have a support contract you can call them at 4 am if your network breaks.

          The term "open-source router" is extremely vague. A router is a physical device that forwards packets at layer 3. In the case of the Vyatta OFR (as wel as Zebra, OpenBGPd, etc), the routing software (roughly, the RIB) is Vyatta, the forwarding software (roughly, the FIB) is the Linux kernel, and the hardware is a PC. In addition to various silicon-based solutions to speed up packet forwarding, software projects such as the Click! modular router exist that replace the routing code in a commonly available kernel (Linux, BSD, etc) that increase packet forwarding performance exponentially. The fact is that the commodity packet forwarding code in off-the-shelf OSs (OSS or commercial) hasn't evolved much in a long time, because it hasn't needed to.

          Procket (founded by Tony Li, bought by Cisco for the engineering team) had also developed software forwarding based products that had similar performance without using custom forwarding hardware (1+ mpps on x86). Too bad they will never see the light of day. Of course, their hardware was also capable of 12bpps (yes, billion) in 2003....

Hardware support lacking

Too bad they only support Sangoma serial cards.

That reminds me...

... of an almost perverse little daydream that I had some time ago. I thought that it would be fun if someone were to made T1, T3, and other interfaces that connected via... USB. A USB connection has enough bandwidth (at least on paper) to run a T3 with ease, and you could pop 32 (or more) USB 2 ports in a machine very easily. And for the lesser-bandwidth interfaces, you could run them off of a USB hub.

    So, imagine a single machine with 30, 50, or 60 network interfaces coming out of it, all sprouting USB cables. What a mess.

    The more that I think about it, that's not so much of a daydream as a nightmare.

steve

Cisco Salesperson Here

Hello, in my experience people don't really buy Cisco routers because they just want to shift packets. The days of edge routers dumbly sending packets out to the internet for buildings of 100-500 users are over.

Instead they want built-in VPN concentration, firewalling, the ability to automatically dial-out their VoIP calls if the WAN goes down, flexible WAN links, content caching &c &c &c.

While a software router is a great solution for switching packets, it quickly stops scaling in a single box when you want to add the extra features offered by Cisco's 2800 and 3800 series, which this product is touted to compete with.

Here's an example: A big bank wants to install routers in one of its branch offices. Naturally, it'll need a firewall. Then it wants to push content to each bank for digital signage, such as videos to play on their flatscreen TV's. It also wants Quality of Service and redundant PSTN links for their IPT solution and local call processing, just in case the WAN link fails. The branch has only, say, 25-50 users, but the dedicated single-box hardware is more effective for them because they are able to buy the unit and incrementally upgrade it, adding these features with no performance loss and have it all supported from a single phone call... No need to handle different vendors. As around 80% of the cost is running solutions as oppossed to procuring them, this is a good deal.

This is why software routers are addressing the wrong market. How many businesses now intend to just shove packets out to the 'net?

Cisco switch performance review

http://blogs.sun.com/sbullen/entry/who_buys_cisco_ 3750_s [sun.com]

tripe.....

I would NEVER take our main CCisco driven backbone down and replace it with Open Source. Again, support is the issue. If there's an issue with the router, Cisco probably definitely knows about it....even the obscure. Sometimes they don't, but they still help you anyway. Can support contracts be had for a Open Source router?? Will the support actualy help me or tell me something like L1nux R0x0rs, RTFM y0u n00b?

Re:Huh?

Was working on it. Trying to (gasp) RTFA first...

Re:do NOT modify the hardware - or it may cost you

Huh? What?

It's my hardware. If I buy a Cisco router via eBay, you're telling me I'm not allowed to put Linux on it if I can figure out how?

Re:do NOT modify the hardware - or it may cost you

The Linksys WRT54G firmware is released under the GNU GPL... That's like an invitation to modify it.

Re:do NOT modify the hardware - or it may cost you

Excuse me sir, can you please put down that glass (FUD) pipe.

Thanks!

Stupid lawyers

So you're claiming to be a lawyer, but I have a few questions.
Why post as an anonymous coward?
Are you violating your client confidentiality with the parent post?
If I wipe their firmware (which I have a license to use) how am I violating their copyright?

Re:do NOT modify the hardware - or it may cost you

Your understanding of technology is obviously zilch, zippo, nada, nothing. And that leaves me with the feeling that your understanding of law is also generally diminished. And I presume you've never changed any software on the PC you own?

Re:do NOT modify the hardware - or it may cost you

You are kind of nuts.
Many people, thousands of them in fact have bought LinkSys and other routers and have modified them with new more functional software.
As far as I know the DMCA has been used only once to "protect" hardware from modification. It never went to court and the company pretty much went out of business.
The modification of purchased hardware is protected under the first sale doctorin. The same laws that allow you to buy a car and then sell of the parts one buy one.
The DMCA would only come into play if some form of encryption was broken. Just deleting the firmware on a system and replacing it with new firm ware would not be covered.

If this isn't true then why hasn't Microsoft shut down all the sites offering Linux for the XBox?

Re:Speaking as a fulltime Free Software zealot

Wow- switches for high speed stuff? Jesus what networks do you work with? Where is my OSPF, EIGRP, and BGP on Linux? Where is my VRRP, HSRP and GLBP? Where are the DS3 and OC3 interfaces? Linux works fine for smaller isntallations. If all you are trying to do is connect your office to the Internet then we're not even on the same page. If you work at an ISP, large corporation, or otherwise handle core routing requirements you would never even consider using Linux.

-sirket