Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Hardware

Supercomputers - Does the Cabling Matter? 95

Posted by Cliff from the behold-the-rat's-nest dept.
papaia asks: "Having watched, for a while, the development in the area of high-density server hardware solutions (i.e. blade servers), like IBM's 'top gun', and their increased presence in Data Centers, I have been wondering if anybody has had any experience (thus comments) in regards to how important - in such highly priced solutions - is (or could be) the [always neglected] cabling, connecting the servers. One such comment caught my attention, in this regard. Slashdot, how important is the server cabling infrastructure in your Data Centers, and how do you resolve the cable management aspect of it?"
This discussion has been archived. No new comments can be posted.

Supercomputers - Does the Cabling Matter? More

Supercomputers - Does the Cabling Matter?

Comments Filter:
  • any experience (thus comments) in regards to how important - in such highly priced solutions - is (or could be) the [always neglected] cabling

    im sure you could fit more parentheses and brackets in there! .....you werent even really trying
    • probably some lisp/scheme side-effects.

      • Re:sentence (flow) (Score:1, Informative)

        by Anonymous Coward
        paren characters in lisp/scheme don't indicate parenthetical remarks, they are simply syntax for function calls and lists.
        print("hello"); in C is (print "hello") in lisp. The name of the function moves inside the left paren when compared to C, that's all.

        A Lisp program is like a c program with lots of function calls within function calls:

        funca( funcb( 1,y ), funcc( 3 ) ); in C would be (funca (funcb 1 y) (funcc 3)) in Lisp. Yes, they are different. But one is not particularly easier or harder than the
    • Any sentence where you need to break out Order of Operations, right? God, the Internet has destroyed our communication skills. Kinda ironical....
    • I - being a non-native (not born in) [american-]english speaker - don't [really] know.

  • Always neglected??? Speak for yourself... (Score:5, Informative)

    by Anonymous Coward on Sunday January 02, 2005 @03:59PM (#11239433)
    When you're running a large linux cluster, it is absolutely vital to plan your cabling and get it right. It just is, for access, fault troubleshooting, avoiding restricting airflow. Get cable tray, lay it to each rack site (you have planned out banks of rack sites and planned the airflow to your air con, right? - and please, tell me you have air con???).

    Allow for larger racks so there's room for cables and PDUs. Really.

    Keep things neat with a raised floor, cable tray underneath. A second tray overhead can be handy, but can interfere with moving equipment, so don't fit it unless you'll need it.

    Be aware high speed cables, particularly fibre, have a minimum bend radius - go beyond it, and you have fucked up the cable. Also, be aware of sag due to weight of cables - compress the cable too much with the weight of the bundle against cable ties, and you can damage it.

    Crosstalk is seldom an issue these days, but be wary of laying power and network too close - even if there is negligible interference, you're safer if they're separated by a decent amount.

    Patch panels are useful - use them. Run cables to patch panels and patch panels ->switches, don't go machine -> switch directly (unless you're doing really high-end stuff (for 1GigE copper or fibre, patch panels are fine).

  • Reliability (Score:2, Insightful)

    by Leroy_Brown242 ( 683141 )
    I think that it'd be fairly important. Even if only for reliability. Because speed and reliability are equally important.

  • Cray (Score:2, Interesting)

    by Tersevs ( 168108 )
    This isnt really an answer - more a bit of trivia:
    The ol' Cray-X supercomputer where round (their cabinets where placed in a circle) so that the length of the cabling could be kept down. Back then the synchronisation between pulses in different cables where a problem. And there really where a *snakepit* of cables between the cabinets.

    • Re:Cray (Score:2)

      by booch ( 4157 )
      Actually, I think they were round so that the cables would all be the same length, not so they'd be shorter. Having the cables of different lengths would mean that the latency in different paths would be different, potentially causing synchronization problems. If they're all the same length, all the data arrives at the same time.
    • Yes, but do you know why the Cray-1 has a clear case? So you can see more Cray! (hope somebody will chime in on this one)

      Oh, also, consistent length cables was the reason that I remember hearing, so that signals would not lose synchronization just because of the length of the wire.

  • literally speaking, no (Score:4, Insightful)

    by St. Arbirix ( 218306 ) <matthew...townsend@@@gmail...com> on Sunday January 02, 2005 @04:22PM (#11239539) Homepage Journal
    Literally speaking the cabling won't matter at all. Whether the resistance of your wires is high or low the electrons are going to travel through it at the same rate. What may be a worry is freak occurances of inductance between wires which could possibly mess with your data, but I'm not sure how common that is. The signal is digital so it's going to be either a 1 or a 0 depending on the voltage of the line, and it's usually difficult to make the voltage do something as drastic as go from +5 to 0 or +5 to -5.

    I always have to laugh anytime I go to the store and look at the number of things that say digital nowadays. Wal-Mart sells digital telephone cable for your computer to connect to the wall as if 5 feet of high quality cabling with gold plated tips is going to make a difference in the 30-odd mile trip to your ISP. A friend of mine boasted his "digital" headphones to me once and I had to beat him down to the opinion that "there must be something better about them, so I'm happy" (this sort of pacifist optimism is the bane of IT in general).

    Best Buy sells Monster digital audio cables at something like $20 for a 5-ft cable. I had to argue with my father trying to convince him that the cheap RCA cables we already had back at home would be perfectly capable of communicated a digital signal the 5 inches between the DVD player and the receiver. I could have ripped two wires from a speaker cable to connect the two devices and would have gotten just as good sound.

    People don't seem to want to realize that digital implies lossless or error-corrected. They don't understand that the "premium quality sound" transmitted between devices can be done using the cheapest electronics equipment available.

    I'd save a fortune on car stereo...

    • You don't want to raise the ire... (Score:4, Interesting)

      by mosel-saar-ruwer ( 732341 ) on Sunday January 02, 2005 @04:50PM (#11239649)

      People don't seem to want to realize that digital implies lossless or error-corrected. They don't understand that the "premium quality sound" transmitted between devices can be done using the cheapest electronics equipment available.

      Digital, maybe, but you don't want to raise the ire of the analog stereophiles: You'll get everything from Stereo cables make a difference [thesafetyvalve.com] to Debunking the Myth of Speaker Cable Resonance [audioholics.com], not to mention forests worth of dead tree sacrifices for Speaker Cable Face Offs [audioreview.com].

      And please, please, please, please: Don't get them started on Solid State -vs- Vacuum Tube...

      • Too late. My last roommate was a car audio guy studying mechanical engineering while I was in electrical engineering. As time went by I started asking him about all the components he was using for his system. I managed to help him on a few things and he pointed out some clever stuff that made sense and I thought kinda cool (like how he'd often take two speaker wires and a third unused wire and braid them so no resonance would occur).

        The one thing* that sort of soured our relationship was a gold-plated powe
        • You are perpetrating another myth. There is a very good reason to use thick cable for 12 volts. As you decrease the voltage, you need to transmit more current to get the same amount of power. An amplifier that draws 10 amps from a 120V outlet will have to draw 100 amps if run from 12V. Thus, the cable needs to have 10x the cross-sectional area. As far as gold plating: it prevents corrosion and has low electrical resistance, which is important at those amperages.
          • It still seems like overkill. The cross-sectional area increases exponentially with diameter and yet the wire is *much* larger than the speaker wires leading out of the amp. I know amps get pretty hot when they're running but the need for so power implies a *lot* of energy turning into heat. I've seen in-car coffee makers with wires just like you'd see in your home, not 1/4" monsters.
            • The gold plating isn't so much for corrosion resistance or for the amazing electrical conductive qualities - it is because gold makes the best interface between two electrical conduits. When the two wires or connectors come together - gold does this best. As far as the actual wire goes for moving the electricity around silver is better than gold, once the connection has been made - IIRC satellites use gold connectors, but silver wires (since money is no object, and failure isn't an option.)
            • Many car audio enthusiasts have amplifiers that require kilowatts of power. That means hundreds of amps. You need something about as thick as welding cable for running hundreds of amps. Look at some automotive jumper cables sometime. Even the cheap ones are about a 1/4 inch thick.
            • "... The cross-sectional area increases exponentially with diameter ..."

              The cross-sectional area increases in proportion to the square of the diameter. That's a _polynomial_ function, not an exponential one.
      • I decided to look up one of the cables feature in your last link and eventually found that I would have to pay $36.22 for 0.5 meter of Cobalt Cable's digital audio coax cable...

        Ah hahahahahahahahaaaaaaa [cobaltcable.com]

        We provide ultra high bandwidth connectors to provide excellent signal isolation and signal transfer for any application.

        Oh my god! Kill me now... hahahaahaaa

        This cable provides brilliant clarity and top-notch performance for stereo or multi-channel (Dolby Digital*, DTS, etc.) digital audio.

        Kill m
        • Holy cripes.

          Heck, I'm using a 50' section of Cat5 spliced to get digital audio from my main computer over to the living room...

          I know there can be issues at high speeds, where things like crosstalk and impedance can play a part, but come on, 1.5 meters?

          You could realistically use vacuum cleaner cords for the signal transfer....

      • I'm the furthest creature possible from a stereophile, but I don't see what the problem with the Speaker Cable Face Off article is...

        I found it very informative, it tells me a few companies are selling overpriced shit (as expected), and it gives you some hard, testable numbers.

        I'd prefer to use that "generic" brand that they ranked a runner-up if I had to purchase a large amount of speaker wiring for a new house or something. You have to buy the cable from _somebody_. It's nice to know if some cheaper bra
    • It's not a 30 mile trip to your ISP. It's a 1000 meter trip to your PBX. And all the cables leading up to your house are better than the ones inside it.

      I had a 50 foot crummy telephone cable running from our one phone jack to our DSL modem. Whenever someone began a phone call, the DSL connection would die for 30 seconds. I was whining about it here, on /., and someone who seemed to know what they were talking about said that could be due to an excessively long telephone cable leading to the DSL modem.

      May
      • Maybe it'd have been ok if we used a 50 foot coax cable.

        Nope. Phone lines are designed for twisted pairs, just like modern Ethernet. Coax is no good since you have to ground the shield which will screw up a phone line.

        Cat 5 would be a good choice, or real phone wire. The extension cord probably has no twist to it, which makes it more prone to picking up noise.

        • When I was a phone tech for a small dial-up ISP, I frequently talked with people with connection quality issues. First suggestion? Pick up a "round" (read, twisted pair) phone cable from Radio Shack to connect the modem to the wall.

          It worked, for most people. Of course, if it had just rained, I told them to expect it to get better after it had dried out outside, since the local phone company's hardware didn't handle moisture well. (Our equipment connected to the telco via a PRI. Our customers, of cours
    • While the information in your post holds true in the audio world, things start getting a little different when networking comes into the picture. While Monster Cable is quite possibly the single greatest scam perpetrated on consumers of the last 10 years, things change when you move from nice, low-frequency audio into the world of computing.

      As you probably know, standard twisted pair cable comes in several grades, or categories. Cat 3 is the minimum acceptable for 10baseT, Cat 5 for 100baseT, and Cat 5e f

      • I've seen 10 meg go over a barbed wire fence...

        So that's how you're supposed to handle physical security for your workstation. My new hero.

        • Re:literally speaking, no (Score:5, Informative)

          by forged ( 206127 ) on Monday January 03, 2005 @09:31AM (#11243722) Homepage Journal
          I've seen 10 meg go over a barbed wire fence...

          Happy you got modded funny, however Long Reach Ethernet [cisco.com] (LRE) does exactly what it says with very good throughput (we're nowhere near the alledged 80% packets loss of the parent post).

          Oh, and the video clip which shows Ethernet over barbed wire is at the same url on the right-hand side where it says "Video: Charlie Giancarlo Demonstrates LRE Technology". It's nice to see it once for the "Wow!" effect. You'll also see the demo go over Cat3, Cat5, speaker cable, coax and lamp cord...

    • Please, if you don't know what you are talking about, shut up. You are spreading misinformation.

      First, there is no such thing as a digital signal. You can't send numbers through a wire, you can only send voltage levels. This is an example of an analog signal. Poor quality cabling or interference can and will cause errors in transmission.

      Second, nobody uses 0/+5V signaling for anything modern. This is not compatible with high bitrates. For example, USB 2 high speed uses 400mV differential signaling a
      • By the way, you can't use audio cable for SPDIF. SPDIF requires coaxial video cable (75 ohm impedance). It will not work well with anything else.

        I'm currently using audio cable to couple my DVD player to my receiver, and have never had an issue with this set-up. I don't understand how a short distance run (especially 6 feet) would have impedance screw it up totally.. maybe I just have a very forgiving receiver...
      • By the way, you can't use audio cable for SPDIF. SPDIF requires coaxial video cable (75 ohm impedance). It will not work well with anything else.

        See, you had me on the voltages thing because I really didn't know. Stuff in class has always been in terms of 0, +5, and -5 and every power lead I've ever seen used in a computer was 5 or 12 volts so I just assumed most data connections were like that.

        But cheap wire works just fine for any digital audio connection I've ever made, and I don't think I have any wi

        • Re:literally speaking, no (Score:4, Insightful)

          by alienw ( 585907 ) <alienw.slashdot@ ... inus threevowels> on Sunday January 02, 2005 @07:53PM (#11240581)
          Well, actually, SPDIF requires 6MHz bandwidth. As far as the 75 ohm impedance: I didn't pull it out of my ass, it's specified by the relevant standard (IEC958), and it's not some kind of audiophile debate. Basically, it's the impedance of standard TV coax. Pick up any book about radio and read about impedances. Here's a link to a short description of SPDIF, in case you are curious: here [epanorama.net].

          As for your link: don't believe everything you read. The Audioholics article shows many gross misunderstandings. For instance, the reason direction is marked on some high-end cables is for optimal grounding, not because cable manufacturers don't know audio is AC. Also, they seem to have failed physics when they claim that a battery cannot do anything if the circuit is not complete. Ever hear of FETs?

          Cable quality certainly makes a difference for just about any application, including audio. Even a "digital" protocol like USB imposes a number of requirements on the cable quality. Cable quality for sensitive analog signals is even more critical. If you have a few hundred thousand dollars' worth of test equipment, you can probably quantify the differences, calculate bit error rates, and so on. It's easier to just listen, though.
          • It's easier to just listen, though.

            I've seen many people fooled by listening.

            Take two stereo's. One has 0.005% THD and the other has 0.1% THD. Do a side by side test, but have the 0.1% reciever set 3 DB louder. Guess which sounds better?

            The Audioholics article shows many gross misunderstandings

            What I don't understand people who spend $10/foot for speaker cables, then don't want to damage an expensive cable so they have a 30 foot cable to go 5 feet to the speaker.. If you are measuring results of
            • Take two stereo's. One has 0.005% THD and the other has 0.1% THD. Do a side by side test, but have the 0.1% reciever set 3 DB louder. Guess which sounds better?

              Just to clear up a popular misconception: THD has very little to do with the perceived quality of an amplifier. There are many different types of distortion, and humans are not equally sensitive to all of them. It's difficult to hear 1% THD if it's mostly 2nd and 3rd harmonic distortion, but 0.2% THD of mostly 7th harmonic will sound nasty. This
        • Are you still a EE major? If so, your spreading of misinformation is troubling. I'm a MechE major, and your posts are making me really question Clemson's schooling.
    • No, yourself.

      It matters how many wires are in the cable. A cable is a mass of wires, and the more wires, the more data can be sent at once. Hence the sucktasticness of phone lines as internet connections with their few measly threads of single wires. Now, a broadband cable line has a whole mess of wires in it. A T1 or T3 line has an unfathomable number of seperate cables twisted upon each other within it. It's not the wire that matters, it's the number of them.

    • Best Buy sells Monster digital audio cables at something like $20 for a 5-ft cable. I had to argue with my father trying to convince him that the cheap RCA cables we already had back at home would be perfectly capable of communicated a digital signal the 5 inches between the DVD player and the receiver. I could have ripped two wires from a speaker cable to connect the two devices and would have gotten just as good sound.

      Never forget that digital operates in a low-level analog way (because the world is a

    • I had to argue with my father trying to convince him that the cheap RCA cables we already had back at home would be perfectly capable of communicated a digital signal the 5 inches between the DVD player and the receiver. This is 100% true, but the only thing I would ever pay more for is gold-plated connectors to avoid corrosion. Fortunately, one doesn't have to spend much money to get gold-plated connectors, because the amount of gold involved is tiny.
    • When it's transmitted over an analogue medium. Be that the aether or be it a piecr of copper or glass.
      You CAN NOT get a perfect, unlimited bandwidth square wave over an analogue mesium (which includes just about everything in the real world) and when you're talking about 1GHz or 10GHz signals, these are microwave frequency signals that, when interpreted correctly, correspond to a digital bit pattern. Your analogies with consumer audio are only valid up to a point, you're never going to push much more than
    • Speaking as someone with a recording studio (and a computer science degree) --

      Quality makes a difference for analog transmission. Quality analog cables are shielded well, and have thick woven ductile copper and thick insulators, and low impedance. That being said -- dont believe the hucksters who tell you that their cables are specially wound to deliver your audio at the correct phase. Utter nonsense

      Digital cable, makes much less a difference, but you can avoid retransmission with cables that arent te

  • Crosstalk (Score:3, Interesting)

    by macz ( 797860 ) on Sunday January 02, 2005 @04:43PM (#11239623)
    Short of a faraday cage (around each cable) there isn't any way to prevent ALL cross talk, but it is surprising how important using quality cables (Cat 5e or better yet Cat 6) is to reducing overall network latency.

    If nothing else, in an extremely complex environment, if you use a quality cable and quality connectors (skillfully attached) you can eliminate the bus as "one more thing to check" if you are getting unexplained slow downs. It is a nice way to shorten the troubleshooting to do list when you are up to your eyeballs in alligators and the pager wont stop buzzing.

    • Short of a faraday cage (around each cable) there isn't any way to prevent ALL cross talk

      It's called a shielded cable. They are used quite often, though not for networking (due to their higher cost).

      • The cost is negligible when you're looking at the cost of the cable vs. the labor costs of installation.

        I've used shielded for long, crowded data runs, when you're trying to get to the building demarc, or through some other area that's not in your control.

        The real problems I ran into were that it was thicker than unshielded [which is a real problem when you have to go through a rather small hole that's already loaded with cables, which is why we were using shielded in the first place], and it's less flexi
        • shielded cat5e seems to help in my office where most of our cabling in in the drop ceiling and we can't all ways give the florescent lights a wide berth. Always try to keep the computer cable at least 1M from the Florescent lights; the ring signal on telephones can cause havoc too. Life is to short to pull cheap cable.

    • Generally when the term "crosstalk" is used in reference to ethernet and similar technologies, what they are referring to is basically the TX pair and RX pair in one cable interfering with each other, as opposed to two whole seperate cables interfering.

      And you're right that cable quality can affect latency, but only indirectly. The real latency of the packets is unaffected, but if your bad cabling causes lots of transmission errors, then the packets have to be re-transmitted more often, causing latency up

  • Use high-quality cabling, but don't overspend. (Score:3, Interesting)

    by mewyn ( 663989 ) on Sunday January 02, 2005 @04:46PM (#11239630) Homepage
    Quality cabling always will make a difference. Not quite a computer situation, but still similar, my friend was recently hooking up a new DVD player up to his new projector with component video inputs. He first just grabbed the first pair of RCA cables that he could find. The projector kept resyncing with the YPrPb inputs. Despite soomeone else's refusal to accept it, I told him to pull out some video monster cables. Once he did that, it eliminated the resyncing.

    Cable quality will affect both digital signals and analog signals alike. A bad quality cable will generate a good share of dropped packets, or corrupted data, causing more resends or less accurate data. Also, take care if crimping your own cables, make sure you untwist wires as little as possible, and break the insulation and sheilding as little as possible.

    With that said, don't be like a crazy audiophile (key word here crazy) and spend thousands of dollars just on cabling (I know an audiophile who spent 500 dollars on a 6" cable, when a $25 monster cable has the exact same specs. He claims to hear a difference, but I call b.s. on him.). Spending more means getting better, but only to a point.
    • $500 for a 6 foot audio cable? Your friend prolly got some cheap low-end swill. :-)

      Here's the real good stuff [amusicdirect.com]. I wish I weren't making this shit up.

      This previous /. thread [slashdot.org] has a few other interesting examples, including the one above.

      Me? I just hook everything up using lamp cord.
      • While we're buying solid silver cables, why don't we go shopping for a solid gold rocket car!
        • I doubt there's 31 kilograms of silver in that cable. (7600 US Dollars worth of silver). And even if there was even 3 kilograms of silver in that cable it's still a pretty ridiculous mark-up. :-)

          (Ignoring for the moment the fact that other raw materials and production cost enter into the picture, since by comparison, they're laughably small.)

          But then, if they can make these cables and make a profit -- and the guys buying them actually notice a difference and are willing to pay the price, more power to the
    • no, WORKING cables make a difference.

      as long as the cables aren't totally fucked(as to drop insane number of packets for it to matter and so on) and as long as they work.. it doesn't matter.

      his rca cables obviously didn't work.

      and on cables that cost anything over 20 bucks the specs are largely pulled out of someones asses as they would be the same always(which is after some fancy marketspeak begins)..

      'high quality' cables are an example of how you would be able to sell refrigators to eskimos, so to spe
      • Bullshit. Cable quality is important for just about anything -- ESPECIALLY video. If you are using dollar-store RCA cables that barely work for audio, they will not work at all for video or digital audio. Audio requires frequencies of less than 20KHz, video requires several megahertz. If you don't believe me, go run gigabit ethernet over Cat 3 and tell me how well your setup works.
        • I'm saying it doesn't matter as long as they WORK in their job. you're saying that they don't work.
          ergo,simpslaspimdapum etc - > THEY DON'T WORK.

          I said that cables don't matter AS LONG AS THEY WORK ON WHAT THEY'RE DOING.

          the main point of this whole slashdot post is "Do i need to opt for the ultra high highest of high luxury cables and crimps?" to which the answer is an obvious "no, you don't need diamond triple weaven super spirrers, you just need cables that work, your cpu's wont get any faster if you
    • He first just grabbed the first pair of RCA cables that he could find. The projector kept resyncing with the YPrPb inputs. Despite soomeone else's refusal to accept it, I told him to pull out some video monster cables. Once he did that, it eliminated the resyncing. The problem wasn't with the original quality of those RCA cables, it was that they were probably physically broken from age or abuse. The fact that the working cables were Brand Name cables is irrelevant, because any current flowing would work

  • Yes, and documentation (Score:4, Insightful)

    by rueger ( 210566 ) * on Sunday January 02, 2005 @04:48PM (#11239644) Homepage
    In any setting the quality of cabling does matter. Or, more specifically, the quality of the connectors on the cable, and the quality of their installation.

    Anyone who argues otherwise should recall that the first step in troubleshooting is almost invariably to check the cables.

    While I am happy to use zip cord [audiovideo101.com] to wire my stereo speakers, I wouldn't trust dollar store [biglots.com] cables for anything mission critical.

    More important though is to document your cable runs, or even better tag each cable so that you have some idea where it begins and ends. You may know what goes into and out of that big ball of CAT-5 on the floor, but the guy who follows you will have no idea.
    • Thinner wires typically have higher resistance to current flow (impedance - measured in ohms) than their thicker counterparts

      Thanks for the bad information in your link. Resistance is a DC measurement of the conductor in a wire. The longer the cable the more resistance. The smaller the conductor the more the resistance.

      Impedance on the other hand has absolutely nothing to do with conductor resistance, or cable length. That's why a Cat5e cable of any length is still about 120 ohms in impedance. Imped

      • Length is not a factor in impedance of a cable.

        A nit, I know, but an important one. Length is not a factor in the characteristic impedance of a cable. The impedance of a cable (by which you mean its total impedance) is related to length. The original article was confusing characteristic impedance, impedance/ft, capacitance/ft, resistance, and loss per foot.

  • It's funny that you mention MareNostrum & IBM's blade JS20. When IBM's Blue Gene occupies the top spot and will continue to get faster. If you dredge around on IBM's Microelectronics site you can find pictures of a part of Blue Gene Prototype (a prototype of a prototype, I suppose). Which has the most beautiful wiring I've seen in years, real attention to detail in both form & function.

    All wiring has specs, all specs come from those nasty equations that made us learn in college (and most engineer

  • Behold the Rat's Nest,

    If your datacenter is 24/7, doing costly (financial), life critical (healthcare) or corporate production, then cabling ranks right up there with A/C and power. In fact all three of these are more important than apps or server platforms.

    I mean, most signal cabling is now part of a network, (IP, FC, ESCON, Token Ring, etc.). A single cable failure can lead to a network failure which, like an A/C or power failure, affects a good portion of the datacenter.

    I've seen poor cabling take o

  • Under the floor? Reconsider! (Score:3, Informative)

    by Myself ( 57572 ) on Monday January 03, 2005 @12:02AM (#11241691) Journal
    I strongly disagree with putting cabling under the floor. Out of sight, out of mind works fine when you're talking a few cat-5 runs to a cubicle farm, but when connections are your main business, put them up where they won't be neglected.

    The under-floor space can be used for AC power, if you use AC, but that's usually just for convenience outlets. Downflow air handling units that use the floor for air distribution are good too. I've even seen installations where the DC power cabling was run under the floor, and it simplified things greatly. But please, don't put your signal cables down there.

    For one, it's easy to drop a tile into the floor while trying to remove it. One bad suction cup can cause the crushing or cutting of a cable. For another, it's awkward to feed cables through the little cutouts at the bottom of a cabinet. I've seen a lot of dirty or damaged connectors because of this.

    If you're not bolting your cabinets to the floor, it also creates a shear point if the cabinet shifts. Please do bolt down your racks and cabinets, because they can tip.

    Hiding the cabling also encourages poor workmanship. When someone has 20 feet of slack to store, and they throw it in a clump under the floor, it's a nightmare when another cable in the same area has to be pulled out. The initial infraction would've been noticed immediately if it'd been overhead, in plain view.

    A well-designed overhead cable rack system is superior to any floor system. It's cleaner, because there's literally less dust colleting on it. Running cables overhead doesn't involve dragging them through a pile of connection-ruining crud. It's easier to install, because you don't have to contend with tile supports. It's easier to expand, because you can visualize the whole layout easily, and see where the congested areas are.

    Furthermore, overhead rack is a natural companion to fiber trough systems, most of which are intended to be overhead. If you have a mix of fiber and copper, and most of us do, you owe it to yourself to plan a system that accomodates large amounts of both. As equipment density rises, the amount of cabling you'll need to bring to each rack also rises. Plan for that.

    Also, plan for slack runout areas. The cables are never the exact length you need. Running them back and forth in the rack can create all sorts of tangle problems. Having a designated path to run your slack loop down can really make tearouts less dangerous.

    Also, don't underestimate the sheer size of the cabling you're dealing with. I saw one particularly bad example, where a company had laid out their aisle of patch panels very carefully. The bottom of each bay was for panels that went to transport equipment, and the top was panels for other equipment. That way, most cross-connects could be made without leaving the bay. There were cable management rings to accomodate the occasional jumper that had to go between bays. It worked great.

    Then they merged with another company, and the recordkeeping system changed. The new system made port assignments automatically, and it didn't respect the physical layout of what was where. Now the majority of jumpers were long, inter-bay runs. Over time as circuits got moved around, the management rings got filled, overoaded, and eventually stuffed to the point that the mass of wire was essentially solid. You could punch the bundle and it would go "thud". Pulling out a jumper was likely to burn through its neighbors simply due to friction, so they stopped pulling old ones out.

    Eventually they added a dedicated piece of cable rack, and run all interbay jumpers up there. It was ugly, and awkward, but it worked. The initial system was much better, but relied on a level of care and planning that the new owners weren't willing to provide. Consider this: Will your successor's successor curse your name, or laud you for laying out a comfortable, expandable environment?

  • Cabling Strategies (Score:3, Interesting)

    by bill_mcgonigle ( 4333 ) * on Monday January 03, 2005 @12:42AM (#11241941) Homepage Journal
    It's not the cables, it's how you cable. Kentucky bred a cabling strategy [arstechnica.com] for their cluster. The Big Mac project at VT was supposed to release a software package that made cluster cabling easier, IIRC, but I can't find it anywhere.

  • colors and numbers (Score:3, Interesting)

    by smoon ( 16873 ) on Monday January 03, 2005 @05:57AM (#11243057) Homepage
    One thing we do with 'cat 5' cables is color-code different length cables, so black == 3 feet, green == 7 feet, yellow = 10 feet, orange == 14 feet, red == 25 feet, grayish-white == 50 feet, red with yellow boots == crossover cable. This has been helpful in a number of ways. -- it's unlikely a white cable will be to something else in that cabinet, crossovers are easily identified, longer cables are probably for servers further away from the switch/patch panel/whatever, it provides some color distinction in otherwise monochrome patch panels, it's easy to stock and order cables like this.

    One thing I wish we did is have unique serial numbers on both ends of each (and every!) cable. While it's possible to trace cables using the tried-and-true tug-and-feel method, in reality it sucks and printed documentation is difficult to keep in sync with reality.

    I've also seen cables color-coded for other purposes, but these haven't worked as well e.g.: one color is for network, another for KVM, another for switch uplinks, etc. This works well until you need a KVM cable, but don't have the right length in the right color so substitute "temporarily", blowing the scheme completely since 'temporary' is a synonym for 'permament' in most datacenters. another example: Use every color available randomly in the hope that there are only so many hot-pink cables with a green stripe in your datacenter making it easier to trace things. In reality this last example doesn't scale well and makes patch panels look really untidy.

    As far as what I *think* you were asking, which is whether there is some qualitative difference between cables -- there is. Make sure you get 'certified' cables from a trusted vendor, preferably each one individually tested with the results pasted on a sticker on the (sealed) bag each cable comes in. Also make sure you get 'plenum' cables where necessary to comply with fire codes and just plain common sense. I'd say any permament infrastructure cables (not patch cables) should be plenum whether they are legally required to be or not -- if you have a fire you'd be better off without a few hundred extra pounds of fuel to keep it going. Beyond plenum/pvc and tested cables there isn't much else to stress over -- thank god "Monster" doesn't make patch cables with 24k gold connectors to hoodwink unsuspecting people -- if the cable tests good the rest doesn't matter.
  • The patch panels can make all the difference.

    In some cases, cabling can be a beautiful thing. [neatpatch.com]

  • The quoted post refers to the heavy bending of optical cables used in one of the pictures. Optical cables should not be bent more than a certain amount to prevent the actual fibre from breaking.

    In essensce, no there is no need to buy "air cables" for over 1000 per metre. Computer engineers are usually not as guillable as "audiophiles" :-)

Slashdot Top Deals

Two can Live as Cheaply as One for Half as Long. -- Howard Kandel

Close