Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
AMD Hardware

Cray Supercomputers to be Based on AMD Opterons 197

Posted by ScuttleMonkey from the all-eggs-in-one-basket dept.
PsychicX writes "AMD and Cray have announced an agreement to base Cray supercomputers on AMD's Opteron line until the end of the decade, and to collaborate on Cray's 2006 proposal for Phase 3 of the federal government's DARPA HPCS (High Productivity Computing Systems) program. Cray already offers the XT3 and XD1 supercomputers based on Opteron."
This discussion has been archived. No new comments can be posted.

Cray Supercomputers to be Based on AMD Opterons More

Cray Supercomputers to be Based on AMD Opterons

Comments Filter:

  • excellent (Score:5, Interesting)

    by harryoyster ( 814652 ) on Monday November 14, 2005 @02:29PM (#14028452) Homepage
    That is excellent news for AMD even though there wont be massive volumes compared to home markets it will still be some heavy industry weight backing the AMD opteron processor. Hopefully AMD will adopt some additional features that could make the Opteron even better suited to the super computer market.

    • Re:excellent (Score:3, Interesting)

      by 14erCleaner ( 745600 )
      Except that this "industry heavyweight" is actually Tera [com.com], the little company that bought out the Cray name as the supercomputer industry was dying. Only the name Cray remains, not the old-time reputation.

      • Re:excellent (Score:5, Informative)

        by fgodfrey ( 116175 ) <fgodfrey@bigw.org> on Monday November 14, 2005 @02:41PM (#14028548) Homepage
        Tera bought far more than a name when they bought us. They also bought a bunch of software and hardware people, many of whom (myself not included) have been with Cray Research (the original Cray) for many years. So, while it's certainly not the Cray of the mid-1980's, the tradition still goes back there, especially with the vector machines like the Cray X1/X1E and its impending follow-on.

        • Tera bought far more than a name when they bought us. They also bought a bunch of software and hardware people, many of whom (myself not included) have been with Cray Research (the original Cray) for many years. So, while it's certainly not the Cray of the mid-1980's, the tradition still goes back there, especially with the vector machines like the Cray X1/X1E and its impending follow-on.

          You work for Cray. Cool.

          Please tell me that this deal implies that AMD is going to add some proper vector instructions to

          • Re:excellent (Score:5, Informative)

            by flaming-opus ( 8186 ) on Monday November 14, 2005 @05:32PM (#14029990)
            No they won't! They have no reason to. The vector units that a cray uses aren't like altivec, sse, or other "bolt-on" vector units. The vector unit on a cray (or NEC) is a latency hiding mechanism. It's a method for forcing the programmer/compiler to structure the code such that the data loaded from memory is used a significant period of time after the load is initiated. This works pretty well on the HPC code that is used on crays, but not at all for the everyday server/workstation code that opterons run. Furthermore, to support that sort of vector unit, you need to have about eight times as much memory bandwidth as an opteron, which means many more pins on the socket, which are very expensive.

            I think you're much more likely to see the cray vector processor retooled with lots of hypertransport connections, so it can use an opteron as its scalar unit, and use the same seastar routers that the xt3 uses. On the X1, the scalar unit already runs ahead of the vector unit, so I bet it's not all that important for the scalar unit to be on-die.
            • its a cm5. as to why anyone would choose to build that machine again,
              i dont know.

            • Re:excellent (Score:4, Informative)

              by joib ( 70841 ) on Tuesday November 15, 2005 @07:19AM (#14033675)

              No they won't! They have no reason to.


              Yes, you're probably right that it doesn't make sense for AMD economically. But I want to run numerical codes at more than 5 % peak performance on my cheap Opterons, so I want to believe. ;-)


              The vector units that a cray uses aren't like altivec, sse, or other "bolt-on" vector units. The vector unit on a cray (or NEC) is a latency hiding mechanism. It's a method for forcing the programmer/compiler to structure the code such that the data loaded from memory is used a significant period of time after the load is initiated.


              Yes, I know. And that's precisely the reason why I'd like to see real vectors instead of the sse/altivec toy ones. Main memory latency is hundreds of cycles, and it's getting worse all the time.

              Additionally, from a microarchitecture perspective, vectors have quite a few advantages there too.


              This works pretty well on the HPC code that is used on crays, but not at all for the everyday server/workstation code that opterons run.


              I'm not sure about that. I guess technical apps vectorize just as well as HPC codes (well perhaps not the UI, but the code that runs the actual simulation or whatever). Heck, even some database code vectorizes nicely (sorting and hash joins).


              Furthermore, to support that sort of vector unit, you need to have about eight times as much memory bandwidth as an opteron, which means many more pins on the socket, which are very expensive.


              Yes, as I said some Alpha Tarantula like design is probably overkill for the vast majority of the market. My point was that a vector ISA extension with modest execution resources wouldn't need that much die area, and could help make better use of the available bandwidth, whatever that bandwidth is. As you said yourself, the expensive thing is IO. Transistors are cheap by comparison. So not having instructions that allow one to effectively use the available IO resources is a real shame.


              I think you're much more likely to see the cray vector processor retooled with lots of hypertransport connections, so it can use an opteron as its scalar unit, and use the same seastar routers that the xt3 uses. On the X1, the scalar unit already runs ahead of the vector unit, so I bet it's not all that important for the scalar unit to be on-die.


              Yes, that sounds feasible. IIRC it is something like this that Cray has cooked up for the Cascade project; I.e. a node consists of 8 (or was it 4) scalar processors connected to memory (I guess these could be Opterons or further in the future some kind of Processor-in-memory (PIM) stuff), and a vector unit with its own cache and fast access to the main memory via the scalar cpu:s.

              As for the seastar thing, I think you're right that that's what they'll use for inter-node communication. Currently X1(E) uses Numalink licenced from SGI, so they're certainly looking at replacing that with existing in-house tech. BTW, 2H2006 will see the XT4, with the new Opteron sockets with DDR2 memory and the Seastar2 router that provides twice the BW compared to the existing Seastar.
        • forrest, tera is finally fully dead, which you know full well.

          so its all up to you now to make it happen, you have the
          company to yourself. the success or failure of which will
          be entirely of your own making.

        • Re:excellent (Score:2, Interesting)

          by Frumious Wombat ( 845680 )
          So, just curious, but why don't you put your efforts into crunching your traditional vector processors onto some more affordable piece of silicon, rather than trying to recreate the T3E out of Opterons?

          I know chemists who claim there there are still algorithms than don't run as well on modern MPars as they did on mid-90s vector Crays. I know we're not a huge market, but I bet there are some other fields that would rather have a deskside T90, rather than a multi-proc Opteron box.

      • Re:excellent (Score:5, Interesting)

        by Aardpig ( 622459 ) on Monday November 14, 2005 @03:07PM (#14028777)

        Only the name Cray remains, not the old-time reputation.

        That's not quite true; they still sell Cray-specific technology. One of my colleagues has just bought a small 24-core Opteron system. Each node contains two dual-core processors, and the 6 nodes are linked together by 80 Gb/s Craylink cables. I think this interlink technology is also licensed to SGI for use in their Origin computers.

        • One of my colleagues has just bought a small 24-core Opteron system

          Small? Small? You've lost your sense of scale with all the weather modeling and nuclear RSA NP-hard protein folding simulations you've been running.

        • Re:excellent (Score:2, Informative)

          by tzot ( 834456 )

          ...by 80 Gb/s Craylink cables. I think this interlink technology is also licensed to SGI for use in their Origin computers.

          Which they call NUMALink (for Non-Uniform Memory Access --a node obviously accesses its memory faster than through the interconnection), and they're used in their Altix (Linux on Itanium2) computers too; I don't know how much they have evolved the technology, I just know that it's 3.2GiB/s each direction.

          NUMALink [sgi.com]

          Have you got any links to some page describing some configuration simi

        • Re:excellent (Score:5, Informative)

          by flaming-opus ( 8186 ) on Monday November 14, 2005 @05:13PM (#14029847)
          Close.
          Craylink was designed at SGI, and renamed to craylink after they bought Cray. They introduced craylink in the origin2000, which they started selling half a year after buying cray, so I'm sure they couldn't have integrated any cray-designs into their product in that span.

          After they sold Cray to Tera, SGI started calling the technology Numalink, and currently use it in their origin3, altix3, and altix4 product lines. They are on the 4th generation of the technology, which is 3.2GB/s per direction. The cray that was sold to Tera included the half-finished X1 system, which also uses numalink. It uses the older 1.6GBps/dir links, but uses 32 networks in parallel for a total of ~50GB/s/dir per node.

          The Cray XT3 uses a newer network interconnect called seastar, which offers 3.8GBps/direction. This is probably what will be used in the X1's successor.

          The Cray XD1, which your colleague bought, is a product cray acquired when they bought OctigaBay. They use an interconnect called the RappidArray switch, which provides 4GBps/direction of interconnect.

          All of these interconnects are high-bandwidth and low latency. The XD1, is also very inexpensive for a cray, which is always nice.

      • Re:excellent (Score:5, Insightful)

        by flaming-opus ( 8186 ) on Monday November 14, 2005 @03:37PM (#14029022)
        Which is true only insomuch as the old-time reputation could not possibly exist today. That was the cold war, this is the post-coldwar era. The old cray was a mammoth beast with its own share of myopia, but a lot of technical tallent. This allowed a few really brilliant concepts, and a lot of clever implementation to power two decades of brilliant computers. That said, they were brilliant solutions for the era. Old-school cray systems were built in an era when doing fundamental pieces of math was still pretty difficult, and the government was willing to pay ten million dollars for a machine that was proficient at doing math, and many tens of millions for a machine that was really good at it.

        The difficult problems in building computers has changed, and the financial climate around supercomputers has changed quite a lot. Among other things, CMOS finally became fast enough to put bipolar in its grave, single microprocessor workstations became powerful enough to do all but the hardest of scientific tasks, and the average price of high performance (not top 10 on the list, but still fast) computers has plummeted. To ask the new cray to be like the old cray would be foolish.

        That said, New Cray is still offers impressive products. All of Cray's 3 product lines have much lower entry-prices than similar crays of the 90's. They all have more managable power/cooling/physical size characteristics. They make much greater use of industry standard Disks and networks, and also can be administered and programmed much more like any other unix computer. You program a New Cray more or less the same as other contemporary HPC systems.

        When cdc introduced the 6600, the president of IBM complained to his staff aking (paraphrase) 'how has cdc managed to best IBM's fastest computer with a staff of just 14 engineers and 4 programmers?' Seymor Cray responded "It seems like Mr. Watson has answered his own question." Because new Cray is tiny does not mean that it is not capable of making impressive innovations. Old Cray's Gorilla days were very wasteful, and not necessarily full of the best moments of innovation.

        Now, if only they could put four X1e CPUs into an air-cooled, rack-mount server and charge a reasonable amount for it. I'd much rather have a handful of vector processors than a few dozen opterons, anyday.

        • Now, if only they could put four X1e CPUs into an air-cooled, rack-mount server and charge a reasonable amount for it. I'd much rather have a handful of vector processors than a few dozen opterons, anyday.


          NEC sells an entry-level deskside SX-8, called IIRC SX-8i, with one processor. Unfortunately "entry-level" in this case means $100000+. :(
          • Yes True.
            And NEC has done a great job of doing this for the last several generations of their vector machines. I have not ever programmed for an SX, and don't know much about them. The really nice thing about the X1, is that under the covers it's running Irix, which is a pretty reasonable Unix variant. Anyone know anything about super/UX?

    • Re:excellent (Score:2, Insightful)

      by Rude Turnip ( 49495 )
      This is really just a marketing play on AMD's part. Now they can sell you:

      "The SAME CPU used in CRAY ***SUPERCOMPUTERS***, now available for your desktop!"

      And some rube will buy one based on that statement.

  • It only makes sense (Score:5, Insightful)

    by heatdeath ( 217147 ) on Monday November 14, 2005 @02:33PM (#14028477)
    Specialized computing hardware for supercomputers has always seemed like a fiscally bad choice. It'll be good to see what kinds of improvements we can see in research possibilities as supercomputing costs come down from using mass-marketed parts.

    • The Chipset is the key. (Score:4, Interesting)

      by mosel-saar-ruwer ( 732341 ) on Monday November 14, 2005 @02:50PM (#14028629)

      Specialized computing hardware for supercomputers has always seemed like a fiscally bad choice. It'll be good to see what kinds of improvements we can see in research possibilities as supercomputing costs come down from using mass-marketed parts.

      Cray likes to build classical vector-driven machines. In that space, you can't rely on some external kludge like Myrinet for your communications; instead, your value-add is in the chipsets that get all those CPUs talking to one another [and to the memory subsystem].

      In one of Cray's previous incarnations, they once possessed a chipset/backplane tech for the Sparc processor that Sun purchased off of Silicon Graphics for a song and a dance, and immediately turned into the insanely profitable Sunfire series. The big question here is whether this new agreement requires Cray to share their chipset/backplane tech with AMD [in which case some of it might filter its way back down to the level where mere plebians like us would be able to afford it].

      • Of course, cray didn't invent the 6400 either. They bought out a competitor in bankruptcy. Sun, at that time, wasn't quite ready to go head-to-head with s/390s, which is essentially what they did when they finally got around to selling the s10000 (same computer, new name). Chipset is one key, but software is an even bigger key.

    • Re:It only makes sense (Score:5, Informative)

      by fgodfrey ( 116175 ) <fgodfrey@bigw.org> on Monday November 14, 2005 @02:55PM (#14028681) Homepage
      I've never completely understood this argument (yes, I admit, I'm heavily biased). If I want to build a skyscraper, I'm not going to use the "mass market" crane that puts up the roof of a residential house. I'm going to use a specialized crane that's meant for building skyscrapers.

      That doesn't mean that there isn't a place for commodity hardware in supercomputing, but to say that there's no room for custom hardware either misses the point. The only thing "off the shelf" about an AMD based Cray is the AMD. The logic board, and, most importantly, the network that interconnects the processors is entirely custom. Not to mention the fact that Cray will still build some entirely custom processors...

      By the way - this is hardly the first Cray based on a commodity processor. The T3E and T3D were both Alpha processors, yet nobody calls those machines "commodity".
      • By the way - this is hardly the first Cray based on a commodity processor. The T3E and T3D were both Alpha processors, yet nobody calls those machines "commodity".

        I really don't consider Alpha to be a commodity chip. While Opteron and Athlon64 share a lot of the same designs and even might have the same masks, it doesn't necessarily make it a commodity chip. I suppose either chip were produced in much greater volume than Cray's custom processors.
      • Convex made the prediction years ago that as the technology powering commodity processors advanced, the market for specialized supercomputer hardware would die off.

        Looks like it's happening. The Superdome can either run PA/RISC or Itanium2. Opterons on a Cray system?

        And it's not just limited to processors either. I have a bag of sticks of 72-pin DIMMs in the closet (yes, the kind PCs used to use) that came out of a supercomputer. Technology advances in supercomputing will make their way back into the genera

    • The whole point of a supercomputer is that it performs well beyond what commodity systems are capable of. When supercomputers are made entirely of commodity parts, there will be no supercomputers.

  • #34. Your "supercomputing" vendor has an AOL email address.

    From the press release...
    Contact:
    AMD
    Teresa Osborne, 512-602-0040
    teresa.osborne@amd.com
    www.amd.com
    or
    Cray Inc.
    Steve Conway, 651-592-7441
    sttico@aol.com
    www.cray.com
    Sooooo... if I scrape together a few million bucks and buy a computer from these guys, will I still be able to contact my Cray rep once his 500 FREE TRY AOL NOW HOURS have expired?

  • nVidia motherboards (Score:5, Funny)

    by CCFreak2K ( 930973 ) on Monday November 14, 2005 @02:45PM (#14028583) Homepage Journal
    Hey, maybe the motherboards are nForces too. I bet all the new Crays will have digital 5.1 sound, an important feature for today's supercomputers.

  • The irony of it all. (Score:3, Interesting)

    by Anonymous Coward on Monday November 14, 2005 @02:47PM (#14028606)
    Crays supercomputers were known for their high performance vector operations. These operation have very little to do with PC world except its close cousin - SIMD operations (gaming, graphics). Now the fact that AMD tops cray (at least on commercial merits) is like having an AMD instruction set adopted by Intel (oh, wait).
    More ironic is the fact that the compiler that will be used for those supercomputers is probably the PathScale variant of Open64 - SGI's compiler that was released as open source after it was retargetted to the Itanic architecture.
    I might have some misconceptions, careful readers, please fill-in the blanks.
    • Aren't you missing the point though? It's not that the AMD systems are prevailing on the merits of their amazing vector math (they aren't) - it's that they do a PRETTY good job of both vector and scalar math, but at the prices you can get them, your cost per computation is SIGNIFICANTLY lower then it would be with one of the massive vector systems.

      The research / development arm of the organization I work for just got a 4000+ CPU XT3. Last I checked, they planned on using the PGI compiler for most stuff.

    • Re:The irony of it all. (Score:4, Interesting)

      by convolvatron ( 176505 ) on Monday November 14, 2005 @04:01PM (#14029299)
      disclaimer: i worked at cray on the xt3

      its not ironic at all. its a question of resources and volume.
      cray has a few very bright people (still, sort of). they are
      essentially a us government lab. they do a bad job, but its insane
      to think that 100 people can build and maintain several different
      supercomputer architectures.

      a $300 opteron is almost always more effective than a $60000 X1
      processor. they have alot of bright people too, and alot more of
      them.

      the only reason that cray still exists is support for parallelism
      and the provision of high memory bandwidth systems. but even that
      niche is being eroded pretty severely. the xt3 communications chip
      runs at 3.5GB/s in each direction. it costs about $250 for cray to
      have each of them made. for the same $250 i can buy a mellanox nic
      that runs at half the speed

      its no suprise that cray is using opterons. they actually got lucky
      by committing to amd early and having it turn out so well.

      the real question is whether there is any more room for a cray at
      all. the commodity world moves so quickly. the xd machines (which
      they purchased) are really their best asset, but it still hard to
      justify that kind of margin for what is essentially a well
      constructed cluster.

      • The Irony of Marketing (Score:2, Informative)

        by scoobrs ( 779206 )
        The parent poster posts well for one ignorant of the simplest precepts of marketing. The first things a marketer learns is he must segment a market and only compete in the segments or niches in which competition is profitable. Cray isn't competing directly against clusters because clusters don't have the bandwidth necessary for the sorts of problems Crays are aimed at and Crays tend to be overkill for the problems clusters are aimed at. Cray doesn't seek out customers $.5M for that reason. Anyone who ac
        • are you seriously trying to claim that cray occupies a
          position of marketing excellence?

          from a business point of view its completely whacked. cray spends
          4-5 years of time to build a machine, just to sell a very small
          few of them, throw almost all of the technology away and start
          over again.

          and are you really trying to claim that any cray machine built in
          the last 10 years has particularily good mtti rates? the sv2
          really was basically unsuable for the first two years after it
          was shipped. its still kind of a dubi

  • Finally.... (Score:4, Funny)

    by Prince Vegeta SSJ4 ( 718736 ) on Monday November 14, 2005 @02:49PM (#14028622)
    A Computer capable of running Duke Nukem Forever....oh wait...

  • It had to be said... (Score:4, Funny)

    by rubberbando ( 784342 ) on Monday November 14, 2005 @02:51PM (#14028645)
    Finally! Something that can run Windows Vista at a descent speed... :-P

  • Behold! (Score:5, Funny)

    by katana ( 122232 ) on Monday November 14, 2005 @02:54PM (#14028662) Homepage
    I give you...the Crapteron!

  • Does this mean... (Score:3, Funny)

    by __aaclcg7560 ( 824291 ) on Monday November 14, 2005 @02:54PM (#14028663)
    That prices on the older Clay computers will drop? Holy flipflops! Now I'll have something to put into my empty warehouse building. :P

  • Not to be a jerk...but this is old news.... (Score:3, Informative)

    by jamesgomez ( 808411 ) on Monday November 14, 2005 @02:56PM (#14028687) Homepage
    dated from June 16, 2005

    Check out the article here...
    http://www.hypertransport.org/consortium/cons_pres srelease.cfm?RecordID=79/ [hypertransport.org]

  • Sun.. (Score:2, Funny)

    by Bulmakau ( 918237 )
    I was just about to buy 40 sun machines, based on AMD. Maybe I should wait for Cray to come out with their product? Anyone knows the estimated retail price that machine is expected to hit the market with? ;)

    • Re:Sun.. (Score:3, Informative)

      by iamlucky13 ( 795185 )
      I know you're trying hard to make it look this is work-related, but for goodness sake, don't make major purchasing decisions based on what you read on Slashdot!
    • Insider sources say that the smaller 1024 node model will have a MSRP of 10 MegaBucks. Ofcourse shipping, installation, and remodelling the server room are extra.

      Cray already offers the Opteron Based XD1. The smallest 12 processor model is estimated to cost $100k USD.

  • Makes me wonder ... (Score:5, Insightful)

    by cpu_fusion ( 705735 ) on Monday November 14, 2005 @04:39PM (#14029590)
    The continued big-name backing of AMD (e.g. Sun, Cray) makes me wonder how sweet a deal Apple must have gotten to go with Intel over AMD. :)
    • Intel and Apple are like those kids in school who always got to go on ski trips with their year passes and high end skis, etc. They think they're all important because things are good. Then when they get into the real world they get eating alive.

      Apple as far as I can see has a bit of a disconnect with "reality". They're not customer-oriented [e.g. shitty dead pixel policies, really costly vendor-locked in gear, etc]. It was always "oh that's apple gear only because the quality is higher" yet all the fol

      • Apple as far as I can see has a bit of a disconnect with "reality". They're not customer-oriented [e.g. shitty dead pixel policies, really costly vendor-locked in gear, etc]. It was always "oh that's apple gear only because the quality is higher" yet all the folk I talk to say "just get a wintel laptop and be done with". And it isn't like the PPC isn't a nice processor [the G4 actually looks fairly sweet]. It's just they're a bunch of pricks.

        Maybe Apple's computer "just work" and look cool to boot [daringfireball.net] (and not

        • Flash news: Current AMD Turions are, in fact, 64 bits (which *may* be why their name really is AMD Turion(TM) 64)

          The 32-bit comment was about PPC not Turion....But yes I did word that wrong and it was misleading. Sorry.

          The biggest flaw I see with most incarnations of PPC [like the G4] is they shoot themselves in the foot with bandwidth. 133Mhz FSB? What's this the mid 90s? Put a good ol' dual-channel DDR400 [or DDR-II-533] on the front of it and be done with.

          I'm sure with a slightly longer pipeline [iir

    • Re:Makes me wonder ... (Score:4, Insightful)

      by Watts Martin ( 3616 ) on Monday November 14, 2005 @06:51PM (#14030613) Homepage
      My impression is that Intel has been irritated that they come up with new technologies that the PC world isn't particularly interested in. Despite the "Wintel" moniker, Intel has had a love-hate relationship with Microsoft; back in the BeOS days, Intel offered engineering assistance (and I suspect financial support) directly to Be to get the OS ported. And today, Apple doesn't have 20 years of backward-compatibility hardware baggage to deal with, so they have the potential to be a showcase partner in a way that the Dells of the world just aren't.

      As for why not AMD, though, Intel has placed a much higher focus than AMD has on very low-voltage chips, and from what I've heard, that's what ultimately gave them the Apple nod. Arguments about production capacity aside, AMD doesn't have the R&D resources of Intel, and they have to pick their battlegrounds carefully. They've picked them wisely, but as of right now, they don't have anything competing with chips like Intel's low-power, dual-core lineup for 2006. If that's remedied in 2007, I'm sure Apple would have no problem revisiting it, but right now, AMD just doesn't have the chips they want.

      (As for the production capacity arguments, I've seen people here point out that Intel has had problems meeting their demand recently and AMD hasn't. While this is true, it's important to keep in mind that Intel's overall demand is still over five times that of AMD's, and the gap in notebooks -- the segment where Intel's production capacity fell behind demand temporarily -- favors Intel by an even greater margin.)
  • AMD chips outperform Intel, for less money, using less power, for something like five years now.

    Shouldn't AMD stock be doing better? If you bought 5 years ago, AMD is flat, but Intel is up like 50%.
  • Are there not much better chips out there? ( like PPC, MIPS, SPARC, totally custom ASIC's.....)

    I guess being a commodity chip helps for supply issues, but when you are building machines that are this expensive, is that really a deciding factor?
  • I can afford that 4-way Crayteron workstation I have been dreaming about..

Slashdot Top Deals

We are Microsoft. Unix is irrelevant. Openness is futile. Prepare to be assimilated.

Close