Earth Simulator, G5 Cluster Drop In 'Top 500' List 343
daveschroeder writes "The November Top 500 supercomputer list has been published at SC2004. Topping the charts is IBM and the US Department of Energy's 'BlueGene/L DD2' beta system, at 70.72 TFlops, followed by NASA's 'Columbia' at 51.87.TFlops. For the first time in several publications of this list, Japan's Earth Simulator is no longer in the number one slot, falling to third. Virginia Tech's 'System X' Xserve G5 cluster, while 20% faster than the original cluster that debuted at number 3 last November, has fallen to number 7 due to the new entries, but remains the fastest supercomputer at an academic institution. Here's an excellent cost comparison (Google cache) of the top machines ('System X' is significantly cheaper than anything else in the top 20, not to mention cheaper than many things far below it in performance)."
Hmm (Score:5, Insightful)
Re:Hmm (Score:5, Funny)
Re:Hmm (Score:3, Funny)
Sounds right, actually (Score:4, Insightful)
"I'm not sure if the costs can be accurately compared in this way. [
Actually that sounds like a perfectly valid comparison, SETI included. In bang for the buck SETI deserves to win hands-down in that scenario, and fairly. System X deserves its place as well.
Re:Sounds right, actually (Score:2, Insightful)
Re:Hmm (Score:2)
Re:Hmm (Score:5, Insightful)
Re:Hmm (Score:5, Funny)
Re:Hmm (Score:4, Informative)
More Information here. [wikipedia.org]
Not to mention that BG beats BigMac in flops/$ (Score:5, Interesting)
Recently IBM announced their commercial prices for BG machines (see e.g. theregister.co.uk or news.com.com). Prices start at $1.5 million (1 fully equipped cabinet). Using this price and published linpack figures one arrives at about 2.9 Mflop/s/$, compared to the maximum value of 2.2 Mflop/s/$ he quotes for the best apple system.
Add in the fact that the BG uses much less space and power than a comparable xserve cluster, that it has a faster and lower latency network, and we have a winner.
MOD PARENT DOWN (Score:3, Funny)
Re:Not to mention that BG beats BigMac in flops/$ (Score:5, Interesting)
Of course, now I get to be flamed by a bunch of mac haters who think pointing out a factual error in your statement means I don't know anything else about clustering and will blindly chose a mac above all others.... *sigh*
Re:Not to mention that BG beats BigMac in flops/$ (Score:5, Insightful)
That UoM 512 CPU one is significantly smaller than the top 5 super computers. You should probably only be comparing in class. Costs do not increase proportionally with size of the computer.
Re:Not to mention that BG beats BigMac in flops/$ (Score:5, Funny)
Re:Hmm (Score:2)
Well, presumably it's a tiny cost compared to that of a true supercomputer, but SETI costs them in bandwidth - both client downloads and updates, and uploading packets of work and downloading processed packets. Even if SETI itself isn't paying for the bandwidth, *someone* is...
Re:Hmm (Score:2)
Yes, Ms. Booth Babe, I can see your points... uh, point. Flops? What are you talking about, they're quite firm in fact. OH! Oh right, FLOPS. Gotcha. Right. Yes. What's that? Hey baby, you can swipe my badge anytime.
The Dept. of Energy (Score:5, Funny)
Pizza arguments (Score:5, Insightful)
Before anyone says "Of course System X is cheaper! Virginia Tech had free student labor to put it together! They paid them in pizza!"
The only thing anywhere close to System X is NCSA's Tungsten, a 2500 processor Pentium IV Xeon Dell Linux cluster. It cost $12 million, just for the asset (comparable to System X's $5.8 million overall price, including the upgrade to Xserve G5s). That's twice the cost, and over 2Tflops less performance. 2Tflops is a top 100 supercomputer...so it's a whole top 100 supercomputer poorer in performance, for an extra $6.2 million.
Another example is PNNL's 1936 processor Itanium2 cluster: 3.5Tflops less performance than System X, for $25 million.
Any way you slice it - no pun intended - System X is still a LOT cheaper, even if you allot, say $2M for professional installation and systems integration - an EXTREMELY liberal estimate, probably by an order of magnitude.
System X also has the highest Rmax per CPU of any system on the list, except for specialty non-commodity systems like Earth Simulator.
And on top of it all, last November, they hit #3 in the world, #2 in the US, and #1 academic, as well as the first academic site to ever exceed 10Tflops, all for less than $7 million in total - including all improvements to buildings, physical plant, and other infrastructure.
That first system might not have had ECC, but what it did do is break into the top 5, following all the rules of the Top 500 organization, for relative pocket change - for a price that was absolutely unheard of, sharing the spotlight with systems that cost $100 million or more - and also catapulted Virginia Tech to a supercomputing center of national prominence overnight, able to attract additional attention, funding, grants, and publicity. Not to mention testing and proving the suitability of a completely new OS, platform, processor, and interconnect for high-performance computing, increasing choice for all (and resulting in new clusters based on the same technology, such as the US Army/COLSA cluster). And even as new systems enter the top ten in the tens and hundreds of millions of dollars, System X retains the title of #1 at any academic institution, and shares the top 10 with the best of the best.
Seems to me that Virginia Tech pulled a real coup here, and a full year later, is still considerably cheaper that anything else. And now, it's being used for real scientific work. To bring a whole new platform onto the scene in essentially under a year and break into the ranks of the supercomputing elite virtually overnight, and to do it significantly, and sometimes ridiculously, cheaper than everyone else, is a feat that can't be ignored.
Thanks for that post (Score:5, Funny)
Re:Pizza arguments (Score:4, Insightful)
No, my real question would be: What is the ongoing operating expenses of System X? After all, I'm interested in total cost of ownership, not in acquisition cost.
Re:Pizza arguments (Score:4, Interesting)
power costs? (Score:5, Interesting)
Re:power costs? (Score:5, Informative)
Several of the researchers at Virginia Tech have referred to this in various news stories numerous times - one estimate was over two times less power than comparable systems.
Re:two times less (Score:3, Funny)
Try "half as much." Damn, I hate grammar nazis like me.
Re:power costs? (Score:2)
Re:need? (Score:3)
Re:need? (Score:4, Interesting)
uh-huh. If G5 runs so cool, then surely they could have kept the original cooling-system for the 2.5GHz model, instead of going for an complicated liquid-cooling system? Really, why did they move from heatsink/fan to liquid-cooling? AFAIK the original G5's were already quiet. And looking at reviews such as this seems to suggest that the G5 does indeed run very hot. [computerworld.com]
And looking here and [arstechnica.com] here [amd.com] I can see this:
2.5GHz G5: 75-85C during load
2.2GHz Opteron: 48C during load
G5 runs cooler? Hardly.
Re:need? (Score:4, Informative)
Re:need? (Score:4, Insightful)
Re:Pizza arguments (Score:4, Informative)
-Don.
Re:Pizza arguments (Score:2, Interesting)
Huh? Very few of all the possible faults in a computer have any kind of temperature change associated with them. When a memory stick, a HD, a CPU, a system chip, a connector fails, you don't get any overheating. (You may get a slight temp decrease after the component has failed and stopped consuming power.)
Your scenario only applies to failing fans and heatsinks prying th
Re:Pizza arguments (Score:3, Interesting)
Just like the IBM xSeries servers I'm deploying right now. Our servers automatically order replacement components if/when components fail. This includes CPUs, memory, fans, hard drives.. just about anything. IBM big boxes have been doing this for decades, and distributed systems for a
VT ongoing cost of ownership evaluation... (Score:5, Funny)
Re:VT ongoing cost of ownership evaluation... (Score:2, Funny)
Re:Pizza arguments (Score:5, Informative)
We're going to consider the worst-case scenario, under which we have a 100% load, year round, on all 1100 nodes. That gives us a power consumption of 385kW and 1320kBTU/h of heat generation.
Now, we need to get rid of that heat, and that's going to require a lot of power. My research indicates up to 300kW may be required, but that's a high number and actual requirements may be lower.
So, here we are, with 685kW required for power and cooling. That means a 6000MW/h a year.
Now, the cost of power is high, since you need to amortize and maintain the UPS equipment and the generators. We'll use a figure of 0.15$/kW/h, or 150$/MW/h. Very generous.
So here we are. The absolute worst case for power and cooling. Full load, year round, expensive cooling, overpriced power and amortized UPS and generators.
900 000$/yr. Below a million. It's not that bad, is it ? The real cost is likely below a half-million.
As for the rest, well, how much pizza is really required to entice graduate students and professors to work on that machine ?
Re:Pizza arguments (Score:2)
(I'm assuming you meant MWh, as in "mega-watt-hours", of course)
Don't get carried away with LINPACK flops/cost! (Score:5, Informative)
"Another example is PNNL's 1936 processor Itanium2 cluster: 3.5Tflops less performance than System X, for $25 million"
What is not captured by the LINPACK scores is that PNNL's machine will absolutely spank the BigMac cluster at what the PNNL machine is intended for: running computational chemistry codes such as NWChem. A lot of the cash for the PNNL machine went into large memories and fast I/O that simply does not show up in the LINPACK benchmark. Furthermore, there are a lot of very high-profile scientific publications that have come out of the computational chemistry abilities of the PNNL machine. That's something else extremely important that doesn't show up in the rankings.
There are a lot of similar examples, but the PNNL one is one that I know something about, so I chose it. Basically, I'm saying to not read too much into those cost comparisons. It really is comparing Apples to oranges... er, HPs in this case. =)
Re:Pizza arguments (Score:2)
Re:Pizza arguments (Score:3, Interesting)
-Dan
Re:Pizza arguments (Score:5, Informative)
Power architecture does well (Score:5, Insightful)
Re:Power architecture does well (Score:2)
Re:Power architecture does well (Score:5, Informative)
Re:Power architecture does well (Score:5, Informative)
Re:Power architecture does well (Score:4, Interesting)
That's because x86 is a horrible architecture. On top of that, x86 instructions are translated into microcode before they're executed, so you end up with an unknown (maybe you could ask the folks at Intergraph about it) architecture emulating a crappy architecture in hardware. Better architectures exist (ARM, MIPS, POWER, 68000, PA-RISC, toy architectures used in introductory computer architecture classes, everything else), but Intel won out in the marketplace. You can still get better chips, but you pay more and have less support.
That's why you'd be better off investing in AMD over Intel. AMD hit upon what Intel should've done years ago. The x86-64, for those who don't know, supports x86 binaries as well as its own new architecture. Think of it like an x86 chip with the underlying hardware exposed. If Intel had exposed the hardware that x86 instructions get translated to, they'd have had a clear upgrade path instead of having to dork off x86 out of the blue. AMD embraced and extended x86, and marginalized its future without doing any actual damage to it or x86 users. It's flat out genius.
In the meantime, almost anything performs better than x86, and with less power consumption. It makes those mini-ITX boards look like jokes, because instead of engineering a low powered MIPS board/processor, the VIA folks did another x86. It may have been good from a business point, but it's horrible from an engineering standpoint, and that sums up Intel and x86 fairly nicely.
Who has coffee? (Score:5, Insightful)
Re:Who has coffee? (Score:2)
super computations? (Score:4, Interesting)
Re:super computations? (Score:5, Funny)
Re:funny or funny? (Score:3, Interesting)
how much? (Score:2)
The Matrix has you, Neo.
;)
It's still running (Score:5, Funny)
Well, I've noticed a vew glitches (disappearing keys, poor AI in girlfriends, crazy presidents in some countries, etc.), but I'd say most of the Earth has been running reasonably well.
Sla.... (Score:2)
Maybe they could run the top500 website on it, using up some of those spare cpu cycles...
Re:super computations? (Score:3, Funny)
The closest they got was 41.9998645234, which is pretty good in my book.
Re:super computations? (Score:2)
Anyway, that just clarifies the competition. If others have spent more FLOPs as the "superest", what have they done with their time?
VA Tech Supercomputer (Score:4, Informative)
Re:VA Tech Supercomputer (Score:2)
Re:VA Tech Supercomputer (Score:5, Informative)
Re:VA Tech Supercomputer (Score:2, Interesting)
it was housed in the CRC about 1 mile off campus in those days. Probably freed up the room for the cluster when they decomissioned the old 3090 behemoth.
Erm ... (Score:3, Interesting)
I can assume it was put to some sort of use. But I honestly get the feeling it was more to have fun, and look cool (which means more bling bling from sponsors, alumni, etc)
Sunny Dubey
Re:Erm ... (Score:2)
Re:Erm ... (Score:3, Funny)
hehehe, IBM knows how to play (Score:4, Funny)
Funny MIPS (Score:4, Funny)
On any one of those systems, you could emulate a Beowulf cluster of this one, and still have time to play Thermonuclear War.
cluster operating system (Score:2, Interesting)
Only with Mac OS X can you get the combination of commercial software (such as Microsoft Office and Adobe Photoshop), user friendliness, no known viruses, best available security, and stability/scalability suitable for world-class superclusters.
Re:cluster operating system (Score:2)
Re:cluster operating system (Score:2)
Earth Simulator has liberal bias (Score:5, Funny)
Clearly it suffers from liberal bias.
CPU benchmarks (Score:4, Informative)
AMD is beating the crap out of Intel.
Re:CPU benchmarks (Score:2, Informative)
AMD Athlon (TM) 64 FX-55 1 core, 1 chip, 1 core/chip 1750 1854
Here is the best intel:
3.4 GHz, Pentium 4 Proce 1 core, 1 chip, 1 core/chip with HT Technology enabled 1667 1705
Here are the best specfp numbers for AMD:
AMD Athlon (TM) 64 FX-55 1 core, 1 chip, 1 core/chip 1741 1782
Which just edges out the best Pentium
3.6GHz Xeon) 1 core, 1 chip, 1 core/chip 1700 1721
But gets b
Re:CPU benchmarks (Score:4, Informative)
Why does slashdot keep posting these stories? (Score:4, Funny)
Fair to compare cost? (Score:2)
I can't get the cost comparison link to work for one reason or another. But I was wondering if they were comparing current day cost, or the cost when the machine was built?
I mean, the cost of processors has gone down significantly in the past several months, so a machine built a year ago with the same speed processors as a machine built today would cost much more.
Not to mention that some of the machines on the list are most likely second (or greater) renditions of an earlier super computer, only with mor
What is with the Apple fan-boyism? (Score:3, Interesting)
Now Apple markets good computers. Tend to be on the expensive side, but they are usually high quality.
The Power970 is decent enough in itself. The opteron is more powerfull, but is also much more energy hungry. The Intel Itanium is nice but it's very expensive. etc etc
But what is this worship of Apple? It makes no sense.
Story 1: Earth simulator.. blah blah blah., but Mac cluster!
Story 2: SGI supercluster.... blah blahblah, But Mac cluster!
Story 3: Blue Gene cluster, 65000+ cpus... blah blah blah, but Mac cluster!
Realy? Who gives a fvck about the 7th place computer, and who gives a damn about cost analysis at this point? What about the Top5?
Did you know that Blue Gene is PowerPC?
Did you know that Linux now runs the majority of top super computers...
Did you know that Blue Gene proccessors only run a 700mhz??!!!
Did you know that #4 is 3564 Power970's running at 2.2 ghz? And that beats out 4000+ Intanium2's running at 1.7ghz?
This is a Geek site.. what about OSes?
By ranking:
1. Linux, 2. Linux, 3. Unix, 4. Linux, 5. Linux, 6. Unix, 7. OS X, 8. Linux, 9. Unix, 10. Linux (most powerfull x86 btw), 11. Unix, 12. Unix, 13. Linux, 14. ?, 15. Linux, 16. Linux, 17. Linux, 18. Linux, 19. Linux, 20. Unix.
Were is the most powerfull Windows computer? Well there is one cluster that is probably still on the top500. I dare you to find it, though. It's probably around #200 or #300, which is stil freaking fast.
Ok, So the big Mac is still #7. That's great, but there are 6 wonderfull computers that have all sorts of great technology that your completely ignoring because Apple wets your pants.
Did you know that Blue Gene will eventually have over 65,000 proccessors??
Re:What is with the Apple fan-boyism? (Score:5, Funny)
Fine, I'm willing to talk about number 6. [apple.com] ;-)
Re:What is with the Apple fan-boyism? (Score:5, Insightful)
Apple worship because it's a smack in the face to those who still continue to bash Apple for reasons that no longer exist.
"OS 9 sucks!"
"We're on OS X now, and it's unix-like"
"oh...um...Well one button~"
"And yet I'm still more productive on it than my Windows box"
"Well um, I want linu-"
"You can install that too."
"Well, they're slow and-"
"I suggest you actually try using one before saying that."
"They're overpric-"
"Really? I didn't think $900 was that expensive for a mid-range machine."
We do the Apple worship thing just to fustrate the anti-mac crowd even more.
Re:What i really like is (Score:4, Funny)
Not on the list (Score:3, Funny)
Apparently, the top 500 list is not actually hosted on one of the top 500 machines.
I'm still hanging in (Score:4, Funny)
Software (Score:2, Interesting)
Apple has created software development packages specifically designed for their G5's with optimized code for the 64bit architecture such as complex math functions.
So not only is Apple providing a cheaper and power efficient system for academic institutes, they make it easier for professors and assistants to create the softwar
#4 is also academic (Score:3, Informative)
I work at UPC and there has been a lot of hype here for machine #4, which is (or is going to be) a >4500 PPC970s machine running linux (nice work, ibm). I disagree with the claim that the Virgina Tech cluster is the first academic supercomputer. As far as I'm concerned the Technical University of Catalonia (UPC) is also an academic institution.
Anyway. we now got europe's fastest supercomputer. That's what matters. ha!
Fan boys read only what they want to see! (Score:2, Insightful)
Re:Fan boys read only what they want to see! (Score:4, Insightful)
So we should say "It's really no achievement to have a supercomputer in a pricerange available to institutions other than military. The fact that they use Apple's G5 and OS X, an almost out of the box solution is totally irrelevant. If you like you can build your own courtesy of Virginia University, but who would want a supercomputer that's cheaper than the other twenty first contenders in the list of supercomputers. Remember, they're Apple, so they're crap. And expensive, whatever the calculations say. They must be. They're Apple. I repeat, they're Apple. Crap. Be realistic, don't be a fan-boy."
Fan-boy indeed.
A supercomputer application (Score:3, Insightful)
Can I vote for a supercomputer thread so that I can elect to have it not displayed in my preference? I wouldn't want to miss out all the other tasty hardware goodness. I don't mind news about new Supercomputer technology, but whoever holds the most teraflops at a certain point in time is not of interest.
http://apple.slashdot.org/article.pl?sid=04/11/09/ 0126220&tid=137&tid=126&tid=181&tid=1
0 6/2239245&tid=136&tid=137&tid=14
4 5&tid=137&tid=139&tid=1
2 7/0147206&tid=137&tid=139&tid=14&tid=106
/ 0636230&tid=137&tid=3
2 0/1727255&tid=137&tid=136&tid=14
November 9th, 2004
http://science.slashdot.org/article.pl?sid=04/11/
November 7th, 2004
http://slashdot.org/article.pl?sid=04/11/03/16142
November 3rd, 2004
http://science.slashdot.org/article.pl?sid=04/10/
October 26th, 2004
http://apple.slashdot.org/article.pl?sid=04/10/26
October 26th, 2004
http://science.slashdot.org/article.pl?sid=04/10/
October 20th, 2004
Virginia fastest academic? (Score:3, Informative)
Beg to differ: #4 is about 5 mins from home (by bus), in the northern campus of universitat politècnica de catalunya. And, yes, part of the institution, not some loaned space or something. Mind you, one wishes Spanish Universities involved their students a tenth as much. S-2.
Why is nobody copying the Big Mac? (Score:2)
Fan-boy complex (Score:2)
So what I propose is that even when Apple related posts are few and far between, we should always add one of the following conclusions:
1) Apple is still expensive, even if they're proven cheaper than the alternative.
2) Apple is still slow compared to anything my cousin Ned can build in his back-yard.
3) Jobs is an ass-hole.
4) You know, any objective computer user has to acknowledge the simple truth: Apple sux and anyway is dead, gone an
Re:EARTH TO MAC ZEALOTS: (Score:3, Interesting)
Re:EARTH TO MAC ZEALOTS: (Score:4, Insightful)
Re:EARTH TO MAC ZEALOTS: (Score:2)
Re:EARTH TO MAC ZEALOTS: (Score:2, Funny)
Re:EARTH TO MAC ZEALOTS: (Score:3, Informative)
Christ, clusters are not the end all and be all of high performance computing systems.
I guess I don't get the arugment because many of the other entries are clusters and not not single image either. Of course, given OS and architecture differences, all the supercomputers may perform differently in real world applications than
Re:Just imagine (Score:2)
Re:Just imagine (Score:2, Funny)
So true, we should use a complex, awe-inspiring game, which will push the limits of any machine. I suggest Nethack.
Re:Just imagine (Score:2)
Re:Excellent comparison? (Score:2)
Or alternativly install the EditCSS (or webdevbar) plugin and just add a rule of * {background-color: white;color:black;}
Re:And we thought Macs were expensive (Score:4, Funny)
Re:Are prices on that comparison adjusted for date (Score:3, Insightful)
the floating point operations thing (Altivec) (Score:3)
While there may be a bias (personally I am sick of the way EVERY FreeBSD, OpenBSD, and NetBSD post is being mishandled - it's unprofessional and destructive, and irresponsible given
Re:These are the ones you're allowed to know about (Score:3, Insightful)
Only if we're counting fictional computers thought up by conspiracy theorists.
So, no.
"Big Brother uses in the very near future if not already."
You mean filming some no-marks around the clock in the name of entertainment? Or the fairly silly idea that Europe is spearheading an effort to slap everyone into a database. Have you ever seen the EU decide anything? Do you know that the EC meets in Brussels, not the UN?