Student and Professor Build Budget Supercomputer

Luke writes "This past winter Calvin College professor Joel Adams and then Calvin senior Tim Brom built Microwulf, a portable supercomputer with 26.25 gigaflops peak performance, that cost less than $2,500 to construct, becoming the most cost-efficient supercomputer anywhere that Adams knows of. "It's small enough to check on an airplane or fit next to a desk," said Brom. Instead of a bunch of researchers having to share a single Beowulf cluster supercomputer, now each researcher can have their own."

Imagine...

A beowulf cluster full of these!

(Okay, now back to responsible mature posting)

Re:Imagine...

(Okay, now back to responsible mature posting)

You forgot to provide a link to that...

Re:Imagine...

In this case, I think it's a somewhat serious idea. This design has only four nodes, so connecting several in a modular fashion might make sense, and retain some of the advantages in portability and cost. You could move the individual Microwulfs around, but bring them together for really big problems. Think of it as a LAN party for scientists.

Re:Newbie translation please?

So 1 Hz equals 1 FlOp? And a 3.2 GHz CPU can do 3.2 gigaflops, right?

No, one hertz is one cycle of the processor.

Can they execute multiple FlOps per tick then?

Yes. A single processor will perform several steps in one cycle. Typically, the steps are something like:

1. fetch (an instruction from memory)
2. decode the instruction
3. execute the instruction
4. access (some memory location)
5. writeback (some values calculated during this cycle)

In reality, this cycle is usually more complex and processors are designed to predict certain events in order to pack more into a single processor cycle. On top of this, note that the processors used in this machine are all dual-core processors. This means that instead of the 4 processors listed on the hardware manifest, it's really more like 8 processors (well, not quite).

And do we care that these will bottleneck at the rather limited bus (even forgetting about the switch).

No.

Hey, those computer engineering classes I was forced to take as a part of my CS major have actually proven useful! Oh wait, this is Slashdot.

Re:

A processor normally takes 2-3 clock pulses to perform any instruction

A modern processor may in fact take a dozen or more clock cycles to finish a single instruction. However, by utilizing pipelining, reordering and multiple execution units, a single co

Re:Imagine...

(Okay, now back to responsible mature posting)

No, stay with us on Slashdot!

not so impressive...

It's just four motherboards sitting in a single frame. connected by an ethernet switch.

True supercomputing machines (sun, ibm) have a little bit better interconnectivity between the components than a mere 1Gb/s line. This can serve its purpose though, VASP will run wonderfully on it. GAMESS probably as well.

B.

Re:

Mobs like Verari were selling something similar a while ago - not cheap though and I can't see it on their web page now. What is nice now from other places is things like 2 x 8 core machines in 1U (maxtron and probably a few others). The relatively small

Re:

Others have pointed out that this is useful for tasks where the interconnect speed doesn't matter. I'll point out that the first "node" only costs $765, and the next seven are $564 each (then you need a bigger switch). Of course, the 8-way version won't

Which is fine

But you aren't really a supercomputer at that point, you're a cluster. These days the line is more blurred than in the past but more or less the difference is interconnect speed. In a real supercomputer, there are very high speed interconnects, so you can run things that heavily rely on one part communicating with another, like particle simulations. That's why the US Department of Energy buys so many, rather than clusters. They do things like weather simulation and simulation of nuclear weapons, where every node as to be able to talk to every other node with essentially no penalty.

Now if you have a job that doesn't use a lot of inter-node communication, like say 3D rendering, then a cluster is a better answer. Normal hardware with Ethernet interconnects. Works great and is cheap since you can use commodity parts. But don't confuse that cluster with a real super computer, you throw one of those intense inter node problem at it, it'll fall over because the interconnects are too slow.

Unfortunately these days people really blur the distinction. You'll see systems on the top 500 list that are really questionable. It'll be commodity hardware connected with something like infiniband. Ok, great, that is faster (both more bandwidth and less latency) than Ethernet, but it still isn't necessairily up to what you'd get from a real supercomputer.

However in the case of this deal, no, not a super computer. It's a small cluster and they are just calling it a super computer as marketing, effectively.

Re:not so impressive...

http://www.calvin.edu/~adams/research/microwulf/bu dget/ [calvin.edu]

          AMD Athlon 64 X2 3800+ AM2 CPU x 4

It's two clicks from the summary.

Slack++

**Lets chop that price down...the newegg,com way**

Motherboard: MSI K9N6PGM-F MicroATX [newegg.com] $62.99 * 4 = $251.96

CPU: AMD Athlon 64 X2 3800+ AM2 CPU [newegg.com] $67.50 * 4 = $270

Main Memory: Kingston DDR2-667 1GByte RAM [newegg.com] $48.49 * 8 + $4.99sh = $392.91

Power Supply: (can't beat price): $76.00

Network adapter (node to switch): (cant beat their price) $164.00

Network adapter (switch to node): (cant beat their price) $15

Switch: Trendware TEG-S80TXE 8-port Gigabit Ethernet Switch [newegg.com] $46.99+$7.04sh = $54.03

Hard drive: Seagate 7200 250GB SATA hard drive [newegg.com] $69.99

DVD/CD drive: (can't beat their price): $19

Cooling: (can't beat their price): $32

Fan protective grills: (can't beat their price): $10

KVM: (can't beat their price): $50 Grand total (incl. 15 in hardware): 1416.89 $1000 saved by using Newegg!

Re:**Lets chop that price down...the newegg,com wa

Discovering that you can build an even more cost effective supercomputer than these guys: priceless

Re:

>Are Intel processors "super computers" now or something?
No, processors ARE NOT supercomputers (actually, the are not computers at all). But if you put enough of them together in the appropriate way, they BECOME a super computer.
Super computers are

Re:How does it compare to a PS3?

Sorry for replying to myself, but I found an interesting paper [netlib.org] about the subject. Seems that a PS3 should have Rpeak of 14 Gflop/s with double precision floating point operations. Sounds to me that with a proper clustering solution a four-node PS3 cluster would be significantly faster than Microwulf. And it would probably be a smaller, too :)

Re:How does it compare to a PS3?

Good paper - it also says that by using mixed precision (iterated 32-bit math for rough matrix factorization then fine-tuning the precision in 64-bit) the double-precision matrix performance is up to 155 Gflops.

Re:

IBM have already done just that. For example they have a demo [youtube.com] of cluster of PS3s providing a real time ray tracing scene.

I expect the design is very well suited to clustering. The PPUs handle all the data dispatching & balancing with the SPUs left to

On an airplane?

It's small enough to check on an airplane

With security concerns nowadays, it's the amount of cables coming out of it that worries an airline, not the size or weight of this machine.

Re:

You are overestimating the amount of EM noise emitted by a motherboard outside the case. Very few computer components are noisy. The ones that are like some modems, wireless cards, etc feature additional individual shielding.

Check in on an airplane ?

It looks rather fragile, quite like the iRack (http://www.youtube.com/watch?v=xcjLEwZqcQI), and I dont think it would survive checking in on an airplane given how some suitcases looks like at baggage claim.

Cool achievement nevertheless.

Re:Check in on an airplane ?

Checking something called an iRack onto a plane is just asking for a full cavity body search and possibly a nice orange jumpsuit.

heat buildup issues?

And it looks like they'll be running into heat buildup issues. An enclosure ventilated by one or two desktop fans would have provided sufficient cooling. Mere convection (outside of the tiny on-board fans) is often not enough. The Sun E450's were well ventilated machines, with a clear air path going from the front to the back. The temperature monitors (ambient, cpu (x4), PSU (x3)) were useful as well. One was used for a long time at Stack (www.stack.nl) as a room temperature monitor.

B.

Great!

Now Microsoft have their next development target for Office.
 

But

is it powerful enough to run Windows Vista?

Re:But

Only if you don't plan on playing MP3s.

Lame.

I am impressed with how amazingly lame this story is. It should have been entitled, "College Senior and Professor discover Ethernet, MicroATX, and PXE boot. Funding dried up before paying for cases. News at 3 am because we can't find anything else to report."

Honestly, our whole research lab is filled with PXE booting MicroATX computers connected via ethernet. And I guarantee that four "nodes", aka Linux PCs, are cheaper than $2500. Whoop-de-freaking-do.

Re:Lame.

And I guarantee that four "nodes", aka Linux PCs, are cheaper than $2500.

Indeed. After I saw the component prices I was left dumbfounded. I mean, AMD Athlon 64 X2 3800+ processors at 165 dollars a pop? A kingston 1GB DDR-667 stick of RAM at 124 dollars? Are they on drugs? I mean, I've just bought an Athlon 64 X2 4000+ EE for 68euros (the 3800+ was selling for 59 euros) and each kingston 1GB DDR-800 stick for 46 euros. Where did all the rest of the money went?

Re:Lame.

They probably padded the budget and spent the remaining money on hookers and blow. That would explain how they got delusions of grandeur and thought they built something new and innovative when all they did was link 4 motherboards via cheap gig-ethernet.

This story is literally a 'nothing to see here, move along' one.

the google way

This seems pretty similar to the way google builds their racks, with just mb's and no cabinets. What would have been really cool was if someone made som e kind of network driver for a pci express slot, with them being able to use external cables, is it possible to use a dedicated pci express slot as a interface to another computer, skipping the network bottleneck ?

Re:

Myself and some other students (back when I was in college) played with doing this via PCI SCSI cards, it worked to a point but wasn't quite the same as all you were really doing is providing SCSI access to each systems HDDs. Still it would have allowed q

Not to rain on their parade, but...

...this is *hardly* a supercomputer. This is 152.57 times slower than entry number 500 on the Top 500 List [top500.org]. There isn't a nice neat definition of what a supercomputer is anymore, but "capable of running Beowulf" isn't it. Leaving aside the more custom machines that the company I work for (and a few others) build, there are plenty of Linux clusters that *do* qualify. The fastest one seems to be number 8 on the current Top 500 list (a Dell Infiniband cluster at NCSA).

Actually... Microwulf might well be revolutionary

One of the problems with supercomputers is that there aren't really very many of them, because of the size and cost. It means that the tools you use to run your supercomputing applications are similarly unusual. The skills to use and develop on parallel systems are then equally scarce. Access to a supercomputer isn't exactly common.

Microwulf could make all of the above common. For the price of a high spec PC. The commodity nature of it could bring super computing and super computing applications to the masses.

Then you can scale your application from microwulf to miniwulf to superwulf with little more effort than installing it on the bigger machine.

Course, they'd have to produce a commodity pre-built system.
 

Re:Actually... Microwulf might well be revolutiona

One of the problems with supercomputers is that there aren't really very many of them, because of the size and cost. It means that the tools you use to run your supercomputing applications are similarly unusual. The skills to use and develop on parallel systems are then equally scarce. Access to a supercomputer isn't exactly common.

Revolutionary? Everything old is new again...

http://www.mini-itx.com/projects/cluster/ [mini-itx.com]
http://news.taborcommunications.com/msgget.jsp?mid =494184&xsl=story.xsl [taborcommunications.com] -- 8 way parallel cluster that fits on an airplane for under 3 grand
http://www-03.ibm.com/systems/bladecenter/ [ibm.com] -- a 7U chassis that holds 14 blades, and is a bit spendy, but not completely unreasonable for some situations
http://www.linuxjournal.com/article/8177 [linuxjournal.com] -- My personal favorite, this page talks about several small portable miniclusters that have been made over the last six or seven years...

Yes, 8 cores of Athlon64 is faster than 8 cores of low power VIA CPU's from several years ago, but the concept isn't revolutionary, and there isn't a lot of headline worthy engineering that goes into a project like this... I'm sure it's a very handy tool, and I'm not suggested it shouldn't have been built, or that it was entirely trivial to build, but in the end, it's just four ordinary motherboards and ethernet.

It's about the possibilities, not the technology

I see a few people making the expected "It's just four motherboards wired together with Gig E"-comments. While I won't object to that, I'd say this is not about a groundbreaking evolution in hardware, more a case of demonstrating what's possible today with COTS parts. Adding to that the compact packaging, and the ability to run off of a single power cord, it's a nice setup IMHO.

While it does not have the interconnect of "true HPC" hardware (a bit of a fleeting distinction, but bear with me) it'll surely be suitable for a lot of the simpler, yet still compute-intensive tasks out there ("simple" here meaning not needing a lot of intra-node communication).

On the flip side, it might fuel the "hell, I'll just build my own cluster"-mentality going around these days. I work in the HPC group at a university, running linux clusters, IBM "big iron" and a couple of small, old SGI installation, and we certainly see a bit of that going around. Problem is, sure, the hardware is cheap and affordable, but getting it to run in a stable and sensible manner without spending large amounts of time just keeping the thing together is a challenge, mainly due to the immature state of clustering software. As many researchers are not exactly keen on spending time solving problems outside their specific field, they're usually better off letting somebody else administer things, so they can just log on and run their stuff.

But for individuals and small groups of people who are computer savvy enough to handle it, things like these are definately a "good thing" (TM).

4 psus, isn't that a waste?

Sure, nothing beats off-the-shelf components... but powering 4 motherboards using 4 separate PSUs sounds like waste!

Look at this design: http://www.mini-itx.com/projects/cluster/ [mini-itx.com]. It uses DC-DC converters on each motherboards (mini-itx, so low power), a single 12V PSU and a UPS for regulation:

The DC-DC converters require a clean, well-regulated 12VDC source. I chose to use a heavy duty 60 ampere 12VDC switching power supply capable of delivering 60 amperes peak current which I ordered from an online electronics test equipment supplier. Since badly conditioned AC power is potentially damaging to expensive computing equipment, I use a 1 KVA UPS purchased at an office supply store to make sure the cluster can't be "bumped off" by power line glitches and droputs.

GigaFlops

Is 26 GigaFlops significant anymore? I hear that the PS3 can do 20-25 from Folding@Home people. And it is only about a 5th the price. But I hear so many different numbers that I can no longer make sense of them. Why do they bother comparing with DeepBlue, an over 10 yr old super computer? Can anyone with a PS3 can report what their PS3 with Yellow Dog Linux is doing? And what are the numbers for the latest desktop processors? Any recommendations on software to benchmark in flops for my own computers?

gigaflops?

gigaflops, schmigaglops.

this is /.

i thought performance was measured in fps?

Re:gigaflops?

Only true if your user ID contains 7 digits...

Lousy Latency Performance, Though

The cluster depends on gigE for the interconnect, which means data transfers are going to be slow, and have a high latency. He'd be better off spending a little more and using Infiniband equipment.

Beowulf = pain in the a**

Beowulf is a good idea for a very limited number of number crunching applications, or as a student learning tool for comp sci or related studies. Yeah, we built one of those a couple years ago, the professors ended picking up intel quad core machines that were faster (no effing network latency). Beowulf is gathering dust.

Oh, and try writing your own lam-mpi code sometime...

Re:

Lisa, I'd like to buy your cat!

Re:How is this interesting?

They just linked 4 motherboards together. My cat could do that.

Sure. But then your cat would have to moonlight as a mouser, run errands for the neighborhood dogs, and -- worst of all -- give up catnip; all in order to pay for the project.

I would not want to live in the same house as a sleep deprived cat going through catnip withdrawl.

Wussywulf?

I'm to lazy to run the numbers tonight to compare actual speeds but our dual CPU four-core Xeon (8 cores total) servers cost around $2500 each to build. Looking at their specs I doubt they could be doing much better and they require special clusterish prog

Re:How is this interesting?

Doubt it. You think you can hook up gigabit ethernet without at least five cats eh?

Re:How is this interesting?

They just linked 4 motherboards together. My cat could do that.

Would your cat be alive at the end of the process? We wouldn't be sure till we opened the case.

Re:What would you do with one?

This is kind of like the old joke about a dog chasing a car...what's it gonna do with the thing if it catches it.

I've thought several times about building a small cluster, just for the experience and the nerd factor. But I never do because I also get in to the issue of just what am I going to do with it once its finished, other than heat my workshop.

Re:What would you do with one?

Aren't these things like chick magnets?

Re:But..

Minesweeper under XP - Yes

Minesweeper under Vista - No

Re:

It might just about be powerful enough to run Vista though....

Re:Definition?

The basic definition of a supercomputer is a system which has top performance compared to other computer systems (within the top 500 or 100).

In the past, this could only be achieved by having custom CPU's to perform pipelining or parallel processing. Processors in the Cray supercomputers had extremely deep vector pipelines, which was good for three-dimensional simulations like CFD or computer animation. But other systems followed the parallel processing method. The Connection machine had 2^16 one bit processors which was good for encryption/decryption. Other systems used standard CPU's (Intel 80x86's, DEC Alpha's and M680x0's) connected together through a high-speed bus network.

The different types of systems could be defined according to how these processed instructions/data.

SISD - Single Instruction, Single Data - Early home computer
SIMD - Single Instruction, Multiple Data - Vector processors
MISD - Multiple Instruction, Single Data - Fault tolerant systems
MIMD - Multiple Instruction, Multiple Data - Parallel processing CPU's

Some systems had hardwared interconnect configurations - either a 2D square grid, a 3D square grid or torus network, or even star networks, while others had dynamic routing capability. Transputers only knew about the adjacent processors in the four compass directions (NESW).

But all of these techniques have been incorporated into mainstream CPU's now - you now have dual-core and quad-core CPU's that can be used by laptops.

Modern day methods are to make the systems super-scalar. Multi-core CPU's can be arranged side by side onto multi-CPU boards which in turn can be rack mounted into chassis which communicate through high-speed interconnect systems. There is no limit on the number of racks that can be used except space and money.