The Father of Multi-Core Chips Talks Shop 90
pacopico writes "Stanford professor Kunle Olukotun designed the first mainstream multi-core chip, crafting what would become Sun Microsystems's Niagra product. Now, he's heading up Stanford's Pervasive Parallelism Lab where researchers are looking at 100s of core systems that might power robots, 3-D virtual worlds and insanely big server applications. The Register just interviewed Olukotun about this work and the future of multi-core chips. Weird and interesting stuff."
Re:The Future Is Non-Algorithmic (Score:4, Interesting)
To simplify: Dataflow. It's been too many years, but I recall that DataFlow was a company name. Their lack of commercial success was based on the combination of being way ahead of their time.
The recent advent of multiple on-die asynchronous units ("cores") is leading to a resurgence of interest in the dataflow model.
Anyone who has implemented networked event-driven functionality has already started down the path of dataflow model of computation, though obviously it's not fine-grained. The "non-algorithmic model" looks like a fine-grained implementation of a normal network application. (I agree with a downthread post that claims that current and classical Java-based server applications are already there, accepting the idea that event-driven multithreading applications are essentially coarse-grained dataflow applications.) And when the research gets going hot and heavy, I'll wager that the research will end up focusing on organizing the connectivity model.
As far as I am concerned, one place to look for multicore models to shine would be in spreadsheets and similar applications where there is already a well-defined pattern of interdependency among computational units (which in this case would be the spreadsheet cells). I also think that database rows (or row groupings) would be naturals for dataflow computing.
An efficient dataflow system would be the most KICK-ASS life computation engine! :-) (Now you know how old I am...)
Re:The Future Is Non-Algorithmic (Score:1, Interesting)
An efficient dataflow system would be the most KICK-ASS life computation engine! :-) (Now you know how old I am...)
Actually National Instruments has had a graphical data-flow dev tool for years. Their latest incarnation even has support for multicore processors (i.e., multithreading). However, what I'm proposing is not dataflow but signal flow as in logic circuits, vhdl, spiking neural networks, cellular automata and other inherently and implicitly parallel systems.
Well (Score:1, Interesting)
IAASE and if I recall correctly, Donald Knuth said that the all the advent of multi-core systems showed was that chip developers had run out of ideas and from what I can see happening in the industry today, he was right.
multi-core = multi-kruft
Re:Multi-core chips will be constrained by (Score:3, Interesting)
Actually, Sun's Niagara has that "problem". The way they solved it is to place Gbit networking close to the cores. There are also multiple DDR-2 memory buses and (I think) PCI-E lanes to feed the processor's prodigious need for memory bandwidth.
The comments to the Register article include a comment about the Transputer. (In case it's not familiar history, the transputer was a really slick idea that went nowhere... 4 high bandwidth connections, one for each neighbor CPU, with onboard memory. I recall that they were programmed in "Occam", a dataflow-oriented language.)
I believe that large-count multi-core systems will remain niche solutions until dataflow "locality", "discovery", and unification of control and data become well understood in a theoretical framework. The niches are nice places to be, though. A high quality game is "merely" a simulation of some virtual reality and simulations are perfectly matched to high-count multi-core systems.
The idea of unifying control and data is no new invention, and anyone trying to patent it should be shot. It's just an ordinary spreadsheet cell. Or possibly a neuron.
Re:Illogical, Donald Knuth is smarter than that. (Score:3, Interesting)
I'll accept the argument that the single-threaded model is (temporarily) being preserved in current systems. That said, I believe that there is a natural progression toward multithreaded computing as the technologies become more pervasive.
What do you think of such things as SQL and spreadsheets already starting down a road of declarative style of "programming", which would implicitly allow the engines to make their own decisions about how to run multi-threaded?
I had good experience with a quad-phenom running a classical web application recently: The system used all 4 cores very effectively without our needing to make a single adjustment to our extremely simple application.
To me, it appears that we (the developers, theoreticians as well as practical implementers) are already naturally moving to use the resources that the hardware developers are providing. However, I really don't see multi-core systems as "cruft"y as your first comment claims.
Re:The Future Is Non-Algorithmic (Score:3, Interesting)
Oh yes, that is why we have (or are currently developing) purely functional programming languages that can often mimic this model quite nicely, and efficient compilers capable of compiling the code into (potentially) whatever paralellism model you are using. Threads should ideally be just a means of implementing paralellism for such languages, of for parallel computing frameworks. Today, you are probably not supposed to write threaded code by hand in most cases. Once you have a reasonable compiler (latest versions of GHC look really promising in this respect), you can stop worrying about how the damned thing works (in most cases) and just write your code.
Just one question - as "algorithm" is essentially just a "recipe" for computing certain results from its input. How a supposedly universal computing machine could work without such recipes to change its mode of operations? No matter whether the machine is a Turing machine, a lambda calculus abstract machine or just a reconfigurable electric circuit.
Re:Multi-core chips will be constrained by (Score:3, Interesting)
the transputer was a really slick idea that went nowhere... 4 high bandwidth connections, one for each neighbor CPU, with onboard memory. I recall that they were programmed in "Occam", a dataflow-oriented language.)
Mainly because CPU clock speed and data bus speed were doubling every year. By the time an accelerator card manufacturer had a card out for six months, Intel had already ensure that the CPU was faster and so the accelerator card rapidly became a de-accelerator card. If you look at the advert pages of old Byte magazines, you will see all sorts of accelerator card that tried to offload work away from the CPU (i860's, TMS34020's, quad transputers, video cards with built in networking). All of these were squelched one way or another (Intel created a custom video bus, added the 80x87 FPU, then put it on-core, created the Xeon with a built in i860, added a larger cache, multi-stage pipelines, superscalar architecture, doubled register size from 16-bits to 32-bits, and so on...)
Also, most applications that benefit from parallel processsing required data to be stored in three-dimensional grids, which needed both floating-point acceleration and six and more memory accesses (up and down as well as north, south, east and west). Both Parallel C and Fortran were available for the transputer, but the problem was cost - a transputer accelerator board cost well over 500 pounds just for a four transputer board.
It's a shame the transputer never made it, but in the PC world a manufacturer needed to have a new product out every six months to keep up against Intel.
Re:Multi-core chips will be constrained by (Score:3, Interesting)
The memory controller is onchip of course, and it has a bandwidth of about 50-60GB/s I believe
Which is in fact, around the amount of memory bandwidth Niagara systems have [fujitsu.com], with 6 memory controllers per socket.
Re:Illogical, Donald Knuth is smarter than that. (Score:1, Interesting)
No. Sheesh. Supercomputers almost all use a message passing model in practice. Messages between concurrent processes, not threads and mutexes. They may have shared memory, but that means message passing is zero-copy, not that that they're programmed with anything other than MPI.
The grownups have been playing with concurrent systems for decades now.
There's a lot of wheel-reinvention going on as the kiddie home computers get multiple processors. Eventually they'll realise what we've known for years - the ONE concurrent model that humans program well is message passing between concurrent processes.
Re:Multi-core chips will be constrained by (Score:1, Interesting)
I'm not sure about Niagara, but it should be noted that for GPUs to obtain anything even close to the advertised bandwidth requires very specific access patterns. I'm not familiar with ATI's GPUs, but in the case of nVidia GPUs, "proper" access requires ensuring that the memory address of the "first" process in a warp (group of processes) meets certain criterion and that the addresses accessed in parallel with this by other processes in the warp are contiguous with a stride of 4, 8, or 16 bytes. Doing anything else results in the "86.4GB/s" bandwidth on a G80 being reduced significantly (more than 50%).
Re:First mainstream multicore? (Score:4, Interesting)
Incidentally, I worked with Transputers,and the concept died for many reasons
1) The comms channel was a wierd, proprietry protocol, and not HDLC - completely fatal
2) In the event of an error, the entire Transputer netowork locked up - competely fatal
3) Mrs Thatcher eventually agreed to fund the project with $50,000,000 the same day that United Technology (can you say 6502, or was it Z80) cancelled a project saying "in the world of Microprocessors $50,000,000 is nothing". - Two fatal errors here (a) expecting the UK government to fund anything reasonably sensible, and (b) Making it clear that the project is insufficiently funded to survive
4) The project was taken over by the French - whose previous achievements in both hardware and software are [white space here]. 5) Inmos, who made it, (a) tried to force people to use a new language, at a time when there was a new language every month, (b) took two years to discover that the target market wanted C, and (c) never discovered the appropriate language was Algol68.
In short, the company was run by a clever but narrow minded geek, who failed to take advice from others in the industry (including other narrow minded geeks, like me, etc).
Re:The Future Is Non-Algorithmic (Score:3, Interesting)
Hence the downmod. You just do not learn.
Really? I take it you don't use the internet or a computer then? Fail.
I just do not see this: considering the amount of guts you've got, where is the COSA toolchain? The COSA OS, with the COSA web-browser, with the COSA frigging interactive editor? Why should I believe anything you say? You HAVE DONE NOTHING.
Who's ass am I kissing? Turing? Hawking? Zeno? Have you heard of proof by intimidation? It is not effective. I mean, there are discrete topologies with "infinite divisibility" which are (consequentially) non-continuous, take
Where the frig is the contradiction?! I WANT TO SEE; PLEASE SHOW ME!
I am not here to make an enemy or collect blood-karma from you. I am here to make a point. I am here because I see fallow potential in you.
Re:The Future Is Non-Algorithmic (Score:2, Interesting)
So sir, what does 0.999... = ?
I guess you're the kind that would say this isn't 1.