Intel Says to Prepare For "Thousands of Cores" 638
Impy the Impiuos Imp writes to tell us that in a recent statement Intel has revealed their plans for the future and it goes well beyond the traditional processor model. Suggesting developers start thinking about tens, hundreds, or even thousand or cores, it seems Intel is pushing for a massive evolution in the way processing is handled. "Now, however, Intel is increasingly 'discussing how to scale performance to core counts that we aren't yet shipping...Dozens, hundreds, and even thousands of cores are not unusual design points around which the conversations meander,' [Anwar Ghuloum, a principal engineer with Intel's Microprocessor Technology Lab] said. He says that the more radical programming path to tap into many processing cores 'presents the "opportunity" for a major refactoring of their code base, including changes in languages, libraries, and engineering methodologies and conventions they've adhered to for (often) most of the their software's existence.'"
Memory bandwidth? (Score:5, Interesting)
Disagreement about this trend (Score:5, Interesting)
At Supercomputing 2006, they had a wonderful panel [supercomputing.org] where they discussed the future of computing in general, and tried to predict what computers (especially Supercomputers) would look like in 2020. Tom Sterling made what I thought was one of the most insightful observations of the panel -- most of the code out there is sequential (or nearly so) and I/O bound. So your home user checking his email, running a web browser, etc is not going to benefit much from having all that compute power. (Gamers are obviously not included in this) Thus, he predicted, processors would max out at a "relatively" low number of cores - 64 was his prediction.
Ok.. so how do I do that? (Score:3, Interesting)
Re:Not Sure I'm Getting It (Score:4, Interesting)
IANACS, but if your program structure changes a bit, you can process the two different styles of instructions in different ways, such that when the data needed from or to some sequential group of tasks is needed it is already there, sort of like doing things 6 steps ahead of yourself when possible. I know that makes no sense on the face of it, but at the machine code basics of it, by parsing instructions this way, 5 or 6 operations from now you will need register X loaded with byte 121 from location xyz, so while this core plods through the next few instructions, core this.plus.one prefetches the data at memory location xyz to register X.... or something like that. That will break the serialization of the code. There are other techniques as well, and if written for multicore machines, the program machine code can be executed this way without interpretation by the machine/OS.
There are more than one type of CPU architectures, and principles of execution vary between them. Same for RISC CISC. I think it is likely that the smaller the instruction set for the CPU, the more likely that serialized tasks can be shared out among cores.
Re:Memory bandwidth? (Score:3, Interesting)
Memory would have to be completely redefined. Currently, you have one memory bank that is effectively accessed serially.
If you have 1000 cores that depend on the same data, you would have to have a way of multicasting the data to the cores, which could then select the data they want.
Basically, hardware and software architecture has to be completely redefined.
It is not impossible, though. Just look around. The universe computes in parallel all the time.
Declarative languages is the answer (Score:3, Interesting)
Re:Memory bandwidth? (Score:2, Interesting)
3D integration schemes, which IBM and Intel are both pursuing, help deal with this problem. As you noted, you can't put enough pins on a chip with traditional packaging to achieve a sufficient memory bandwidth. But with 3D integration, the memory chips are connected directly to the CPUs with "through-chip vias". You can have tens of thousands of these vias, and as a bonus, the distance to the memory is extremely short, so latency is reduced.
- Trevor -
[[self-construction] [blogspot.com]]: The autotherapeutic diary of a crazy geek's journey back to mental health
Lookahead/predictive branching is one option... (Score:5, Interesting)
I do see this move by Intel as a direct follow up to their plans to negate the processing advantages of today's video cards. Intel wants people running general purpose code to run it on their general purpose CPU's, not on their video cards using CUDA or the like. If the future of video game rendering is indeed ray-tracing (an embarrassingly parallel algorithm if ever there was one) then this move will also position Intel to compete directly with Nvidia for the raw processing power market.
One thing is for sure, there's a lot of coding to do. Very few programs currently make effective use of even 2 cores. Parallelization of code can be quite tricky, so hopefully tools will evolve that will make it easier for the typical code-monkey who's never written a parallel algorithm in his life.
Is that really a good idea? (Score:4, Interesting)
I'm all for newer, faster processors. Hell, I'm all for processors with lots of cores that can be used, but wouldn't completely redoing all of the software libraries and such that we've got used to cause a hell of a divide in developers?
Sure, if you only develop on an x86 platform, you're fine, but what if you want to write software for ARM or PPC? Processors that might not adopt the "thousands of cores" model?
Would it not be better to design a processor that can intelligently utilise single threads across multiple cores? (I know this isn't an easy task, but I don't see it being much harder than what Intel is proposing here).
Or is this some long-time plan by intel to try to lock people into their platforms even more?
Re:Not Sure I'm Getting It (Score:5, Interesting)
While prefetching data can be done using a single core, your post in this context gives me a cool idea.
Who needs branch prediction when you could just have 2 cores running a thread? Send each one executing instructions without a break in the pipeline and sync the wrong core to the correct one once you know the result. You'd still have to wait for results before any store operations, but you should probably know the branch result by then anyway.
Desperation? (Score:4, Interesting)
Honestly I wonder if Intel isn't looking at the expense of pushing per-core speed further and comparing it against the cost of just adding more cores. The unfortunately reality is that the many-core approach really doesn't fit the desktop use case very well. Sure, you could devote an entire core to each process, but the typical desktop user is only interested in the performance of the one progress in the foreground that's being interacted with.
It's also worth mentioning that some individual applications just aren't parallelizable to the extent that more than a couple of cores could be exercised for any significant portion of the application's run time.
Even 64 sounds optimistic (Score:3, Interesting)
I'd be surprised if a desktop PC ever really uses more than eight. Desktop software is sequential, as you said. It doesn't parallelize.
Games will be doing their physics, etc., on the graphics card by then. I don't know if the current fad for doing it on the GPU will go anywhere much but I can see graphics cards starting out this way then going to a separate on-board PPU once the APIs stabilize.
We might *have* 64 cores simply because the price difference between 8 and 64 is a couple of bucks, but they won't be used for much.
Intel is building an FPGA (Score:3, Interesting)
Dozens, hundreds, and even thousands of cores are not unusual design points
I don't think they mean cores like the regular x86 cores, I think they will put an FPGA on the same die together with the regular four/six cores.
Start! What do they mean, start? (Score:3, Interesting)
Re:Useless (Score:4, Interesting)
We got to keep reminding ourselves, the world we live in runs in parallel, why shouldn't our computers?
Re:Great... (Score:2, Interesting)
...then again, I can see it as an argument for vendors to finally --finally!-- stop counting "processors" as their license limit metric. And yes VMWare, I'm talking to you too when I say that.
Heat issues (Score:4, Interesting)
How are they going to cope with excessive heat and power consumption? How are they going to dissipate heat from a thousand cores?
When the processing power growth was fed by shrinking transistors, the heat stayed at manageable level (well, it gradually increased with packing more and more elements on die, but the function wasn't linear). Smaller circuits yielded less heat, despite being much more of them.
Now we're packing more and more chips into one package instead and shrinkage of transistors has significantly slowed down. So how are they going to pack those thousand cores into a small number of CPUs and manage power and heat output?
Re:Not Sure I'm Getting It (Score:4, Interesting)
Indeed, and any tasks that are flagged as repeating can be repeated on a separate core from cores executing serial instructions such that IPC allows things that happen serially to happen coincident with each other. A simple high level example is reading the configuration for your process that may change at any time during your process due to outside influences. Let the reading of that happen out of band on the processing as it is not part of the sequential string of instructions for executing your code. That way config data is always correct without your serially oriented code needing to stop to check anything other than say $window.size=? such that it's value is always updated by a different core.
Sorry if that is not a clear explanation. I just mean to say that since most of what we do is serially oriented, it's difficult to see how at the microscopic level of the code, it can be broken up to parallel tasks. A 16% decrease in processing time is significant. Building OS and compilers to optimize this would improve execution times greatly, just as threading does today. If threads are written correctly to work with multiple cores, it's possible to see significant time improvements there also.
All we need to do now... (Score:3, Interesting)
Re:Not Sure I'm Getting It (Score:2, Interesting)
So instead of a pipeline you have a tree.
Great, except for the fact that it's incredibly inefficient and the performance gain is negligible.
Quantum computers will (in theory) allow us to do both at once.
Re:Disagreement about this trend (Score:4, Interesting)
You've excluded gamers as if this had been some nearly extinct exotic species. Don't they contribute the most to PC hardware market growth and progress?
Re:Disagreement about this trend (Score:3, Interesting)
We've pretty much already hit a per-core speed limit, you really can't find many CPU's running over 3GHZ, whereas back in P4 days you'd see them all the way up to 3.8.
Architectures have changed and other stuff allow a current single core of a 3.2 to easily outperform the old 3.8's but then still why don't we see new 3.8's?
look what happened to ps3 (Score:4, Interesting)
So now we have a shit load of cores all we have to do is wait for the developers to put some multi-threading goodness in their apps.... or maybe not.
The PS3 was ment to be faster than any other system because of it's multi-cores cell architecture, but in a interview John Carmack said, "Although it's interesting that almost all of the PS3 launch titles hardly used any Cells at all."
http://www.gameinformer.com/News/Story/200708/N07.0803.1731.12214.htm [gameinformer.com]
Interesting challenges (Score:3, Interesting)
If people are writing their applications using threads, I dont see there should be a big problem with more cores. Basically, threads should be used where it is practical and makes sense and does not make programming that much more difficult, in fact it can make things eisier. Rather than some overly complicated reengineering, threads when properly used can lead to programs that are just as easy to understand. They can be used for a program that does many tasks, processing can usually be parallelised when you have different operations which do not depend on the output of each other. A list of instructions which depends on output of a previous instructions, which must run sequentially, of course cannot be threaded or paralellised. Obvious example of applications that can be threaded is a server, where you have a thread to process data from each socket, a program which scans multiple files, can have a thread for processing each file, etc.
Re:Disagreement about this trend (Score:5, Interesting)
Architectures have changed and other stuff allow a current single core of a 3.2 to easily outperform the old 3.8's but then still why don't we see new 3.8's?
The Pentium 4 is, well, it's scary. It actually has "drive" stages because it takes too long for signals to propagate between functional blocks of the processor. This is just wait time, for the signals to get where they're going.
The P4 needed a super-deep pipeline to hit those kinds of speeds as a result, and so the penalty for branch misprediction was too high.
What MAY bring us higher clock rates again, though, is processors with very high numbers of cores. You can make a processor broad, cheap, or fast, but not all three. Making the processors narrow and simple will allow them to run at high clock rates and making them highly parallel will make up for their lack of individual complexity. The benefit lies in single-tasking performance; one very non-parallelizable thread which doesn't even particularly benefit from superscalar processing could run much faster on an architecture like this than anything we have today, while more parallelizable tasks can still run faster than they do today in spite of the reduced per-core complexity due to the number of cores - if you can figure out how to do more parallelization. Of course, that is not impossible [slashdot.org].
it's not about cores (Score:3, Interesting)
If you put 1000 cores on a chip and plug it into a PC... very little would happen in terms of speedup.
What we need to know is the memory architecture. How is memory allocated to cores? How is data transferred? What are the relative costs of accesses? How are the caches handled?
Without that information, it's pointless to think about hundreds or thousands of cores. And I suspect even Intel doesn't know the answers yet. And there's a good chance that a company other than Intel will actually deliver the solution.
Re:It's already here. (Score:3, Interesting)
Last time I checked my computer had only one GPU core, which had a multitude of functional units. So does my CPU, in fact, but the GPU has more. Each CPU has its own "context" (the state of certain registers which store pointers, and the flags register.) More CPU cores means more contexts means less context switches means cheaper threads. Pretty simple!
CUDA &c are cool in that they offer you a way to use your video card for non-video applications when it is idle. However, their use is likely to be cyclical. It seems that we go through phases of having lots of custom hardware, and then getting cheap horsepower to throw at problems and thus having less custom hardware and doing more things in software, then having things flop back the other way. The PC was originally an expression of software-heavy use, but these days we have standard graphics processors and physics processors are even gaining some ground. Eventually the processors will take another big jump (having a thousand cores would qualify) and then everyone will want to do all this stuff on the CPU again, because a) it will be able to do it and b) you won't have to mess with two processors to get one job done.
Intel is dead... (Score:2, Interesting)
I have two comments:
1) Intel is doing this because they've run out of optimizations on single core systems, basically this is the only thing they have left to preserve their market. I expect this time next year you'll see ARM SoC's with 1Ghz+ processors that draw under 1W of power and sell for under $10. These cores will be changing the low end of the market. Intel won't be able to continue to charge $50 for a processor when you get the same or better perf for 1/5 the cost. The only real advantage Intel has is that Windows XP/Vista doesn't run on ARM.
2) The Processor Company Graveyard is filled with companies that have touted parallel processing solutions that were going to revolutionize the world of computing. Parallel processing is extremely difficult, and only fits a subset of computing needs, we will need fast single processor systems for a long time to come. I wish Intel luck on this endeavor, everyone else has failed miserably.
Re:Not Sure I'm Getting It (Score:3, Interesting)
Of course, the billion threads design doesn't solve the "how do n cores efficiently share x amount of cache" problem at all.
Re:Not Sure I'm Getting It (Score:4, Interesting)
That's what I'm curious about. Having 2 cores is enough for most consumers, one for the OS and background tasks and one for the application you're using. And that's overkill for most users.
Personally, I like to multi task and am really going to love when we get to the point where I can have the OS on one core and then have 1 core for each of my applications. But even that is limited to probably less than 10 cores.
Certain types of tasks just don't benefit from extra cores, and probably never will. Things which have to be done sequentially are just not going to see any improvement with extra cores. And other things like compiling software may or may not see much of an improvement depending upon the design of the source.
But really, it's mainly things like raytracing and servers with many parallel connections which are the most likely to benefit. And servers are still bound by bandwidth, probably well before they would be hitting the limit on multi cores anyways.
Cores? (Score:4, Interesting)
Can't they just make the existing ones go faster? Seriously, if I want to start architectures around 1000's of independent threads of execution, i'd start with communication speeds, not node count.
It's already easy to spawn thread armies that peg all IO channels. Where is all this "work" you can do without any IO?
I think Intel better starting thinking of "tens, hundreds or even thousands" of bus speed multipliers on their napkin drawings.
Aside from some heavy processing-dependent concepts (graphics, complex mathematical models, etc) the world need petabyte/sec connectivity, not instruction set munching.
Databases and implimentation-neutrality (Score:5, Interesting)
Databases provide a wonderful opportunity to apply multi-core processing. The nice thing about a (good) database is that queries describe what you want, not how to go about getting it. Thus, the database can potentially split the load up to many processes and the query writer (app) does not have to change a thing in his/her code. Whether a serial or parallel process carries it out is in theory out of the app developer's hair (although dealing with transaction management may sometimes come into play for certain uses.)
However, query languages may need to become more general-purpose in order to have our apps depend on them more, not just business data. For example, built-in graph (network) and tree traversal may need to be added and/or standardized in query languages. And, we made need to clean up the weak-points of SQL and create more dynamic DB's to better match dynamic languages and scripting.
Being a DB-head, I've discovered that a lot of processing can potentially be converted into DB queries. That way one is not writing explicit pointer-based linked lists etc., locking one into a difficult-to-parallel-ize implementation.
Relational engines used to be considered too bulky for many desktop applications. This is partly because they make processing go through a DB abstraction layer and thus are not using direct RAM pointers. However, the flip-side of this extra layer is that they are well-suited to parallelization.
I can see this being helpful (Score:2, Interesting)
This is an effective method as long as the processor is able to manage its load properly internally.
i.e if a processor has say 100 cores..with a combined processing capacity per unit time of Z and there are X threads and the processing capacity of 1 core per unit time is Y XY must always equal Z. The challenge is how do u manage Core loads within the CPU, if Intel can solve that uber multi can really take off.
wait...more cores? (Score:1, Interesting)
recently Intel came out with its Atom core. It is going into all sorts of things because it is smaller (1/10th the size of a normal core)and it draws a lot less power. It also has about half the power/cycles of some of the bigger cores. Yet this is more than enough for the normal user (/.ers excepted) All the normal people (my family) want to do is to surf the web, check email, watch movies, stuff that even Damn Small Linux can do. So it kind of begs the question: if a smaller and less powerful processor is selling so well, what kind of sales could we expect from something with a thousand or more cores?
Re:Not Sure I'm Getting It (Score:5, Interesting)
Now that 64-bit processors are so common, perhaps operating systems can spare some virtual address space for performance benefits.
The OPAL operating system [washington.edu] was a University of Washington research project from the 1990s. OPAL uses a single address space for all processes. Unlike Windows 3.1, OPAL still has memory protection and every process (or "protection domain") has its own pages. The benefit of sharing a single address space is that you don't need to flush the cache (because the virtual-to-physical address mapping do not change when you context switch). Also, pointers can be shared between processes because their addresses are globally unique.
Re:Not Sure I'm Getting It (Score:2, Interesting)
Re:It's all changing too fast (Score:5, Interesting)
For example, "CPU expensive, memory expensive, programmer cheap" is now "CPU cheap, memory cheap, programmer expensive" -- hence Java et al. (I am sometimes amazed when I casually allocate/free chunks of memory larger than all the combined memory of all the computers at my university - both in the labs and the administration/operational side - but what amazes me is that it doesn't amaze me!)
Actually some of the "old timers" may be a more comfortable with some issues of highly parallel programming than some of the "kids" (term used with respect, we were all kids once!) who have mostly had them masked from them by high level languages. Comparing "old timers" to "kids" doing enterprise server software, the kids seem much less likely to understand issues like memory coherence models of specific architectures, cache contention issues of specific implementations, etc.
Also, too often, the kids make assumptions about the source of performance/timing problems rather than gathering empirical evidence and acting on that evidence. This trait is particularly problematic because when dealing with concurrency and varying load conditions, intuition can be quite unreliable.
Really, it's not all that scary - the first paradigm shift is the hardest!
Re:kernel tasker responsibility (Score:3, Interesting)
I think that gcc should insert code to control memory leaks and process safety and the kernel should be in charge of tasking between cores.
Please limit this desire to languages like Java, Python, and Ruby. We don't need this in C. If you can't program without it, you shouldn't be programming in C.
Re:Not Sure I'm Getting It (Score:2, Interesting)
Holy crap. I just realised what you were saying -- use parallelism, vast parallelism for BRANCH PREDICTION.
That's not really how concurrency works at the moment :) it's at a much higher level at the moment, explicitly in the code itself - take matrix multiplication for instance, it's easy to see how that can be split up into multiple threads..
But calculation of every possible state 'n' states into the future, with 2^n CPU cores, that sounds like a good idea, sir! :) and is also not mutually exclusive with explicit multithreading (although each concurrent thread blows out the total number of states).
Re:Not Sure I'm Getting It (Score:5, Interesting)
yes, but if you have 1000 cores each with 64k of cache, then you start to run into problems with memory throughput when computing massively parallel data.
memory throughput has been the achilles heel of graphic processing for years now. and as we all know, splitting up a graphic screen into smaller segments is simple. so GPUs went massively parallel long before CPUS, in fact you will soon be able to get over 1000 stream processing units in a single desktop graphic card.
so, the real problem is memory technology, how can a single memory module consistently feed 1000 cores, for instance if you want to do real-time n-pass encoding of a hd video stream... while playing a FPS online, and running IM software, and a strong anti-virus suite...
I have a horrible horrible ugly feeling that you'll never be able to get a system that can reliably do all that. at the same time, just because they'll skimp on memory tech or interconnects, so you'll have most of the capabilities of a 1,000 core system wasted.
Re:Not Sure I'm Getting It (Score:2, Interesting)
Many core could allow for slower clock speeds, cooler chips and quite computers.
Of course, An OS could be designed so different modular componts run on different cores.
More is possible if you have thousands of cores. A machine with thousands of cores could conceivably pre-compute the possible computational consequences of your in-a-standard-deviation-most-likely actions, based on a genetic learning algorithm to figure out what you do when. In a sense, the more predictable you are, the faster it would get. Imagine an iPhone that does that!
Re:Not Sure I'm Getting It (Score:2, Interesting)
Why do all the processors have to be the same? why not have a x-core processor for the smaller tasks that are easily parallizable and have a high-cycle processor for the ones that aren't? Same might be done for cache requirements?