Linux May Need a Rewrite Beyond 48 Cores 462
An anonymous reader writes "There is interesting new research coming out of MIT which suggests current operating systems are struggling with the addition of more cores to the CPU. It appears that the problem, which affects the available memory in a chip when multiple cores are working on the same chunks of data, is getting worse and may be hitting a peak somewhere in the neighborhood of 48 cores, when entirely new operating systems will be needed, the report says. Luckily, we aren't anywhere near 48 cores and there is some time left to come up with a new Linux (Windows?)."
Linux already runs on thousands of cores (Score:3, Insightful)
SGI has some awfully big single-system-image linux boxes.
I saw a comment on the kernel mailing list about someone running into problems with 16 terabytes of RAM.
What are they talking about (Score:5, Insightful)
Jaguar? (Score:2, Insightful)
Re:based on a 1970s OS and language (Score:2, Insightful)
UNIX and C were great in their days. But perhaps not in the meg-core era.
So, what is better in your opinion? Java? Or maybe even ruby? Oh yes, that would be great. Run-time OS reflection through kernel drivers implemented as ruby modules.
Too bad CPU's don't come with built-in ruby interpreters.
Re:Only Linux? (Score:4, Insightful)
Re:Original Source and Actual Paper (Score:5, Insightful)
Re:Original Source and Actual Paper (Score:3, Insightful)
I don't know, guess I picked a bad title or something?
Slashdot: dramatically overstated news for nerds... since that seems to be the evolution of news services for some reason?
I'm working on a submission: Fox news just had a bit about the internet, I'm assuming that their headline is something like "WILL USING OBAMANET 'IPv6' KILL YOU AND MAKE YOUR CHILDREN TERRORISTS?"
Re:based on a 1970s OS and language (Score:5, Insightful)
Hahaha. Oh arrogances from ignorance, how I loath you.
how is this news? (Score:4, Insightful)
We've known about this problem for ... well, as long as we've had more than one core - actually as long as we've had SMP... You increase the number of cores/CPUs, you decrease available memory thruput per core, which was already the bottleneck anyway. Am I missing something here?
Re:Original Source and Actual Paper (Score:2, Insightful)
Wow, really just wow you sir are the cream of the crop! /sarcasm
the OP has a very valid point, i come to read about technology news on slashdot not scare pieces with little or no information or value, his post was far superior in every respect and yet got passed over for this garbage post. And you devalue his point further by not even giving him the time of day, way to go asshole.
Re:Original Source and Actual Paper (Score:4, Insightful)
Yes, but the submission that got accepted has a bullshit headline.
Of course "Linux May Need to Continue Making Incremental Changes Like It Has Been Doing For The Last Several Years To Scale Beyond 48 Cores" doesn't draw in as many clicks.
Re:Original Source and Actual Paper (Score:2, Insightful)
Re:Original Source and Actual Paper (Score:5, Insightful)
Re:Original Source and Actual Paper (Score:3, Insightful)
Elaborate please. I'm ignorant and curious.
Re:Original Source and Actual Paper (Score:2, Insightful)
Oh look, CmdrTaco published yet another story with a poorly-written, hypersensationalist summary! Par for the course.
Remember back when the slashdot "editors" were part of the community and would actually respond to site concerns raised by users? I haven't seen ANY "editor" post a reply to any slashdot user post in friggin YEARS. Good luck with getting their attention these days if you aren't an advertiser.
Re:Windows 7 scales to 256 cores (Score:4, Insightful)
Linux supposedly scales to 1024 or something like that. This is not what they supposedly scale to, but the performance impact of actually trying to use that many cores.
Re:Only Linux? (Score:3, Insightful)
No, their rewrite is also subject to to this issue. Go publicize Windows somewhere else.
No, it isn't subject to this issue. They removed the dispatcher lock. Go bash Windows somewhere else.
Re:Original Source and Actual Paper (Score:5, Insightful)
The very act of summarization constitutes an act of commentary. You're saying "I think the pertinent parts of this story are these, and the most important questions raised are those."
A good summary invites commentary and frames the questions in a way which makes for better discussion, but don't for a second imagine the OP ought to be value-neutral (if such a thing could even exist.)
Re:Obligatory xkcd reference (Score:2, Insightful)
If your Ubuntu 10.04 system can't play embedded youtube videos then you should get off your ass and fix it instead of wasting your time pasting xkcd links. Ubuntu plays flash videos out of the box without a single hitch for years.
Re:Original Source and Actual Paper (Score:5, Insightful)
I was kind of wondering about the "modern operating systems" comment... I think he meant "desktop operating systems".
What's a "desktop operating system" these days ? The only mainstream OS that hasn't seen extensive use and development in SMP server environments for a decade plus is OS X. For all the others, "desktop" vs "server" is just a matter of the bundled software and kernel tuning.
Even OS/2 could scale to 1024 processors if I recall correctly.
Yeah. Just like those old PPC Macs were "up to twice as fast" as a PC.
Re:Well, they better start coding now... (Score:3, Insightful)
PER CPU. As was pointed out in many other comments. Linux has already scaled to thousands of cores across many cpus.
Re:seeing as Linux does 10240 cores already, WTF? (Score:1, Insightful)
Stop posting the same wrong shit as everyone else.
Before you post:
1) Read the fucking article.
2) Read the fucking comments.
Are there any CPUs with 10240 cores? No.
Re:Original Source and Actual Paper (Score:5, Insightful)
Trolling, I'm sure, but to people who take "GNU/Linux" seriously: how much of any given distro is really GNU code anymore? While GNOME may still be preferred by Ubuntu, there are also a lot of Kbuntu users, and many other distros seem to prefer KDE. Neither XFree86 nor X.Org were ever GNU. Smaller installations, like smartphones and home gateways (which often do run Linux, even if you can't install a custom version like DD-WRT), use busybox for their basic command line tools, and almost certainly do not use glibc. Debian even went for the eglibc fork [osnews.com], partially because Ulrich Drepper makes Theo DeRaadt look like a nice guy. HURD has gone nowhere for 20 years now, even if it does have some neat ideas.
Non-GNU GUI applications and libraries now make up a huge percentage of a desktop distro, Apache and custom web apps make up a big chunk of server code, and smartphones may or may not have any GNU code at all.
So what's left of GNU code now? Well, gcc is likely to keep being the world's de facto C compiler (though even this was mainly because of the egcs fork way back when). I'm sure there will be legions of emacs users for years to come, and I guess a lot of people still prefer GNOME. GNU's basic command line tools and bash will no doubt still be used on servers and desktops. But is this really sufficient to warrant a "GNU/Linux" nomenclature, not to mention all the pedantry that surrounds it?
To the AnonCow troll above: GNU code has nothing to do with how the kernel handles multicore processors, so your whole point is moot within this context.
Re:So what the fuck is he doing here then? (Score:4, Insightful)
No, I read /. because of comments like eldavojohn's. If they were to disable the comments I'd unsubscribe it from my feeds immediately.
Re:Original Source and Actual Paper (Score:1, Insightful)
Despite having such a high UID he's got a solid reputation
Really? I've never thought so. To me, he's come of as a self-important blow hard who always thinks he has something interesting to add on every subject. He writes things that aren't really interesting but he knows will be moderated up. There have been tons of instances where he's had the first post to +5 that adds little-to-no value to the discussion and I'm forced to collapse his thread to get to the interesting stuff posted by people with actual expertise in the subject (yes, those people do exist.)
To me, he's a karma whore of the first degree who I wish would refrain from posting on subjects where he can't add anything productive to the conversation. The moderation system is great for dealing with trolls, since they're -1'd into oblivion and ignored. It sucks for dealing with karma whores because the first +5 post will receive the bulk of the responses and it's way too easy to get a +5.
Comment removed (Score:4, Insightful)
Re:Windows 7 scales to 256 cores (Score:4, Insightful)
The point isn't that NT Scales to 256 cores, the point is how efficient it is when scaling to this many processors. The NT Kernel in Win7 was adjusted so that systems with 64 or 256 CPUs have a very low overhead handling the extra processors.
Linux in theory (just like NT in theory) can support several thousand processors, but there is a level that this becomes inefficient as the overhead of managing the additional processors saturates a single system. (Hence other multi-SMP models are often used instead of a single 'system')
Just simply Google/Bing: windows7 256 Mark Russinovich
You can find nice articles and even videos of Mark talking about this in everyday terms to make it easy to understand.
Re:Original Source and Actual Paper (Score:3, Insightful)
I'm willing to bet that when mainstream 64-core general-purpose CPUs arrive, they will be NUMA and be partitioned in groups with shared cache. I will be surprised if all the cores have a shared cache other than possibly a large slow write-through level-4 cache. It would be very tricky to make an efficient modern cache with deferred writeback and access by 64 cores, and the gains over e.g. 4 smaller caches would be modest. The memory bandwidth requirements of a 64-core chip also make it very tempting to implement separate memory controllers for groups of cores instead of needing an extremely fast shared memory controller.
So all in all, I think a very fast desktop tomorrow will look like a shrunk version of a modern NUMA server, at least when it comes to what the operating system can see.
Re:Original Source and Actual Paper (Score:3, Insightful)
Completely agree.
Of course, this all ignores the fact that Linux already scales well beyond 48 cores. Even more so, it appears the group is confusing bus contention for OS scalability. The problem is, using modern CPUs (cores), they are sharing caching, which is all too frequently the real problem. The shared cache leads to cache contention.
Linux, right now, is capable of scaling well beyond 128 cores (err...cpus)...and more... Its just not standard code because the overhead is less optimal for 99.999% of the current user base. Basically this boils down to, Windows scales poorly. I've not met anyone who doesn't already know this.
Long story short, News at 11, a story everyone already knows. No new news is now news. Basically they documented what everyone already knows for almost a decade now.
Re:Original Source and Actual Paper (Score:3, Insightful)
I just read the original article that said they used 8 6-core processors to _simulate_ 48-core processor. It would be hard to experiment on a real 48-core processor as those are not readily available.
Re:Original Source and Actual Paper (Score:3, Insightful)
Think of in terms of cars. The processes are roads, the CPUs are cars and the cores are the seats in the cars, only the seats can each travel on different roads independently and share resources with the other seats in the same car. If you have a 2-seater and the seats are on different roads, they can obviously only go half as fast as if they are on the same road. Now if you have 48 seats in a car, than it isn't a car anymore, it's a bus, so obviously you'd have to make fundamental changes to the OS.
When it comes to computers, you can never go wrong with a car analogy.