A Million Node Supercomputer 116
An anonymous reader writes "Veteran of microcomputing Steve Furber, in his role as ICL Professor of Computer Engineering in the School of Computer Science at the University of Manchester, has called upon some old friends for his latest project: a brain-simulating supercomputer based on more than a million ARM processors."
More detailed information can be found in the research paper.
Oblig. Question (Score:1)
But will it run Lin.... ah, nevermind.
Re: (Score:2)
Re: (Score:2)
Well, since he's ICL professor of computer engineering I'd expect it to run George 3 or VME/B.
But, then again, maybe it'd run BBC Basic?
The Cloudy Concept (Score:2)
Just might run win....
In a future OS built by vast numbers of developers each chipping in 10 lines of code without knowing what the others wrote, having the code of each in its own node should keep more of it running? Reserve 100,000 or so nodes for guest processes.
To use the distributed GPU, each node will output to a rooftop display, viewed by satellite cam zoomed in to encompass the required number of nodes.
Reliability will be ensured by RAIN (redundant array of inexpensive neighborhoods).
Re: (Score:2)
more like "will it run anything anytime?"
FTFpaper:
[..] we have yet to gain access to silicon [..] But we have exposed the design to extensive simulation, up to [..] models incorporating four chips and eight ARM968 processor
I like the grandness of their vision but it would be nice to see at least a real-world version of one node (the 20 processor thingy). How big are the smallest cases needed for such a node? Is it even realistically possible to place 50000 of those within the range limits of Ethernet?
Re: (Score:1)
Re: (Score:2)
The better question is why do you need to simulate a brain in real time? I mean, if you can make something magical happen with a million cores in real time, why can't you just use the plain old Internet and make it happen with a million cores in 1/100th o
Re: (Score:2)
If you're going to build a 50,000 node supercomputer, I'm pretty sure you're not going to let yourself be limited by ethernet cables.
the Ethernet connection is directly quoted from the paper :) and it was more meant as "nice idea, but there are many not addressed practical problems" but as a show stopper argument.
I mean, if you can make something magical happen with a million cores in real time, why can't you just use the plain old Internet and make it happen with a million cores in 1/100th or 1/1000th real time?
sometimes I miss the obvious; this is a *very* interesting thought. the BOINC project "be part of a brain" (cover title,should be changed before release...) would attract my attention
Re: (Score:2)
It's not obvious to me that a brain simulating algorithm would be parallelizable enough to run effectively in chunks distributed over the internet. Or maybe it would but that's just not as fun as a 50,000 node computer.
Re: (Score:2)
Re: (Score:2)
If an algorithm can be split between 50,000 cores, it can run on one core. I see no reason why the number of cores makes any difference. Sure it will be slower, but computation is computation, whether it's happening usefully fast on a 50,000 core behemoth, or pointlessly slow on a 50Mhz 486..
Re: (Score:2)
Re: (Score:2)
"It's not obvious to me that a brain simulating algorithm would be parallelizable enough to run effectively in chunks distributed over the internet. "
It wouldn't. The timing of the spikes arrival at neurons is critical to within less than the usual jitter across large networks. General purpose CPUs just are not an efficient way to simulate neurons, and never will be.
This comment on TFA links to information on a much more technically advanced and likely to succeed method:
Re: (Score:2)
Re: (Score:2)
A Beowulf cluster of supercomputers actually sounds pretty awesome.
Re: (Score:2)
Isn't that just a ultracomputer made from supercomputers?
In Soviet Russia (Score:1)
Beowulf clusters YOU!
Re: (Score:2)
How about a supercomputer made of a million Beowulf clusters?
Re: (Score:2)
How about a supercomputer made of a million Beowulf clusters?
Imagine a nested Beowulf cluster of those!
Re:Oblig. Question (Score:5, Interesting)
Re: (Score:1)
Re: (Score:1)
Tremendous overhead (Score:2)
Re: (Score:2)
Re: (Score:2)
In other words, it's okay if it runs slow, because your brain does, and makes up for it with parallelism and fuzzy logic.
Re:Tremendous overhead (Score:4, Interesting)
Re:Tremendous overhead (Score:4, Informative)
I am not a neuroscientist. As a grad student I do study artificial neural networks, which means that I must also have a little knowledge of neuroscience.
The brain is not a fully connected network. It is divided into many sub-networks. I think it's estimated at about 500k, but don't quote me on that number. These sub-networks are often layered, so if you have a three-layer feed-forward sub-network of 5 cells in each layer, each of these cells has only 5 inputs except for the 5 nodes in the input layer, which connects to other sub-networks. (If there are connections from later layers back to earlier layers, the network is said to be a 'feedback' rather than feed-forward network.) These sorts of networks can be simulated very efficiently on parallel hardware, as a cell mostly gets information from the cells that are close to it.
In short, your suspicion is entirely correct. Moreover, you not only don't need fast connections between many of your processing nodes, most of them don't need to be connected to each other at all.
This is the reason why neural networks are interesting in the first place: that they can be simulated on parallel hardware when we don't know a good parallel algorithm with conventional computing techniques. (If it interests you: another name for neural networks is 'parallel distributed computing'.)
There is a hard limit on the 'order' (think of it as function complexity) of functions that can be computed with a given network. To compute a function beyond that limit, you need to have a larger number of inputs to some cells, thereby increasing the order of the network but making it less parallel. Most everyday things are in fact of surprisingly low order. Fukushima's neucognitron can perform tasks like handwriting recognition with only highly local information.
Re: (Score:2)
Let's say it's simpler than most earlier estimates have said, say an average of 7 inputs and 1 output per neuron, each firing up to 100 times per second with about 3 bits of phase information encoded in each spike. So that's 300bps of bandwidth out and 2100bps in, so 300-2400bps/neuron depending on how it's implemented. Times 20-100 billion neurons that's still some serious bandwidth, (6 -240Tbps) even if it is nearly all local. Still, a lot of non-local communication takes place as shown by the "connectome
Re: (Score:2)
Let's say it's simpler than most earlier estimates have said, say an average of 7 inputs and 1 output per neuron, each firing up to 100 times per second with about 3 bits of phase information encoded in each spike. So that's 300bps of bandwidth out and 2100bps in, so 300-2400bps/neuron depending on how it's implemented. Times 20-100 billion neurons that's still some serious bandwidth, (6 -240Tbps) even if it is nearly all local.
I won't insult you by checking your calculations but your number is too large for your assumptions. Remember that when simulating a brain each processor won't compute for a single cell, but a local group of cells . The vast majority of the bandwidth use would therefore be between each CPU and its memory rather than between different CPUs. Each CPU would than have connections with those CPUs whose local groups of cells its own communicates with. The less local you get, the lower the total bandwidth requireme
Re: (Score:2)
By "local" I meant on-chip and within-node communication. The communication bandwidth does not drop off with locality as fast as you'd think - much if not most of the brain's volume is white matter devoted to long-distance communication.
Re: (Score:1)
Re: (Score:2)
He is using different speeds for the interconnects based on distance to get around the issue. This is not uncommon in supercomputers today where the cores on the node can communicate much faster than the nodes can communicate over infinity band. It sort of reminds me of the Connection machine that used a hyper-cube type system for interconnects.
The problems that this computer is going to try and solve are probably well suited to this type of system after all I am sure that our brains don't have a zero laten
ARM processors? (Score:3)
Shouldn't they be using BRAIN processors? /duck
Re: (Score:3)
Re: (Score:2)
which makes it all the more painful to have to pay through the NOSE for it.
Re: (Score:2)
With that many processors, they should call it ARMY.
Not Even Close (Score:1)
Re: (Score:1)
Depends ... they didn't say whose brain!
How about Abby? You know, Abby Normal?
Re: (Score:2)
And you base that on... what?
Re: (Score:2)
Well, not that I'm a mind reader, but my take on this is that we don't bloody know how those few lbs of grey matter work, how it self organizes, the exacting details of behaviors that drive it, etc etc
On top of that what do you feed a brain? what is it going to do? They'd need to interface it with something that can challenge it, provide meaningful feedback, etc on and on.
Whatever it is they simulate I'd not call it a brain, or a mind. Perhaps a highly complex neural net, but it will be running in slow moti
Re: (Score:1)
Not only would a brain simulator have to simulate neurons, but also synapses, neurotransmitters, neurotransmitter receptor types, glial cell types (e.g. astrocyte computation), mRNA expression and probably about a library of congress worth of stuff we don't even know about yet.
Re: (Score:2)
Some of that may be true, but I doubt we need to simulate all of that to reproduce a brains function. We don't need to simulate transistors to emulate a computer architecture after all.
Re: (Score:2)
They probably won't make it think, but there's a lot of environmental input and output that is mechanistic in nature. We do know how almost all of that works. It's the language and meme thing we're just scratching.
The boundary between symbology and mental process may be illuminated by how this thing behaves. That'd be worth its price.
Re: (Score:2)
This won't get anywhere near simulating a brain.
Too true. How about trying to simulate self awareness.
Use (say three) processes to monitor and optimize each other. These would run a virtual machine as a single self aware entity.
Laplace's demon, Omega, or Les? (Score:2)
Laplace's demon, Omega, or Les? From my reading this million node supercomputer will be Les.
Re: (Score:2)
Build a mouse brain first. (Score:5, Interesting)
OK, a mouse brain has about 1/1000 the mass of a human brain. So build a mouse brain with 1000 ARM CPUs, which ought to fit in one rack, and demonstrate the full range of mouse behavior, from vision to motor control.
I read the paper. It's a "build it and they will come" design. There's no insight into how to get intelligence out of the thing, just faith that if we hook enough nodes together, something will happen.
About 20 years ago, I went to hear Rodney Brooks (the artificial insect guy from MIT) talk about the next project after his reactive insects. He was talking about getting to human-level AI by building Cog [mit.edu], a robot head and hand that was supposed to "act human". I asked him why, since he'd already done insect-level AI, he didn't try mouse-level AI next, since that might be within reach. He said "Because I don't want to go down in history as the man who created the world's best artificial mouse".
Cog turned out to be a dead end. It was rather embarrassing to all concerned. As one grad student said, "It just sits there. That's all it does."
Re:Build a mouse brain first. (Score:4, Funny)
As one grad student said, "It just sits there. That's all it does."
Sounds like it actually modeled most of human behavior fairly well then.
Re:Build a mouse brain first. (Score:4, Insightful)
Without the need to breath, eat, procreate, keep my brain occupied etc... I would certainly do the same.
Re: (Score:2)
At least that of a government employee...
Re: (Score:2)
As a government employee, I resent that. I don't just sit around, I also post to Slashdot.
Re: (Score:3)
Yeah, well. As an interested bystander (a physicist), to me it also seems AI ran into several dead ends because of setting the aims too high. I remember for as long as i can read people predicted that machines which would replace human translators to be around the corner.
IHMO the fundamental mistake is to define something like intelligence on one hand to be very specific, and then hope to reach it by general methods, but taking a few shortcuts, which are not even known yet...
The general rule of thumb which
Re: (Score:3)
The general rule of thumb which the physicists learned the hard way is that assuming something general and hoping that the shortcuts you need to take to get a result magically fall out once you randomly scatter enough phd students at a subject, is futile.
I heard that exact approach proposed more than once at Stanford in the 1980s.
AI is no longer a hardware problem. Any major data center probably has enough power to do a human brain, if we only knew how to program it. In some areas, like vision, sheer compute power has helped in a big way. Some problems can be hammered into the ground with machine learning techniques and CPU time. In other areas, we're still stuck. There's been almost no progress on "common sense reasoning" in years.
On the other hand,
Re: (Score:2)
Since the stated objective is to simulate a brain, I doubt that marix algebra is going to cut it.
Re: (Score:2)
Likely not. But if you want to simulate something complicated, matrix algebra is just about always the best place to start. You'll often end up there anyway, no matter where you start. You may find something better for some specialized or simplified bits, but it's seldom worth the extra effort - usually more difficult, less accurate and slower (the tridefecta).
Re: (Score:2)
Any major data center probably has enough power to do a human brain, if we only knew how to program it.
I think we're underestimating what these guys know about how to program it. If they can get the architecture to be more like a brain, it may allow them to implement accurate simulations of the neuronal processes we see in a brain, and then start to work on the processes those neurons are used for.
I highly doubt this will be the only iteration of the hardware needed to get it to think, even at a small-furry-mammal level, but getting it to do a few self-aware things could be very helpful.
Re: (Score:2)
Even if they fail to produce anything interesting, that in itself will be an interesting result.
There are likely a number of assumptions about intelligence as an emergent behaviour of non-quantum physical phenomena that could be invalidated by the failure of this experiment.
"Brains can't work according to such-and-such a principle, because if that were true, Furber would've succeeded."
Re: (Score:1, Flamebait)
Great, you read the paper. DId you laso read the paper and studies that lead to thins? did you know they have already simulated a few neurons? and the activity looked like the human brain? no? STFU.
IN our likely life time, we will have a simulated brain. This will just be one piece of a larger model.
Once we have the model, we will be able to do a lot of things. What to simulate damage and look at long term effects? Speed up the model, get a years worth of data in days. Want to studies some thing closer? sl
Re: (Score:1)
Nowhere in TFA does it say they're trying to build an AI or simulate the human brain. They're aiming for 1b neurons, which would be a tiny fraction of the human brain, and they're particularly interested in proving the ability to maximize supercomputing capacity while minimizing energy use.
Re: (Score:2)
It's one thing to copy human a physical human behavior, or to even be able to calculate something like the traveling salesman problem. It's another thing entirely to develop the ability to start an intelligence that naturally absorbs sensory on educational information, identifies patterns and holes in information, turns that into a question, and then follows the correct steps to learn what is unknown. Between us and a machine that can do that, we'd still have personality and drives (survival, success, compa
Re: (Score:2)
---The computer can look at the U.S. budget deficit, and come up with a balanced budget that will make most citizens reasonably happy, or figure out how to structure a health care bill that will provide every one with excellent care at an affordable price. You might think that the failure of the U.S. government to do so means that nobody can, but that is a failure of politics, not intelligence.
I doubt it a computer could do a good job at maxing the desired out puts and minimizing the required inputs. Humans have a different sense of fairness though and how we subjectively feel about it has a lot to do with how happy with it we are.
---When a computer can come into contact with a piece of art (photo, literature, comic), understand it, and have it mean something that the computer can learn from, and then examine something else, and apply the lessons learned to this new thing, and sort among many different things its learned to figure out which ones are most similar.
Again I am not so sure a computer will be able to respond our art they way we do. I am not sure another intelligent biologic organism, say from space, could either. Its our art, and it reflects on some basic levels experience we all share. A computer won't see an im
Re: (Score:2)
Well, it's just a cog in a big machine.
Re: (Score:1)
Since when is brain mass proportionate to "intelligence" (a.k.a. hardware necessary to mimic it)? Or... maybe that's why the elephants look at me condescendingly every time I'm at the zoo...
Putting the cart before the horse (Score:5, Insightful)
This makes sense, how?
It's like trying to simulate a computer by wiring 5 million transistors. Without a deep understanding of how computers work and a plan for implementation, the result will be worthless.
I see this all the time in AI strategies. Without no deep understanding of AI, the project implements bad assumptions.
Some examples: no way to encode the adjacency information, a fixed internal encoding system which cannot change (ie - a chess program that can't learn checkers), linear input->process->output models, and so on.
Before building a system with a million processors capable of simulating the brain, how about we design an algorithm that embodies the simplest possible AI?
Re:Putting the cart before the horse (Score:4, Insightful)
Re: (Score:2)
Re: (Score:2)
Typical IT project philosophy: I'll go to the customer and try to get some [strike]requirements[/strike] money. The rest of you, start coding.
FTFY
Re: (Score:2)
We don't start coding until the check clears. We don't stop adding features until the money runs out. But there is no relationship with requirements.
Re: (Score:1)
The problem with AI seems to be a lack of RI. You can't make an artificial anything if you don't know how the real something works.
Re: (Score:2)
Re: (Score:2)
Project for you: identify the bad assumptions in their model without building one and trying it out.
Okay. Go.
Re: (Score:2)
Without no deep understanding of AI, the project implements bad assumptions.
Speaking of bad assumptions...
how about we design an algorithm that embodies the simplest possible AI?
Computationalism has been dead for 30 years. Anyone still championing that failed philosophy either doesn't understand the problem or is clinging to it out of desperation.
Failed philosophy? (Score:2)
I'm attempting to solve AI and have found a dearth of informed people who can talk about it.
I'd love to chat with you about your views. If you feel up to it, drop me a line:
Niroz (dot) 9 (dot) okianwarrior (at) spamgourmet.com
Whose brain? (Score:2)
Abby someone?
I forsee ... (Score:2)
Re: (Score:2)
The number of nodes is meaningless (Score:2)
The number of nodes and processing power per node is meaningless unless they can connect them together in a similar fashion to the brain, sure they mention a "brain like" arrangement but the reason our brains are so sophisticated is not due to processing power but due to organisation. Brains are slow, really really slow but the parallelism and connectivity is beyond anything we can build at the moment and that is why we keep on failing on AI. An example is adding two numbers together, easy to do for a pro
Re: (Score:2)
Correct. We don't even understand how the human brain is "cognitive" much less how the information is stored or retrieved. Almost everything we know is an assumption at its base. Yes we have some ideas about which regions of the brain control certain functions but we have not a clue as to how those things work.
Re: (Score:2)
Yes we do have a clue, several in fact. we even have an incredibly simple model(few neurons).
We don't need to understand something to simulate it. It certainly helps. People in a factory can assemble a plane the works perfectly well, but never even have heard of bernoulli's principle.
This sort of work is need so we can define 'cognitive' with more accuracy.
We know a hell of a lot more then you imply in your post. The parent clearly doesn't know what he is talking about and did even bother to read the story,
Re: (Score:2)
People are pretty crappy at adding two numbers together too. That sounds like a triumph for AI, not a failing.
Misleading Headline (Score:2)
According to the research paper the goal is a million *processor* computer, not a million *node* computer. Each node described in the paper is made up of 20 ARM processors, so it would technically be a 50,000 node computer.
Re: (Score:2)
Are those single- or multi-core ARM units? (does ARM even do multi-core units?)
And really, if you count all the processor units in a graphics chip, there are probably some computers that could count several million individual processors in their architecture right now.
Re: (Score:2)
The headline is useless. Proposing a million-core computer isn't news, since there's a 1.6 million core computer about to be deployed [wikipedia.org]. The headline should reflect what they're planning to do with this machine.
An once again quantity is confused with quality (Score:2)
The problem in simulating a brain is not computing power. It is software. This is a worthless publicity stunt.
Re: (Score:2)
The problem in simulating a brain is not computing power. It is software.
There is just so much wrong here I don't know where to even begin.
Re: (Score:2)
Indeed.
For example, for a realistic simulation, you would need to have a person in there. How is that going to happen? If (which I doubt, but there are possibilities, after all biological brains do acquire a person somehow as well) this is possible, is turning the simulation off then an act of murder? Also, what good is simulation a brain in the first place? If it is realistic, it is no better then the 8 billion we already have. If not, it is useless.
4 Out of memory, 0:1 (Score:1)
Re: (Score:1)