Project IceStorm Passes Another Milestone: Building a CPU 107
beckman101 writes: FPGAs — specialized, high speed chips with large arrays of configurable logic — are usually highly proprietary. Anyone who has used one is familiar with the buggy and node-locked accompanying tools that FPGA manufacturers provide. Project IceStorm aims to change that by reverse-engineering some Lattice FPGAs to produce an open-source toolchain, and today it passed a milestone. The J1 open-source CPU is building under IceStorm, and running on real hardware. The result is a fairly puny microcontroller, but possibly the world's most open one.
An open cpu? (Score:4, Funny)
I'm pretty sure CPUs are supposed to be closed so as to keep dust out.
Re: (Score:3, Funny)
No, the fan you put on the CPU blows the dust away.
Re: (Score:2)
I'm pretty sure CPUs are supposed to be closed so as to keep dust out.
And the magic smoke in
Most people won't care (Score:1)
It bothers me that most people don't care. They don't understand that Intel/AMD CPUs could or will have backdoors. If not now, then very soon in the future.
Re:Most people won't care (Score:5, Informative)
Forget the future, they are here and they are advertised as features:
https://en.wikipedia.org/wiki/... [wikipedia.org]
Re:Most people won't care (Score:5, Insightful)
It's kind of the same issue with open source software, as far as the "most people don't care" aspect, but at an even greater disadvantage that open source software. I don't have a chip fab (at least I could compile open source software), and so even if I were capable of understanding the chip design, there's not much of a guarantee that the physical chip I purchase doesn't have some proprietary back door built into it.
Like most people, I'm even lazy about the open source software I use. While I try to download from trusted sources, there's no guarantee that what I actually install matches the current stable version in the repo. I'm taking a leap of faith.
In both cases (including the former where I indicated my ignorare about chip design), presumably I am counting on other experts to understand the chip or understand the source code for me, but only in the latter case could I actually assemble the product myself in order to guarantee matching the reviewed, stable code.
Re:Most people won't care (Score:4, Interesting)
At some point you have to trust someone, be it the distributor from whom you're getting the os, the manufacturer or reseller supplying the hardware or even the supplier of the compiler...
The only thing you can practically do, is ensure everything you use is available from multiple opposing sources... Something that's in use by both the us and russian governments is unlikely to be backdoored as both organisations have sufficient resources to perform the due diligence checks.
Re: (Score:2)
At some point you have to trust someone, be it the distributor from whom you're getting the os, the manufacturer or reseller supplying the hardware or even the supplier of the compiler...
Or maybe not? [projectoberon.com]
Re: (Score:2)
I am not much better. I do, sometimes, review the source for interesting parts. At best, absolute best, I will find a bug and then go through the code and see if I can actually determine a fix. If I find a fix I send the code to the authors for inclusion and, obviously, with no strings attached. I have been able to do that on only a trivial number of occasions, I can think of three though it may have been more for I am old and my memory is faulty.
Re:Most people won't care (Score:4, Interesting)
An Intel i7 Quadcore has 1.7 Billions transistors on board. A Haswell E 18 Core monster chip 5.5 billion. Even a simple ARM Cortex has 26 million transistors.
Do you think any single person at Intel knows everything about such a chip? Even the experts of the experts? How do do you think you are going to even comprehend such a thing even if it is open source? It really makes no difference, and no open source community is going to design a modern high-performance CPU. Intel invested 10.6 billion in R&D in 2013.
At some point you are going to have to start trusting someone. Why everything has to be open is beyond me.
Re: (Score:3)
Do you think any single person at Intel knows everything about such a chip? Even the experts of the experts? How do do you think you are going to even comprehend such a thing even if it is open source? It really makes no difference, and no open source community is going to design a modern high-performance CPU. Intel invested 10.6 billion in R&D in 2013.
Intel is bound by the requirement to run legacy software. Even an ARM system effectively has to be able to run it. A designer of a new open system would have no such constraints. Thus, no need to invest $10.6B and 5.5 billion transistors to reach a reasonable result for a wide range of applications. Attempting to review or redesign an Intel system would of course be a folly, but also completely pointless to begin with.
Re: (Score:2)
Actually, most of the billions of transistors in Intel parts go to cache.
Well, that's true. But even 1M transistors for a core might be a bit too much. Aside from the problems with validation, you wonder what other interesting choices for HW design we could have had with much simpler cores. Evolutionary optimization of everything below late-stage compiler intermediate code, perhaps? That would presumably be a massive undertaking, but perhaps very valuable in the long run. I don't think we have any idea how the existing systems fare in the larger universe of potential architectu
Re: (Score:2)
First, compiler optimisation only deal with static code, and not with the dynamic aspects of the code as it runs. Advanced CPU optimisations in silicon does this.
And highly advanced compilers are also very complex, requiring PhD level CS to get into. A simple compiler is easy to write, I have done this twice (one for a lazy Functional language, in 1989). An advanced compiler? No way.
Shifting the complexity to the compiler does not reduce the complexity, but definitely does reduce the performance. The CPU ca
Re: (Score:3)
Eh. When laying out silicon, you generally use libraries of simple parts you chain together. You make a register once, and then you replicate it each place you need a register. A TON of those transistors are cache, which is the same pattern repeating over and over again.
I'm not saying there's anyone who's looked at every transistor, but there's probably somebody who's looked at the layout of a cache cell, a register, an ALU, standard multiplexers, etc.
Re: (Score:2)
An Intel i7 Quadcore has 1.7 Billions transistors on board.
Transistor count is a poor indicator of complexity. Most of those transistors are used to implement cache memory, which is just the same simple cell repeated millions of times.
Re: (Score:2)
The non-cache part of an i7 is still extremely complex.
Re: (Score:2)
a) If you think anyone understands the whole of Linux you are kidding yourself. Besides, I am going to contend that a single CPU is more complex to build than Linux.
b) I am not offended
c) I do not think that a naive bunch of OSS paranoid freaks can product something with the complexity AND performance of a modern CPU.
Maybe there is a tradeoff here, and it is probably worth look at it, perhaps by Intel putting their design in some kind of trusted escrow. But there is whole lot more complexity below the surfa
Re: (Score:2, Informative)
Most people don't care because they aren't even interested in computers.
If you talk about Slashdot readers then they have all read enough articles to know that open source itself doesn't provide a perfect protection against backdoors.
You also have to read the source to see that no backdoors are put in place, and you need to build the executable to make sure that the version you are running isn't built from a version of the source with backdoors in it.
Now, assume that a processor like this becomes as mainstr
Re: (Score:2)
Re: (Score:2)
Sorry, but you need to get real here.
First, these FPGAs don't quite have enough power to run a design comparable with an Intel/AMD CPU as most people know them. So this effort is pretty much irrelevant for dealing with backdoors in mainstream hardware. Their goals are quite different, mostly to do with on the fly reconfigurable computation - something that is hard or impossible to do if you cannot generate the bitstream using your tools.
Second, even if you had a fully open sourced design somehow, how many p
Re: (Score:2)
I disagree. Saying "people couldn't understand the hardware" is the same as people saying "open source software is irrelevant because you can't understand the software."
Some people can. I have an electrical engineering degree and specialized in computer architecture in grad school. I could understand it. And just like anything else...it's not that hard when you know what you're looking at.
Re: (Score:2)
Yeah, right. So it only took a decade to get serious OpenSSL bugs discovered and fixed - despite the project being open from the start. And that is most likely an order of magnitude smaller project than something like even moderately complex CPU core, written in a language that many more people are familiar with.
Yes, if it wasn't open, the bugs probably wouldn't have been found, but that's speculation - we simply will never know. If you hope to find a "backdoor" (which could be simply so subtle as intention
Re: (Score:2)
Oh, you're absolutely right. I'm definitely not saying it would be impossible to hide a backdoor in an open core design. Absolutely could. Same thing with FOSS...just see the Underhanded C Competition.
But today you could have (and probably do have...) explicit backdoors in silicon, besides debugging interfaces, and you'd never know. With an open core design, you'd have to hide it.
Re:Most people won't care (Score:5, Funny)
They don't understand that Intel/AMD CPUs could or will have backdoors. If not now, then very soon in the future.
I usually decap my CPUs and inspect them before installing them. I haven't seen anything suspicious so far.
The real problem is Windows. Having to disassemble/inspect the OS before installing wasn't too bad, it's the constant stream of patches that gets me down.
Re: (Score:2)
You can actually see some of the Windows source code if you want. You must sign an NDA and you have to justify your access (at least you did while I was still in the MVP program - and that was as an MVP, some of the most trusted folks) but the program to do so is called the Shared Source Initiative or at least was. Meh, a quick Google indicates it still is.
So you can see the source, in part, if you want to. We had to tell them what we wanted to do with it (usually it involves tying an application in cleaner
Re: (Score:2)
You can actually see some of the Windows source code if you want. You must sign an NDA and you have to justify your access (at least you did while I was still in the MVP program
The Pirate Bay allows you to skip the red tape.
So you can see the source, in part, if you want to.
"In part".
The patches are the biggest worry for me. Microsoft could backdoor most of the computers in the world overnight if it wanted to. They can even target specific IP addresses if they want to because the own the patch distribution network. This is why I feel it's important to disassemble/inspect the patches before installing. There's no source code available for patches so keeping my Windows PC in tip-top condition is a big drain on my resources. Still ..
Re: (Score:2)
The Pirate Bay allows you to skip the red tape.
Unless something new has happened in the last decade, the only source leaks I'm aware of are NT4 and Win2k. So if you were willing to do without x86-64, most of ACPI, UEFI, IPv6, and pretty much every piece of hardware on your motherboard, I guess you could use that.
You can probably live without Metro, so there is that.
Re: (Score:1)
Patches are generally easier to access in my experience. I have literally used the excuse, "I am curious." I have also used, "I feel like breaking stuff." It does matter who you are and how much you have a history with the company (as to be expected). However, the source is there - it is just not open. I would not want to be the one caught leaking it either.
Re: (Score:2)
The size and complexity of modern CPU's mean that you don't stand a chance at getting a no-backdoor assurance for anything useful.
Down on this scale, on microprocessors and IC's, it's possible but incredibly difficult. If you were serious about no-backdoors (e.g. military), you'd not be using an off-the-shelf American product. You'd be describing the chip you want to build and validating the end-product against your design. Because that's the ONLY way to be sure.
There is absolutely no guarantee that your
Re: (Score:2)
If you were serious about no-backdoors (e.g. military), you'd not be using an off-the-shelf American product.
On the other hand, if you were the US military, you might as well be doing just that! [slashdot.org]
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
You are thinking of the military of the 1970s. After Reagan, there was a push for COTS everywhere. Global Foundries was sold to fat boys in the robes from the UAE. These days, the U.S. military gets its stuff just like you do. The major defense firms probably do a bit of their own programming with their own tools, but they aren't producing chips. Even Altera is being sold to Intel so they can screw up the FPGA market as badly as they did the processor market.
Re: (Score:2)
Re: (Score:2)
FPGA can also have backdoors.
Most people shouldn't care (Score:2)
"most people" have no interest in programming FPGAs, so they shouldn't care. "Most people who are into open source CPU design"... they should.
You can think of this as an open source compiler for a closed source piece of hardware: FPGAs already give you the freedom to implement whatever IC you want (e.g a small open source CPU design) - the only closed part about them is the tool chain provided by the FPGA manufacturer and the more unique parts of a particular FPGA hardware beyond the logic block.
This projec
Re: (Score:2)
buggy and node-locked tools. (Score:1)
Buggy and node-locked tools sure sounds like what I used. Hopefully an open source version can fix the shitty stability along with the shitty vendor lock and pricing.
What's special here?? (Score:4, Interesting)
There are plenty of fully open source CPUs around.
There is a community building CPUs from discrete logic (http://mycpu.selfhost.it/), from transistors (http://www.extremetech.com/computing/128035-how-to-build-an-8-bit-computer-from-scratch) and even from relays (http://www.nablaman.com/relay/).
There are also Forth CPUs which are freely embeddable into a FPGA (the J1: http://www.excamera.com/sphinx/fpga-j1.html) and which can be purchased now (http://excamera.com/sphinx/gameduino/).
Re:What's special here?? (Score:5, Informative)
If you had read the entire summary you would have seen that the CPU design they were using was the J1 that you mention.
Re:What's special here?? (Score:4, Informative)
yeah so why it's marketed in the blurb as an open source cpu when the interesting thing is an open source toolchain for a closed silicon fpga?
Re: (Score:1)
This is not meant to be sarcastic but, after I wrote what I wrote, I came back up here to point this out because it seemed to be interpretable as sarcastic.
So how is this useful in the real world - outside of academia and *maybe* a handful of geeks (world wide) that are able to make use of this? I am quite a geek, I pride myself on this, but I am honest enough to admit that I would have no idea where to start to take advantage of this nor do I see how this can have a meaningful impact on my life as it does
Re: (Score:1)
There we go - something to look into. I had hoped someone would give me a rational answer that actually had some substance.
Re: (Score:2)
Re:What's special here?? (Score:4, Informative)
A few notes regarding your comment.
This is not actually about an "open toolchain to program an FPGA". It's not actually about programming at all. Let me try to explain here (in a simple way) what IceStorm is, or better, what an open-source can do for the FPGA design flow.
So, you start with a description of your design. Your design usually describes the behavour of your system, which is often done in HDL (Hardware Description Languages), like VHDL and Verilog, or even higher-level ones (SystemC, even Python). But, as with every digital design (imagine classical design with TTL 74/CMOS 40 series), you often need more than "logic" - you may need a PLL, you may need a dedicated multiplier (hey, you don't want to design this by hand with gates and flip flops, do you?), or other complex, often mixed-signal primitives. ...
FPGAs are basically a set of basic blocks (think again 74/40 series), which can be interconnected almost at will. The blocks actually differ from classic chips: what you'll find in FPGAs [I'll speak about Xilinx but Lattice should be similar] are LUT (Look-Up Tables), which can provide any N-input/Y-output digital function (often 4 to 6 inputs, and one output), and Flip Flops (with enable, set, reset, so on). In addition, FPGA provide "hard IP cores", like PLLs, Multipliers, Serializers, Multiplexers, so on, so on.
Now, to have your design to work on the FPGA, you need to synthesize it - convert the behavioural model into these building blocks (LUTs, so on). This is the first phase - and it depends on the target FPGA because the available IP hard blocks differ from manufacturer to manufacturer, from family to family, and even from small to larger, same family devices. This synthesis process will get you a "netlist" - a list of all used primitives and the interconnections needed between them.
Next step is to place the design - pick up each of those primitives, and choose an instance on the FPGA where it will be assigned. This can be tricky, because routing inside FPGA is not 1-to-all, and timing is more affected by routing than anything else - so you'll want your critical (timing critical) paths to be placed where you can later route signals faster. This is really FPGA dependant, much more than synthesis.
Then you have routing: after placing all primitives, you need to decide how to route all signals (this is very complex, FPGA dependant). After routing is done, you have the whole design for that FPGA done - everyting is placed, routed.
Well, you have not.
The next step is to generate a bitstream, based on the netlist output from Route, that can be sent to the FPGA (usually via JTAG). This is actually where less information is known - where each bit from the bitstream maps inside the FPGA switch fabric or LUT configuration or SRAM cells, others,
Then comes programming: sending the bitstream to FPGA. This is usually simple, and many open tools exist (at least for Xilinx).
Now, do these tools do?
Yosys: Verilog synthesis
Arachne-pr: Place & Route
IceStorm: Last part, which is the hardest - to map the final, routed netlist into the "proprietary" bitstream.
Hope I shed some light on the matter.
And yes, being J1 or any another CPU (like my own ZPUino or XThunderCore) is irrelevant - what's relevant is to prove that IceStorm seems to generate correct bitstreams for this platform.
Alvie
Re: (Score:1)
Re: (Score:2)
You do have fine characterization of building blocks, and even macrocells - those are in datasheet, for most vendors. These timings relate to a single part (with a specific speed grade), locked to process[1], voltage and temperature (worst case is often used).
But not for routing delays (general-purpose and dedicated [i.e., clock]).
So, indeed, you're absolutely right here.
Alvie
[1] For those not aquainted with what "process" means, think about handcrafting - some stuff will have less imperfections, others wil
Re: (Score:1)
Re: (Score:2)
Thanks, I will read those links in detail.
I came across Cx quite some months ago, but I recall dismissing it because I found it not production ready.
On a side note, I do happen to work closely with systems requiring DO-254 certification [wikipedia.org], mostly FPGA and ASIC. Not sure most HLD will easily fit in, but I'm always open to explore and suggest if feasible.
EASA SWCEH-001 memo [europa.eu] is probably more detailed on requirements than DO-254. You should read it if you're interested in this area.
Alvie
Re:DO-254 (Score:1)
Very interesting that you do DO-254. My background was high assurance and I studied a lot of DO-178B stuff in the process. I didn't work in that market but it generated many high-quality components applicable to other areas. What they pulled off in terms of features amenable to assurance/certification also gave me a guess at what the next project with similar complexity would pull off. Also how I discovered SPARK and Astree. :)
My recent focus is on clean-slate, secure hardware with two aspects being ensurin
Re: (Score:1)
You still need those buggy proprietary tools in order to use them. I think you kind of missed the point here; it's not that there's anything special about that CPU, it's about the progress of the open source toolchain.
Re: (Score:1)
Show me one that doesn't require a special FAB to produce the actual chip, at which point you can't control whats actually going on in the chip itself.
There are NO open source CPUs, only open source designs.
It stops being open source when it needs to run on a proprietary FPGA.
Forth? (Score:2, Funny)
? Program Forth in who want
Re: (Score:2)
Re: (Score:1)
This is why it sometimes sucks to not be an expert in everything. *sighs* I have no idea if you do not speak English and used Google Translate or if that is a joke based on the programming language.
Given the +1 status I am inclined to think it is either a joke or there is someone else who felt left out of a potential joke and decided they'd mod you up just in case it was funny.
It is not quite Poe's Law (or coleslaw) but it is damned close. At this point I guess the only rational choice is to laugh and prete
Re: (Score:2)
https://en.wikipedia.org/wiki/... [wikipedia.org]
I for one find Forth to be a fascinating language. The fact that it has been around for a while does not diminish it in any way.
I first heard of it back in the 80's while learning assembler on the 6502. I noticed it again while getting to know FreeBSD (loader stage 3) and decided to finally learn it when I came across some old manuals and software for my newly acquired PET CBM 8032
Re: (Score:2)
Re: (Score:1)
I will look there too, thanks. It looks kind of interesting actually. I am not sure that I like the syntax but, well, I can adjust I suppose.
And Lattice wont shut this project down because? (Score:2)
What is to stop Lattice from simply shutting down this project for an open FPGA toolchain for their FPGAs?
Or does Lattice not care about this particular project for some reason?
Re: (Score:2)
What is to stop Lattice from simply shutting down this project for an open FPGA toolchain for their FPGAs?
Why and how would they shutdown this project? This project is the FPGA equivalent of writing a open source assembler or compiler for new CPU. They do not publish their bitstream format, but reverse engineering is perfectly legal.
Re: (Score:2)
It IS free advertising!
It intrigued me so much that I just bought an evaluation board.
Re: (Score:1)
With my limited understanding of what this is, I am kind of surprised that they are not actively helping the project. Any competitors can do this more easily than this group likely can. Their stuff is all covered by patents anyhow. Why not help? It looks good and will improve sales in a niche market where no other competition exists. It seems that they could benefit if they helped and the publicity would be priceless (even if limited in scope). Hell, even Microsoft has folks working in the open source realm
Re: (Score:2)
With my limited understanding of what this is, I am kind of surprised that they are not actively helping the project.
Basically, hardware companies are, on the whole totally mental. For some reason, they have all their expertise in hardware and produce hardware for a living and then throw a total shitfit over the software and believe that their super special awful crashy piece of shit software is really the important thing and wrap it up in all sorts of proprietary licensing "solutions" designed to make life
Re: (Score:2)
Re: (Score:1)
That is what I was thinking but he does point out that they are insane. I have no evidence to the contrary, so...
Re: (Score:2)
I've never actually worked with Lattice, so they might be the rare exception. But yeah most hardware companies seem to be crazy.
Re: (Score:1)
Re: (Score:2)
In my understanding, one main reason for this secrecy with FPGAs is a kind of DRM. A lot of gadgets out there use FPGAs, and they contain some proprietary design in the form of the bitstream. If this format were open, anyone could copy and modify the design, much more easily than copying actual hardware design of a chip. Thus the secrecy is in the interest of major FPGA users, not the manufacturers themselves.
IIRC, some FPGAs even provide a kind of encryption for bitstreams, but then there's your usual D
Re: (Score:2)
What is to stop Lattice from simply shutting down this project for an open FPGA toolchain for their FPGAs?
Reverse-engineering for the purposes of interoperability is a protected activity under the DMCA (and basically all other purposes are prohibited.)
more OSS FPGA tools (Score:2)
This is cool stuff. Here's some other stuff I found recently for anyone interested in messing with bitstreams, creating an open-source FPGA, or doing hardware more easily. Hardware designers feedback is appreciated.
Open Source Bitstream Generation without R.E. or license violations: http://www.isi.edu/~nsteiner/p... [isi.edu]
Archipelago - an open-source FPGA with toolflow support: http://www.eecs.berkeley.edu/P... [berkeley.edu]
Cx, open-source, hardware & synthesis language: http://cx-lang.org/ [cx-lang.org]
QFlow Open-source Flow from beh