Video ARM Code for Raspberry Pi Goes Open Source (Video) 91
Slashdot: Alex, we're talking today about open sourcing parts of the
code that run the Raspberry Pi, simply the BCM2835 chip. Can you talk a little bit about what that chip does?
Alex: Sure. So the 2835, that's the core system-on-chip, which is comprised
essentially, the whole of the Raspberry Pi in a sense, in that you've got on there the ARM, the GPU,
and that the various libraries that are used to access that GPU by OpenGL-ES or OpenPG.
Slashdot: And how much closed-source code is actually used to run
the Raspberry Pi right now? In other words, how much is not already open source? I think a lot of people are under the
misimpression that the Raspberry Pi is an entirely open source project because it is so Linux centric and there's been so much open source involvement.
Alex: Yeah. Certainly we had, I mean, a substantial amount of it is open source. The Linux kernel module, which we use to communicate with the GPU, is fully open-sourced, GPL and BSD licensed. And the -- what we're open sourcing now are the libraries which run on the ARM side, the GL, the GL ES implementation, the OpenVG implantation, VGL, and so on. So really, how the architecture of the video core works is that there is some code that's running on the video core side that is quite proprietary, and that's unlikely to change in the near future, but what we've now been able to do is open up everything which runs on the ARM side. So everything which runs from Linux kernel up is now open source, essentially.
Slashdot: And is there new functionality that will -- either immediately or that you anticipate will -- be opened up by having this, the ARM side of the software stack being open source?
Alex: I think that in the near future it's going to make it a lot easier to
get things like native Wayland implementations working because of the EGL implantations being opened up, which allows
programmers to implement the Wayland platform requirements using the toolkits which we already have available. I mean, in general, I think it's going to make it much easier to work with the existing open source Linux stack, because quite understandably most people working graphics on Linux, it's all very centric around MESA and Gallium, the existing open source projects. It's very difficult to work with libraries which are closed and which they're constantly adding features to. Well, we've solved that problem now, and hopefully we'll start to see more people porting that technology to use Raspberry Pi.
Slashdot: Now you mentioned the GPL and BSD licenses are used extensively. What license is the new code or anything that's being open sourced now, is it all going be under the GPL?
Alex: It's all 3-clause BSD.
Slashdot: Okay.
Alex: So the reason, so the stuff we use in the kernel module is BCHIQ,
which is basically the machinery required to create a layer between the user space graphics driver implementation and
the video core. So that's actually dual GPL and BSD, to be friendly to the BSD guys. Now that we've got all the user space stuff also BSD, which is a very, very permissive license, we anticipate it will be very useful to people who are wanting to port -- other people who are working on FreeBSD, NetBSD, Haiku, Plan 9, RISCOS, etc.
Slashdot: Has there been, to your knowledge, much sort of cross pollenization of the existing code that's under that license?
Alex: How do you mean by cross pollenization?
Slashdot: Have any of those projects you just named, have they been interacting with Raspberry Pi developers? You just named several big . . .
Alex: Oh yeah. We have had that to a certain extent, certainly reports on . . .
they've been working with us on further developing through the API for doing low-level graphics, they've tried to build, you know, bring up their platforms. We've actually started to see patches reaching the mainline Linux kernel to start to provide platform support for the Raspberry Pi. We have a ways to go there, but that's something we're keen on pushing. I think we'll start to see much more collaboration with those projects with this open-source code. And we've also seen people who've been [hacking] on all the various device drivers, USB in particular. There's a Plan 9 guy, he got pretty far in doing a crash-free implementation implementation of the USB driver, which
is going to be interesting to a lot of people because it's currently rather large and difficult to do.
Slashdot: Now, one thing, the term "open source" can mean a lot of
things. In what way will this code that you're talking about open sourcing now on the ARM side of the stack be accessible to either developers, other programmers? Is it going to be, say, in
a public facing Git repository? Or will it be distributed on request? How will the Raspberry Pi project get that open source goodness out to other people?
Alex: Yeah, totally, fully, 100% OSI, FSF compliant, open source Free Software. So it's available under a BSD license. We have a GitHub repository -- do your standard "Clone it, send your patches," or pull
requests or whatever. Exactly what we do regarding opening up the development process, that's always quite difficult for us because when you have people who are contributing in their spare time, working on things, they don't always want to make promises they can't keep. So that's quite difficult too. We have to
strike a balance there between talking about what we're actually working on and making sense of people's disappointment.
Slashdot: Now, Alex, Eben Upton basically described you as the
project's Linux guru. Is there a team of people involved, both inside and outside Raspberry Pi, in making this open-source release happen? Or how many people are involved?
Alex: There's a small handful who've been involved in this open-source
release particularly. I mean, the main issue as with most of these open-sourcing efforts, is, of course, working through all the necessary legals, persuading the people who matter that actually there's not going to be a really big downside for not releasing access. They're not good at giving away valuable IP which is
going to downturn market position. So I think most of the work's being done with people like Eben, talking with Broadcom executives. Beyond that, there is a small handful of protocol engineers who then very generously give their time, and they've done the necessary legwork on cleaning up the code and making sure that it's ready for public distribution.
Slashdot: From that, it doesn't sound like you've faced a whole lot
of resistance in opening this code. Is that true?
Alex: Well, my understanding is that it's something which has taken a long
time. So it's been a slow process, but I think that it's partially due to the way that the video core is structured,
whereby most of stuff which runs in user space is pretty much just a serialization of your GL request and then it sends it over to the video core. So there's still plenty of stuff on the video core side, which remains private. I think that structure makes it more palatable.
Slashdot: You describe this as a process, and it's obviously got
both the actual development work, with the programming, coming up with open-source alternatives to closed-source options in some cases, and the licensing. What is the timeline going forward? Is this announcement, is there going to be a big 'We've open sourced all the ARM stuff" that is going to hit users immediately? If someone has a Raspberry Pi already, will they in the month of December be downloading a patch? What goes on for users from this point? For developers as well, for that matter?
Alex: Okay. Well, the big announcement is going to be on Wednesday, and the
GitHub repository will go public and everybody can clone it to their heart's content and get hacking. I would say that there might not be an immediate difference to users. But what we'll see over the next few months is better integration with the open source with its graphics stack.
Slashdot: Okay. What's the most exciting aspect of this?
Alex: Well, for me, I think that this is a major announcement in that this is the first [open-source driven] SOC. Broadcom was the first vendor to fully open source their user space graphics drivers. So I think its a pretty major step for embedded Linux.
People have been campaigning for, or begging for, for Mali
drivers, the Adreno drivers and so on. So I hope that this is a step that other companies will look at it, they'll see the
benefits which we're hoping to accrue over the next few months, and hopefully we'll start to see others following in our
footsteps.
Re:Count me stunned (Score:5, Informative)
Hiding under the video is a "Hide/Show Transcript" link that displays a full transcript if you can't watch the video (or just prefer reading).
Looks like the heavy lifting is done elsewhere (Score:5, Informative)
Haven't had a chance to wade through all the code here, but it looks an awful lot like it's basically rpc stubs and glue (allocator, buffer management, etc) to remote the OGLES calls to the media processor, which presumably handles all the actual translation of OGLES API to whatever the internal architecture of the GPU looks like. Which is a perfectly reasonable approach, just not necessarily 1:1 with the other SoCs (which don't have a GPU block capable of speaking remoted OGLES, thus requiring knowledge of the underlying hardware "secret sauce" in the GL libraries or drivers on the host side).
Awesome that they're releasing this as open source rather than insisting on keeping it closed -- much easier to work with this way.
Re:Looks like the heavy lifting is done elsewhere (Score:5, Informative)
That's a fair observation. We're lucky that, with firmware installed, the VideoCore GPU exposes a much higher-level interface to the ARM side than some other architectures. This has allowed us to provide a viable, useful open source stack while also addressing Broadcom's (completely legitimate) interest in protecting the fine detail of the underlying IP from competitors and patent trolls.
To help people get the most out of this, we hope to sponsor some efforts to formally document the interface exposed by the GPU over the next few months.
Re:Looks like the heavy hiding is done elsewhere (Score:5, Informative)
That was our feeling. This model is an order of magnitude easier to sell to the chip vendor, and provides all the benefits normally associated with open source drivers (provided the interface is adequately documented, which it will be in the near future).
Re:I'm confused (Score:3, Informative)
You're correct. The VideoCore GPU exposes a fairly high-level interface to the ARM side, so passing messages *is* the interface. We're lucky that VideoCore provides for a good tradeoff between the Broadcom's legitimate desire to maintain a degree of secrecy around the register-level operation of the hardware (read concern about competitors and patent trolls) and FOSS users' need for source code access and control.
Re:Interesting, interesting... (Score:5, Informative)
Actually, if you look at Broadcom's recent behaviour around Bluetooth drivers, I think there's good (public) evidence of a sea change in the attitude towards open source.
Re:Looks like the heavy lifting is done elsewhere (Score:5, Informative)
Apart from purists who want to have source for every programmable block on the SoC, everybody wins.
That's my hope. My issue with the purists is that it's not obviously clear why they want to see the microcode running on a proprietary RISC core inside the GPU, but not for example the Verilog. Stallman is one of the few people who has a self-consistent model of what he wants to be able to see, arguing that code which is "equivalent to a circuit" (i.e. in ROM) need not be made visible. Now we don't meet this criterion as our microcode lives on the SD card, but that's largely a cost and flexibility issue and we may yet go there to get the FSF endorsement.
From one point of view the cost to Broadcom to making this open source is not nearly the same as for the other GPU vendors -- I suspect this RPC glue is not among the crown jewels of Broadcom's IP
I should have kept some of my notes from those meetings :)
Does this (or will this) support future / higher end parts using the same VideoCore architecture? It definitely increases my interest in the BCM SoC family if so...
While I can't commit to this, I'm certainly a very vigorous advocate for this position, from a commercial and a community relations standpoint. Fingers crossed.
Re:Count me stunned (Score:5, Informative)
A lot of people have been complaining very loudly about this, because they announced that they had ICS running two months ago and there's been nothing released since, and the community has been unable to assist because not only has there not been any source release, but they haven't even released the videocore binary. Well, here's the sources needed to make it happen.
For the last month or so, I have been advocating the alternatives to the R-Pi specifically because the graphics driver has not only been closed, but has had an inadequate release schedule. This release should address that issue nicely.
Re:Extremely Good News (Score:4, Informative)
Now the question is how much of the work is done by the firmware on the GPU rather than the driver running on the CPU?
Easy answer - the majority of the intricate register-level work is done by the firmware, rather than by the driver. This is how we've been able to reconcile the legitimate interests of Broadcom in keeping the fine detail of the implementation private, while also providing a workable FOSS driver suitable for use in (for example) porting other operating systems to the Pi.
Re:Count me stunned (Score:4, Informative)
Also, maybe the chineese promissing* that the A10 will have an entirely free stack helped a bit on this decision.
* As far as I know, they still didn't deliver it... But just the promisse should be enough to change Broadcom's strategy.
yes, that's the whole point. you play one company off against the other. the first one that *actually* goes and releases full GPL-compliant source code of their 3D GPU for example, i will INSTANTLY be recommending it to our clients. our clients are PRC State-Sponsored companies: one of them has a production capacity of 20 million units a *week*.
regarding the A10: *sigh* yeah i know. they can't actually release the source code of MALI, because that's locked down by ARM playing silly-buggers, including deleting public requests on ARM's forums for them to release the source code, *and* despite loads of ARM employees repeatedly advising ARM that releasing the source code is in ARM's best interests.
so we have to rely on the limadriver project, basically, which is making good progress.
we know that Allwinner made a promise to look at releasing the source code of the CedarX audio/video engine, but again, there, i think there will be more mileage out of reverse-engineering it. a "wrapper" has been written which traps system calls, giving a clear idea of what's going on.
the last part, the DDR3 "setup" phase, has already been reverse-engineered. it was a few hundred lines of assembler, that's all. so, the boot process is at least entirely free software.