Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
×
Firefox News

Firefox Too Big To Link On 32-bit Windows 753

An anonymous reader writes "Firefox has gotten so large that it cannot be compiled with PGO on a 32-bit linker anymore, due to the virtual memory limitation of 3 GB. This problem had happened last year with 2 GB, which was worked around by adding a/3GB switch to the Windows build servers. Now the problem is back, and things aren't quite that simple anymore." This only affects the inbound branch, but from the looks of it new code is no longer being accepted until they can trim things from the build to make it work again. The long term solution is to build the 32-bit binaries on a 64-bit system.
This discussion has been archived. No new comments can be posted.

Firefox Too Big To Link On 32-bit Windows

Comments Filter:
  • by bobcat7677 ( 561727 ) on Wednesday December 14, 2011 @12:46PM (#38372250) Homepage
    And this would be the point where the fact that the Firefox devs have been trying to do too much with a "browser" becomes beyond blatantly obvious. Firefox team: get your stuff together or you will die a slow death of attrition.
    • by dn15 ( 735502 ) on Wednesday December 14, 2011 @12:57PM (#38372428)

      And this would be the point where the fact that the Firefox devs have been trying to do too much with a "browser" becomes beyond blatantly obvious.

      Firefox is a great operating system, if only it included a decent browser :P

    • by jlebar ( 1904578 ) on Wednesday December 14, 2011 @01:11PM (#38372676) Homepage

      Chrome doesn't even try to build with PGO, last time I checked.

      http://groups.google.com/a/chromium.org/group/chromium-dev/browse_thread/thread/533e94237691e2f6 [google.com]

    • by Grizzley9 ( 1407005 ) on Wednesday December 14, 2011 @01:12PM (#38372688)
      I'd already have switched to Chrome if not for FF's Live Bookmarks. Reading a bunch of different sites, multiple times a day it makes for an easy scan of story titles to see if anything is worth looking at. Otherwise I'd switched long ago and was sad Chrome didn't come with it. I don't want to manage a dedicated app.
    • by maztuhblastah ( 745586 ) on Wednesday December 14, 2011 @01:13PM (#38372700) Journal

      An excellent takeaway from this article.

      Unfortunately, it's completely incorrect. TFA is talking about the build process on a 32-bit host, specifically that VS builds using profile-guided optimization require more memory than is available in the address space *DURING THE BUILD PROCESS*, not an issue encountered by the resulting binary.

      I know you want a chance to get in a quick dig at Firefox, but this isn't the article for that.

      • by lpp ( 115405 ) on Wednesday December 14, 2011 @01:21PM (#38372842) Homepage Journal

        I think it's still indicative of the problem GP mentions. The more code you are trying to pull in, the larger the footprint during the build process. You don't see a 'Hello world' program requiring a 3GB+ build footprint do you? No, because it's not doing enough to warrant that. Likewise, Firefox apparently *is* trying to do a lot. More than it used to at any rate.

        • by maztuhblastah ( 745586 ) on Wednesday December 14, 2011 @02:03PM (#38373576) Journal

          I think it's still indicative of the problem GP mentions. The more code you are trying to pull in, the larger the footprint during the build process. You don't see a 'Hello world' program requiring a 3GB+ build footprint do you? No, because it's not doing enough to warrant that. Likewise, Firefox apparently *is* trying to do a lot. More than it used to at any rate.

          Well you're right in that Firefox does need a hell rather large amount of RAM to build... but it's not just them; all browsers are trying to do a lot nowadays.

          Chrome doesn't exactly have light build requirements either. In fact, the Chromium project already seems to have dropped 32-bit build environments:

          A 64 bit OS is highly recommended as building on 32 bit OS is constantly becoming harder, is a lot slower and is not actively maintained.

          (From "Build Instructions (Windows) - Build Environment" [chromium.org])

          That's why I think that the parent poster's implication that it's due to Firefox becoming "bloated" is basically hogwash. Browsers are more complex than they were in the mid-90s. That's what happens when you add 10+ years of new formats and technologies that must be supported for a browser to be considered "usable". Directing one's ire at Firefox is unwarranted, IMHO.

      • by Unoriginal_Nickname ( 1248894 ) on Wednesday December 14, 2011 @01:28PM (#38372960)

        Thank you for posting this. Filling the virtual address space of the linker probably does indicate some problems with the Firefox source code - crazy big translation units, for example - but it doesn't imply anything about the size or quality of the resulting binary.

        I thought people on this site were supposed to know something about computers.

        • by BZ ( 40346 ) on Wednesday December 14, 2011 @01:44PM (#38373248)

          The "translation unit" involved here is the whole binary. We're talking about link-time code generation with profile-guided optimization, not regular compiles.

          So it doesn't indicate much of anything about the Firefox source code other than general overall quantity of code being compiled...

    • by BZ ( 40346 ) on Wednesday December 14, 2011 @01:42PM (#38373198)

      Define "too much"?

      It's been over a year since Chrome had to turn off PGO altogether and move to 64-bit builders even without PGO, because they ran into this same problem.

      So maybe your issue is with the fact that all "browsers" as you call them are trying to do too much? They should drop the fast graphics and jits and video and audio support and all that jazz, right?

  • by Anonymous Coward on Wednesday December 14, 2011 @12:47PM (#38372264)

    E.g. where's the status bar in recent firefoxes?

    • by cruff ( 171569 ) on Wednesday December 14, 2011 @12:51PM (#38372332)
      I immediately added the Status-4-Evar addon to my Firefox installations, works great.
    • by webmistressrachel ( 903577 ) on Wednesday December 14, 2011 @12:58PM (#38372446) Journal

      If Seamonkey adopt this silly fade-in fade-out floating toolbar-as-status-bar-replacement, that'll be the end of the Mozilla line for me. Seamonkey has always been the sensible browser for Netscape-heads from way back when since Mozilla Suite died a death and SeaMonkey came into being.

      • by Cinder6 ( 894572 )

        What do you plan to use otherwise? Chrome does the same thing, so you're stuck with IE and Safari.

        Honestly, I cannot fathom what is preferential about an always-open status bar. For me, the status bar was always of such situational use that the first thing I did on a new browser install was disable it. Having it auto-hide is a much better choice. It's there when you need it (which is to say, rarely), and not there when you don't (which is to say, most of the time). I guess if you really want to see a p

        • by ceoyoyo ( 59147 ) on Wednesday December 14, 2011 @03:20PM (#38374844)

          The other day a coworker walked in to ask me what I use as my browser. I said Safari. He asked why not Chrome. I told him I had Chrome installed, and used it occasionally. I couldn't remember exactly why I prefer Safari, so I started both of them up. Safari was done launching in a couple of seconds, almost before I had time to click on Chrome. We continued our discussion of why I don't use Chrome while waiting for it to launch.

          Checking the .app size... Safari is 35 MB, Chrome is about a quarter of a gig.

  • by Trepidity ( 597 ) <{delirium-slashdot} {at} {hackish.org}> on Wednesday December 14, 2011 @12:48PM (#38372272)

    Size of the Firefox codebase is one factor of course, but the amount of RAM needed by Visual Studio to compile code with all optimizations turned on (especially PGO, which is extra RAM-intensive at the compilation stage) is also a major factor. Notice that this only happens in the 32-bit Visual Studio builds specifically.

    • Re: (Score:3, Informative)

      That's just doesn't make any sense. The amount of RAM is not a factor at all. The problem at hand is caused by the amount of _address_ _space_ required, not the amount of RAM. That's a completely different issue. I hope you understand the difference between the RAM and the address space. And the culprit is not VS or Microsoft compiler. The culprit is the rampant massive-scale global namespace pollution, which has been taking place in FF source code for quite a while. Someone was indulging recklessly in Lin
    • by 3p1ph4ny ( 835701 ) on Wednesday December 14, 2011 @02:25PM (#38373892) Homepage

      Sure, except that (especially in C++ code with templates) VS uses FAR less memory than the GNU toolchain when compiling the same code. This isn't a VS problem, it's a Firefox problem.

      • by tibit ( 1762298 ) on Wednesday December 14, 2011 @04:46PM (#38376178)

        Nope. It is the MS linker problem. It's not about memory usage, it's about stupidly memmapping ALL of the input files during startup. So it's very simple to check if you may have a problem: add up the size of all the .obj files. If it's above 1-1.5G or so, it won't fit as linker needs address space for its own transient data and you need to boot with the /3g switch. If it's above 2G or so, then even the /3g switch won't help you -- you need a 64 bit host.

  • by Pope ( 17780 ) on Wednesday December 14, 2011 @12:49PM (#38372312)

    Firefox devs requesting immediate RAM bailout.

  • Big deal (Score:5, Funny)

    by GameboyRMH ( 1153867 ) <gameboyrmh AT gmail DOT com> on Wednesday December 14, 2011 @12:52PM (#38372354) Journal

    It takes 16GB to compile Android.

  • by HellKnite ( 266374 ) on Wednesday December 14, 2011 @12:54PM (#38372402)

    "First tests indicate that, for example, moving parts of the WebGL implementation to one side could save 300 KB. In a test run, the newer version of Visual Studio required less memory than the one that was previously used, and 64-bit Windows offers 4 GB of address space."

    So, first of all, saving 300KB on WebGL seems like a pittance. Then, there's what appears to be the blatantly incorrect statement of 64-bit windows offering 4GB of address space - shouldn't that be way bigger, or am I stupid?

  • by msobkow ( 48369 ) on Wednesday December 14, 2011 @12:59PM (#38372462) Homepage Journal

    It sounds like a build-tool chain problem, not an issue with the eventual binary produced for Firefox.

    Why not just run the 64-bit tools on a 64-bit platform and have them output 32-bit code, the same as can be done by virtually any cross-platform compiler system. I can't imagine worrying about the fact that I can't run the builds on an outdated 32-bit OS as long as I can produce the binaries for such platforms.

    I shudder to think what it would have been like to develop for some of the military communications systems I worked on for my first job if we had to run the compilers on that pathetically slow mil-spec Sperry AN-UYK system (magnetic core memory -- talk about slowing down the CPU! But it's radiation hard!) We only tested on the Sperry -- all builds and editing were done on a VAX.

    In modern terms, could you imagine having to run your editors and compilers on an iDevice instead of OS/X?

    • by tepples ( 727027 )

      In modern terms, could you imagine having to run your editors and compilers on an iDevice instead of OS/X?

      That depends on how long Apple will continue to sell the MacBook Air instead of replacing it with a high-end iPad, but the purported death of the PC is another topic for another day.

    • by BZ ( 40346 )

      > Why not just run the 64-bit tools on a
      > 64-bit platform

      There is no 64-bit version of the MSVC linker, and no plans for one according to public statements from Microsoft.

      One can run the 32-bit linker on a 64-bit Windows, which gives you 4GB of address space to play with. That's the medium-term solution for Mozilla, but it does require updating the entire Windows build farm to 64-bit Windows and shaking out any compat issues that result. Doable, but will take a few weeks probably.

  • by jlebar ( 1904578 ) on Wednesday December 14, 2011 @01:03PM (#38372526) Homepage

    Summary should read: Visual Studio is too teh suck to link Firefox on Windows with PGO.

    Firefox links just fine with VS, if you don't use PGO. The problem is that Visual Studio's PGO routine loads all our object files in at once, then uses a ton of memory on top of that. And the linker for 32-bit programs is itself a 32-bit program; if there were a 64-bit x86-32 linker, we wouldn't have this problem. But so far, Microsoft has not given any indication that they will release a 64-bit to x86-32 cross-compiler.

    Note that Chrome doesn't build with PGO at all, last time I checked.

    http://groups.google.com/a/chromium.org/group/chromium-dev/browse_thread/thread/533e94237691e2f6 [google.com]

    Note: Visual Studio 2010 seems to help a bit, but not much. We use VS 2005 because it's the last version whose CRT supports Windows 2000 and Windows XP before Service Pack 2.

    • by thsths ( 31372 )

      > We use VS 2005 because it's the last version whose CRT supports Windows 2000 and Windows XP before Service Pack 2.

      That's all very nice, but if you run an obsolete and out of support OS, why would you want to run the latest browser? Those users should be more than happy with Firefox 3.6.

    • by askldjd ( 857313 ) on Wednesday December 14, 2011 @01:23PM (#38372874)
      Parent is right. The software product I work on also used PGO at one point. Our software is roughly 2-3 million lines of C++ code, and links with 40-50 external libraries. Under Window XP, VS2008 could not link our product with PGO turned on. Before you complain about the bloat of FF, please understand that PGO uses many many times more memory than just compiling a regular release build. This is a Visual Studio linker problem, not FF problem.
  • Not FF's fault (Score:4, Informative)

    by Anonymous Coward on Wednesday December 14, 2011 @01:07PM (#38372592)

    The software product I work on also used PGO at one point. Our software is roughly 2-3 million lines of C++ code, and links with 40-50 external libraries. Under Window XP, VS2008 could not link our product with PGO turned on.

    Before you complain about the bloat of FF, please understand that PGO uses many many times more memory than just compiling a regular release build. This is a Visual Studio linker problem, not FF problem.

  • by Okian Warrior ( 537106 ) on Wednesday December 14, 2011 @01:12PM (#38372692) Homepage Journal

    Back in my day we didn't have gigabyte memory and disk space was at a premium.

    It might seem a bit strange now, but back in the good 'ol days we used to have to break up a project into separate components, just in order to compile it!

    This is where your interface and API design skills came in handy. If you could partition some piece of the project off into it's own DLL, you could effectively break up the project into smaller pieces that could be individually compiled.

    That's where the name DLL came from originally: "Dynamic link library". You didn't need to have all the code read into memory when you first executed the application - less commonly used features wouldn't get loaded until they were actually asked for.

    It's not like it is nowadays, where you actually need all the code to be available all the time. "Rich user experience" they call it.

    I suppose it's just the future overtakin' us. Them good old days is gone forever.

    • by jlebar ( 1904578 ) on Wednesday December 14, 2011 @01:24PM (#38372894) Homepage

      It might seem a bit strange now, but back in the good 'ol days we used to have to break up a project into separate components, just in order to compile it!

      Yep, we used to do this. Then we merged them together, because it greatly improves the startup speed of Firefox.

      We have a lot of smart people working on this stuff, believe it or not.

  • by caseih ( 160668 ) on Wednesday December 14, 2011 @01:49PM (#38373332)

    Folks are crawling all over each other to show how ignorant they are. "I ditched firefox for Chrome cause it's lighter!" only to ignore the fact that Chrome also has the same problem with the PGO thing running out of RAM so they don't even bother with trying those optimizations anymore. "Geez Firefox needs more RAM than the kernel to compile? Something's wrong!" Yes if the Linux kernel was built with PGO on VS 32-bit it probably would run out of RAM there too. Then there's the guy that claims this PGO problem is evidence that the Firefox devs need to go back to remedial school. I'm sure he could do it far better (and avoid the PGO linking optimization running out of memory too!).

    Hilarious reading. At least I choose to laugh rather than cry at people's inability to read and understand the issues here.

    • by ledow ( 319597 ) on Wednesday December 14, 2011 @01:59PM (#38373494) Homepage

      I think you miss the subtext that some of us have. If you HAVE to enable PGO in order to get a decent speed out of your binary in the first place (and that's the ONLY way you think you get that increase), then your code isn't that great in the first place (i.e. you are using lots of inefficient methods on the most-used paths of your code).

      The problem isn't DIRECTLY related to the size/quality of the codebase but the fact that they don't even CONSIDER turning off PGO because of the performance drop means they have no idea how to tune the underlying code without using PGO (and PGO-optimised code will NOT necessarily result in the best possible code for any particular user at all!)

  • by pslam ( 97660 ) on Wednesday December 14, 2011 @02:28PM (#38373972) Homepage Journal

    This is basic stuff to anyone who actually maintains a build, but Slashdot hasn't been a forum mostly populated by engineers for a number of years, now.

    This appears to be due to link-time optimization blowing up the resident memory size of the linker, taking it past 3GB (which is already a non-standard hack the 32 bit build has had to do). Firefox is large, yes, but this has nothing to do with the final binary - which appears to be about 100MB total including all libraries in the Aurora builds.

    I used to routinely run out of 32 bit address space compiling executables for a 64MB embedded ARM platform. This was due to symbol bloat, not executable size (which was 8MB). I also ran out of space compiling for a DSP with 288KB of RAM and 1MB flash, but that was mostly piss poor tools (Tasking). In fact, doesn't Chrome and even the Android sources already require building on a 64 bit host?

Seen on a button at an SF Convention: Veteran of the Bermuda Triangle Expeditionary Force. 1990-1951.

Working...