Friday, April 4, 2008 at 10:43 AM
I submitted my first patch to the GNU linker in 1992. In 1993 I knew enough about its shortcomings, and started to argue that we should rewrite it from scratch. In 2006, just 13 years later, and now working at Google, I finally got a chance to do it.
That work came to fruition last month, with the release
of gold to the open source community. gold is a new linker, written from scratch. It really only has one new feature compared to the current GNU linker: it's much faster. I've measured it as five times faster linking large C++ applications. Since for most programmers the linker is nothing more than a roadblock between writing code and running the program, I figure that speed and correctness are the only really important features of any linker.
We've been using gold widely within Google as a beta test, and will soon be deploying it as the default linker internally. Now that it is part of the GNU binutils, my hope is that it will eventually become the default linker for all the free operating systems.
Getting gold to this point was the work of a small team at Google. It's a reasonably sized program — some 50,000 lines of commented C++ code. We're contributing it back to the open source community not because we have to, or because we expect some benefit, but simply because we can. Google gets a lot from the open source community, and we try to give a lot back.
19 comments:
Is gold only available in binutils? Is there an expected release date? I'd very much like to test it out on the Parrot project.
Thanks.
Yes, gold is only available in the binutils. There isn't a currently planned release date. You can build it yourself by downloading the binutils source code and configuring with --enable-gold.
Thanks, Ian! You rock!
Pass it on to your fellow google employees-- you're all absolutely awesome.
what are the chances of this being ported and included with MinGW? windows is usually the slowest to link when using gcc
gold is ELF only, and will never be ported to mingw, at least not as a native linker. Sorry about that.
ELF (used on GNU/Linux, Solaris, and most free operating systems) and PE/COFF (used on Windows) and Mach-O (used on MacOS X) are all quite different. Writing a linker which works for all of them requires a number of compromises, which is part of why the GNU linker is so comparatively slow (the GNU linker does not support Mach-O, but it does support both ELF and PE/COFF, among others). In order to maximize gold's performance, I only focused on ELF.
Congratulations. How about a little bit of technical detail about what's different in your approach and the old linker?
A bit confused
latest binutils is august 2007
this announcement is 4. april 2008
how can --enable-gold in latest binutils work??
anyway, nice to see that, will surely test speed differences the coming time :)
If you are writing this directly for GCC it is most definitely a donation to the free software community and not the open source community.
Sorry to nitpick.
Very cool!! Building the Linux kernel with gold will definitely be a huge win. Does it speed up kernel compiles?
5 times faster? can you back that up with some simple stat graph for the rest of numerically challenged? also which version of "current" gnu linker did you specifically compared against?
Thanks
very nice Ian, and congratulations.
+1 for google
Very cool!
Btw, am I correct that the '-u' flag (force linking a symbol) is not yet supported in gold? Are there any plans to support this?
ralph: The main difference in gold is that it was designed from the ground up to work for ELF. The GNU linker was designed to work for a.out and COFF.
ELF conceptually requires three passes over the object files, a.out and COFF require two. A version of the third pass was hacked into the GNU linker by using a backpatch system on the symbol table, in which the GNU linker makes some decisions when it first reads the object file, and then undoes those decisions when appropriate after seeing all the objects. The backpatching causes the GNU linker to traverse the symbol table multiple times; this is very expensive in a large link. Reducing the number of symbol table traversals is probably the most significant change.
A couple of smaller but significant changes can be found on my blog:
Multi-threading.
C++ templates avoid byte swapping.
markus: I meant to suggest that you download the development version of the binutils from a snapshot or via CVS. You are correct that gold is not (yet) in any released GNU binutils.
On "open source" vs. "free software": this is the Google "open source" blog so it seems less confusing to use the term "open source". And regardless of your position on the terminology, gold is clearly a contribution to both the "open source" and "free software" communities.
adamm: The kernel build is dominated by compilation time. Also, most kernel links use linker scripts, and gold is optimized for the case of not using a linker script. So I'm afraid that gold does not provide a significant speedup for kernel builds. Sorry.
layer3switch: I don't have a graph, sorry. For a specific example, for a large C++ application, I measured the GNU linker as taking 1 minute 11 seconds (wall clock time) while gold took 13 seconds. That comparison was done against a binutils snapshot between 2.17 and 2.18 (070626) with a local patch to speed up the GNU linker for large C++ programs (increasing the size of one of the hash tables, the one used for section merging, in bfd/merge.c).
remko: you are correct that -u is not yet supported. Yes, we plan to support this and all the other commonly used linker options. Naturally, patches are welcome!
Post a Comment