Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
October 19, 2017
arrowPress Releases






If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 

Building for Linux, the smart way

by Leszek Godlewski on 04/03/14 01:19:00 pm   Expert Blogs   Featured Blogs

9 comments Share on Twitter    RSS

The following blog post, unless otherwise noted, was written by a member of Gamasutras community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

The Linux land has a reputation, especially among developers used to Windows, of being –let's say –somewhat savage,uncivilized. We've all heardthe ghost stories: things being downrightbroken, lack of documentation and general despair; people coming, exclaiming: "what the fuck?!" and going right back.

Well, I've been exposed to the Unix philosophy, coding practices and general way of thingsfor just under 10 years. This experience makes me comfortable with the platform, butI've still had a lot to learn in the past year inmaking the transition from Linux development as a hobby to making a living out of it. I'd like to share some of the less obvious tricks I've put up my sleeve – this time regarding thebuild system.

Unless mentioned otherwise, I will assume a 64-bitDebian system, "testing" branch(this is what SteamOS is based off of).

Multilib

This section may not apply to you if you don't meet these two conditions:

  • you still need to provide32-bit binaries,
  • you are wary of the build environment provided by Steam Runtime (because e.g. you like alternating between gcc and Clang, or want to use gold for linking – more on that later).

If you're fine with 64-bit only or the SR toolchain, you may skip all the way to Clang.

If you, however, try building a 32-bit program on a 64-bit system (or the other way around) with the stock toolchain, you will most likely fail to build it, despite setting the correct architecture on it with a-m32, -m64 or a -march switch. Apparently, you are somehow missing the rightlibc and libgccpackages, even though they may seem to be installed.

The point of failure may be different, if any dependenciesapart from the standard library areused in the program. That isusually a trivial issue, though, fixed by enabling multiarch and getting the :i386package (for Debian, more on that here).

I used to maintain a schroot (kind oflike a VM, but native –the kernel and someparts of the file system are shared withthe host)with a 32-bit variant of the system just for the sake of building 32-bit binaries; however, there is a better way.

Enter gcc-multilib.Install this package and suddenly the 32/64-bit cross-compiles start working as expected. It's magic!

Clang

Okay, this one has been repeated ad nauseam recently, but it really makes a world of difference; also in terms of build times. A fullrebuild of my current project takes:

  • with gcc: 3m47s
  • with Clang: 3m05s

You may say ~40 seconds is not a big deal; it is to me.

Not mentioning the not-so-obvious benefit of having another point of reference. And, given the fact that Clang aspires for full command line compatibility with gcc and usually builds everything that gcc can (it even defines the __GNU__ macro alongside __clang__, and supports theGNU extensions to the language), it really is a drop-in replacement.

Itbrings a lot of good and can be quickly switched back for gcc. What's not to like?

Linking

This build step is a major source of grief for anyone who's ever worked with a AAA codebase, aslinking several hundred of object files and libraries can take ages. The Linux situation is made even worse by its aging linker, the GNU linker a.k.a. ld, which is single-threaded, requires objects and libraries to be specified in the order of dependencyetc.

gold

The good news is that we are not doomed to use GNUld. A few years back a small team at Google recognized the problem and decided to fix it, resulting in the development of GNU gold– a brand new linker built from scratch, whose most prominent feature is that it's multi-threaded.

Switching from ld to gold in most cases is a question of a command line switch for GCC/Clang, or at most making a symlink. Here's the command I use to link my current project's binary on a Debian system, stripped of allirrelevant options:

clang++ -B/usr/lib/gold-ld

For my current project, switching the linker from GNU ld to gold took thelink time down from 18 to 5 secondson my XeonE3-1240v3.

Library groups

Another thing the GNU ld is infamous for is the seemingly unreasonable requirement of specifying libraries in the order of reverse dependence (i.e. an object's or library's dependencies follow the object or library; there is a great explanation of that on Stack Overflow, if you're interested). This can get very annoying once you have to deal with circular dependencies, and while there is a way to do this "properly" – specify the library again – I'd much rather not have to think about it.

Turns out there arecommand-line switches that do just that (see the ld manual) – we can define a group ofarchives:

--start-group archives --end-group
The archives should be a list of archive files. They may be either explicit file names, or -l options.

Or the short-hand version:

-( archives -)

Just dump your libs within the brackets and you're set.

There is a caveat, though: this causes the linker to perform an exhaustive searchthrough all the libs in the group until all symbols are resolved. Themanual warns of this having a possible performance impact. I haven't noticed any in my project, but just keep that in mind.

Debugigng symbols

Bruce Dawson has given an excellent talk about Linux debugging at Steam Developer Days 2014, so I'm not going to repeat here what he said. Instead I'll just reiterate the two main points about symbols:

  • use objcopy --only-keep-debug and--add-gnu-debuglinkto split the debug symbols from the binary,
  • use the build IDs (GNU ld option --build-id, see manual) and a network storage to set up a symbol server.

There is one thing that Bruce has not mentioned in the talk, though, and which is – in my opinion –crucial to daily work. Update: He did suggest it to me in a tweet post-factum, though, so kudos to him anyway!

Caching the gdb-index

When working with a large codebase, such as a triple-A-gradegame engine, you may notice that the debugging symbols may be... well, somewhat heavy:

$ ls -sh ADVGame-Linux32-Debug.dbg
244M ADVGame-Linux32-Debug.dbg
$ ls -sh SuperSecretProject.dbg
254M
SuperSecretProject.dbg

Ouch.

What's even worse is that those ~250 megs need to be loaded by the debugger and processed to build an internal data structure forquick symbol lookup; with the files above, this can take 10 to 30 secondson my machine, and it happens every single time you load the binary in GDB.

There goes fast iteration time.

We can, however, cache this data structureoffline instead and fold it into the build process! I think it's a fair tradeoff between build time and runtime, considering that the latter almost always implies the index generation cost anyway.The GDB manual describes the methodwell and concisely; here's what I'm using for my project (assuming SuperSecretProject is the game binary):

mkdir -p $(OUTPUT_PATH)/gdb-index
gdb -batch -ex "save gdb-index $(OUTPUT_PATH)/gdb-index" SuperSecretProject
objcopy --add-section .gdb_index=$(OUTPUT_PATH)/gdb-index/SuperSecretProject.gdb-index --set-section-flags .gdb_index=readonly SuperSecretProjectSuperSecretProject

Some trivial variable substitution makes this look less like a magic incantation.I promise!

As David Konerding has pointed out in the comments, newer releases of the gold linker can build the gdb-index for you at link time, which should be faster and is obviously easier (no GDB batch mode commands hodgepodge!). It's now a question of appending --gdb-index to your linker command line, or -Wl,--gdb-index, if you're using the gcc or clang compiler driver to link.

Thanks, David!

Conclusions

Hopefully, at least some of these tricks have been new and useful to you. If that's the case - glad I could help! If not – why not share your tricks in the comments?

Also, I know what some developers are thinking: we shouldn't need to be doing all this, we shouldn't need to do so much research and documentation crunching just to do such basic things. And you're probably right. I'd like to explore this non-orthogonality betweenthe Unix philosophy and the game developermindset in a future blog post.


Related Jobs

2K
2K — Novato, California, United States
[10.18.17]

SENIOR SERVER ENGINEER
Aurora44
Aurora44 — Wellington, New Zealand
[10.18.17]

Senior Programmer
Firebrand Games
Firebrand Games — Merritt Island, Florida, United States
[10.18.17]

Programmer
Firstborn
Firstborn — New York, New York, United States
[10.18.17]

Game Developer





Loading Comments

loader image