Sunday, 20 December 2009

Using Boost Build on your own projects

While I am a fan of SCons, every now and then I like to dabble in other build systems. One that has intrigued me for some time is Boost Build (BB). You can visit the linked site to find out more about it but in a nutshell, it is a very elegant way to build C++ software.

This post will attempt to give you some steps you can use to get started using the tool on your own projects. Note that it is a bit long but if you are new to Boost Jam as I was a few weeks back, I think it might help you get started. Please feel free to ask any clarifying questions in the comments.

In the following Boost Jam is the build tool and Boost Build is the library on top of the Jam language. I use them interchangeably, and I'm sure people will give me hell for it.

Building Boost Jam


Before getting started, I suggest you build Boost Jam as follows (might as well get Boost too!). I assume you are on a Unix system because there really is no reason to use Windows anymore ;-) but you should be able to get the same results on Windows with some slight modifications.

$ wget http://downloads.sourceforge.net/project/boost/boost/1.41.0/boost_1_41_0.tar.bz2
$ tar -xjf boost_1_41_0.tar.bz2
$ pushd boost_1_41_0
$ export BOOST_ROOT=$PWD
$ pushd tools/jam/src/
$ ./build.sh
$ export PATH=$PWD/bin.macosxx86:$PATH # substitute appropriately

Now, when you type "bjam" at the command prompt, you may get the following output:

$ bjam
warning: No toolsets are configured.
warning: Configuring default toolset "gcc".
warning: If the default is wrong, your build may not work correctly.
warning: Use the "toolset=xxxxx" option to override our guess.
warning: For more configuration options, please consult
warning: http://boost.org/boost-build2/doc/html/bbv2/advanced/configuration.html

error: error: no Jamfile in current directory found, and no target references specified.

The Jam Language


One complaint about Boost Build is that we must use the Jam language. However, it's really not so bad. While I would prefer Python, the Jam language is consistent and very simple. The main things to remember (this is my mental model and may not be technically accurate):


  • Rules are the same as functions in other languages

  • Parameters to functions are separated by ":"

  • All tokens are white space separated (use quotes to embed white space)

  • Results of functions can be used by enclosing the function call in a [] pair

  • Comments start with # and go to the end of the line



Here is an extremely simple example of a rule/function (create a file called "Jamroot" in the current directory and put in the following):

rule show-list ( list-of-stuff + : sep ) #1
{
    for local l in $(list-of-stuff) #2
    {
        echo $(l) $(sep) ; #3
    }
    return "Hello, World" ;
}

echo [ show-list 1 2 3 : "|" ] ; #4


  1. This line declares a new rule called "show-list" which accepts two parameters: a list as the first parameter and a single value as the second. Note the "+" modifier on the first parameter. This indicates to the build tool that at least one parameter is expected. You can use "*" to indicate 0 or more. I believe this can also be used to indicate optional parameters

  2. This line is a for loop using a local variable. Note the variable expansion using the "$()" syntax. In this case, each iteration of the loop will expand to an element of the list in list-of-stuff.

  3. Here we call the echo rule. Note that the line is terminated by the ";" symbol. This is required!

  4. Finally, we call the new rule with a list as the first parameter and a keyword enclosed in quotes as the second parameter. We use the result of that rule and pass it to echo.


If you execute "bjam", the output looks something like:

1 |
2 |
3 |
Hello, World

Pretty boring!

Creating a new project


When Boost Jam is invoked, it looks for a file called "Jamroot" in the current directory or in one of the parents of the current directory. This is where you define project global settings. Let's do that now. Create a new file called Jamroot and include the following contents:

import toolset ;

project app
  : requirements
    <threading>multi
    <link>static
    <warnings>all
    <warnings-as-errors>on
    # Equivalent to <toolset>darwin: <architecture>x86 <toolset>darwin: <address-model>32
    [ conditional <toolset>darwin: <architecture>x86 <address-model>32 ]

  : default-build debug release
  : build-dir build
;

The requirements state that all artifacts should be built using multi-threaded libraries, built statically with all warnings and all errors. Additionally, on darwin, we only want to build 32-bit executables for the x86 architecture.

In the default build (when you type just bjam), both debug and release variants will be built and put into the build directory relative to the Jamroot.

Now, type "bjam". If you are on OSX, you may see something like the following:

$ bjam
warning: No toolsets are configured.
warning: Configuring default toolset "gcc".
warning: If the default is wrong, your build may not work correctly.
warning: Use the "toolset=xxxxx" option to override our guess.
warning: For more configuration options, please consult
warning: http://boost.org/boost-build2/doc/html/bbv2/advanced/configuration.html
...found 1 target...

The reason for this is that BB guesses the toolset but on OSX we should really be using the darwin toolset. When Boost Jam starts up, it looks at ~/user-config.jam (somewhere similar on Windows) for a user configuration file. Add one with the following contents:

# ~/user-config.jam
using darwin ;

Now when you hit bjam, you should see something like:

$ bjam
...found 1 target...

Alternatively, if you don't want to pollute your file system, you can execute:

$ bjam toolset=darwin
...found 1 target...

Adding a project target


Now we will add a simple executable that links to some Boost libraries. Create a directory named "app" in your current directory and create a file in this directory named "Jamfile" with the following contents:

# app/Jamfile
exe app : [ glob *.cpp ] ;

Additionally, create a C++ file in the app directory with a trivial main function and the ".cpp" extension. Execute bjam. Nothing changed! That's because we haven't asked BJam to build our project. Try executing "bjam app". You should see something like the following:

$ bjam toolset=darwin app
...found 20 targets...
...updating 17 targets...
common.mkdir build
common.mkdir build/app
common.mkdir build/app/darwin-4.0.1
....
darwin.compile.c++ build/app/darwin-4.0.1/debug/address-model-32/architecture-x86/link-static/threading-multi/main.o
darwin.link build/app/darwin-4.0.1/debug/address-model-32/architecture-x86/link-static/threading-multi/app
common.mkdir build/app/darwin-4.0.1/release
...
darwin.compile.c++ build/app/darwin-4.0.1/release/address-model-32/architecture-x86/link-static/threading-multi/main.o
darwin.link build/app/darwin-4.0.1/release/address-model-32/architecture-x86/link-static/threading-multi/app
...updated 17 targets...

Note that both debug and release builds were created with one invocation. To restrict to one or the other, execute "bjam variant=debug" or "bjam variant=release".

Using Boost


Remember when we downloaded Boost? Now we will use it! The mechanism for using another Boost Jam project is the "use-project" rule. Add the following to your Jamroot file:

use-project /boost : ../boost_1_41_0 ;
alias boost_thread
  : /boost/thread//boost_thread
  : <warnings-as-errors>>off # bunch of warnings

Here we told the build system where the project with the id "/boost" is located. In this case, it is ../boost_1_41_0, relative to the Jamroot file. Yours might be different. Additionally, we added an alias for the Boost thread library. The main reason for this is that a single alias reduces proliferation of any special handling needed.

If you type "bjam" now, you will not be surprised that nothing is being built. Let's fix that now. Add the following to the Jamroot file:

build-project app ;

Now, whenever you invoke bjam, the app project will always be built. Invoke bjam now. You should notice that Boost thread is not being built. Again, this is not surprising. We aren't using it anywhere! Modify app/Jamfile to look like the following and execute bjam:

exe app : [ glob *.cpp ] ..//boost_thread ;

You will notice that Boost thread is being built in the Boost directory. This is not very useful as build artifacts are spread all over your disk. (Un?)Fortunately, there is a hack to making this work. Create a new file at the level of your Jamroot named "boost-build.jam". Fill it with the following contents:

# Add --build-dir to command line so that boost Jamfiles pick it up and use this directory to build.

ARGV += --build-dir=build ;

BOOST_ROOT = vendor/boost ;
BOOST_BUILD = $(BOOST_ROOT)/tools/build/v2 ;
boost-build $(BOOST_BUILD) ;

Boost Build looks for this file when building Boost (I think) so here we add the --build-dir parameter so that when building boost, it will build to our build directory. That's a lotta building ;-)

Hit "bjam" now. You should see Boost thread being built statically in the build directory now, in both debug and release variants.

Conclusion

In this post, you learned how to build Boost Jam, a little bit about the Jam language and created a simple project utilizing the Boost libraries. Next time, I will build on this post to cover making a plugin-aware C++ application (this is really quite exciting for me!) Again, if you have any question or comments, feel free to leave them below.