a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis

Zutubi

C++ Unit Testing With Boost.Test

Recently I implemented a Pulse plugin to read Boost.Test reports and integrate the tests into the build results. As usual, the first step in implementing the plugin was the creation of a small project that uses Boost.Test to produce some real reports to work from. I found the Boost.Test documentation to be detailed, but not so easy to follow when just getting started — so I thought I’d give an overview here.

Step 1: Installation

First you will need to install Boost.Test, or possibly all of Boost if you plan to use more of it. You can download Boost in its entirety from the Boost download page. Then you just need to unpack the archive somewhere appropriate, so you can include the headers and link against built libraries (see the next step).

An even easier option if you are on Linux is to install a package. On Ubuntu (and, I expect, other Debian variants), the desired package is libboost-test-dev:

$ sudo apt-get install libboost-test-dev

The downside of this is the packages are somewhat out of date, the default being built from Boost 1.34.1 (there is also a 1.35 variant available). I have not seen much impact of this when using Boost.Test, but if you need newer Boost libraries then it may be better to compile your own.

Step 2: Choose Your Compilation Model

Unlike many Boost libraries (which are implemented completely as headers), Boost.Test includes a runtime component which you need to link against: the “Program Execution Monitor”. This component includes the main entry point for running your tests, among other things. If you installed Boost from source, you will need to build the library yourself using bjam — the instructions are quite toolchain specific so I won’t go into them here. You can link statically or dynamically, but will need to configure your includes and build appropriately.

The key thing from the point of view of writing and building your tests is to include the right definitions in your source and add the right flags when linking. I opted for dynamic linking against the prebuilt library installed by my Ubuntu package. To achieve this, I needed two things:

  1. To define BOOST_TEST_DYN_LINK before including the Boost.Test headers in my source file.
  2. The addition of: -lboost_unit_test_framework to my linker flags.

With that plumbing out of the way, we can get down to testing something.

Step 3: A First Test Case

For a start, I cooked up an exceptionally useful function to add two ints, and a test case “universeInOrder” to check that it works:

#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MODULE Hello
#include <boost/test/unit_test.hpp>

int add(int i, int j)
{
    return i + j;
}

BOOST_AUTO_TEST_CASE(universeInOrder)
{
    BOOST_CHECK(add(2, 2) == 4);
}

Notice that apart from the BOOST_TEST_DYN_LINK definition, I also had to define a name for my test module via BOOST_TEST_MODULE. The case itself is defined using the BOOST_AUTO_TEST_CASE macro, giving the case name as an argument1. Finally, within the test assertions can be made the BOOST_CHECK macro. Compiling and running the test gives the following:

$ g++ -ohello -lboost_unit_test_framework hello.cpp
$ ./hello
Running 1 test case...

*** No errors detected

Simple enough, my test passes. If I deliberately make it fail, by changing the 4 to a 5, I get:

 $ ./hello
Running 1 test case...
hello.cpp(12): error in "universeInOrder": check add(2, 2) == 5 failed

*** 1 failure detected in test suite "Hello"

Here we start to see the benefits of the library: I get a nice failure message, complete with line number and the expression that failed.

Step 4: More Assertions

Unlike the assertions in many testing libraries, a failed BOOST_CHECK will not exit the test case immediately — the problem is recorded and the case continues. To immediately fail a test, you can use BOOST_REQUIRE instead:

BOOST_AUTO_TEST_CASE(universeInOrder)
{
    BOOST_REQUIRE(add(2, 2) == 4);
}

To just output a warning instead of failing the test, you can use BOOST_WARN. In fact many Boost.Test assertions come in these three variants: CHECK, REQUIRE and WARN.

Richer assertions are also possible, including these notable examples:

  • BOOST_CHECK_MESSAGE: allows you specify a custom failure message as a second argument. You can pass a string, or any type supporting the << operator.
  • BOOST_CHECK_EQUAL: checks two arguments for equality using ==. Improves upon the normal check in the above examples by showing the actual values when the assertion fails.
  • BOOST_CHECK_THROW: checks that an expression causes a specified type of exception to be thrown.

The full list of available assertions for the the version of Boost.Test I am using (1.34.1) can be found here.

Step 5: Suites

Once you have a non-trivial number of test cases, you need to organise them into suites. Note that each module (defined with BOOST_TEST_MODULE) already has a top-level suite named after that module. Further suites can be nested within a module to categorise as necessary. The easiest way to do this is to continue with the auto-registration model, and simply wrap the test cases with new macros to start and end a suite:

#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MODULE Suites
#include <boost/test/unit_test.hpp>

int add(int i, int j)
{
    return i + j;
}

BOOST_AUTO_TEST_SUITE(Maths)

BOOST_AUTO_TEST_CASE(universeInOrder)
{
    BOOST_CHECK(add(2, 2) == 4);
}

BOOST_AUTO_TEST_SUITE_END()

BOOST_AUTO_TEST_SUITE(Physics)

BOOST_AUTO_TEST_CASE(specialTheory)
{
    int e = 32;
    int m = 2;
    int c = 4;

    BOOST_CHECK(e == m * c * c);
}

BOOST_AUTO_TEST_SUITE_END()

In a normal run, you won’t see that the tests have been categorised. To show the suites in the output, you can set the log level to test_suite:

$ ./suites --log_level=test_suite
Running 2 test cases...
Entering test suite "Suites"
Entering test suite "Maths"
Entering test case "universeInOrder"
Leaving test case "universeInOrder"
Leaving test suite "Maths"
Entering test suite "Physics"
Entering test case "specialTheory"
Leaving test case "specialTheory"
Leaving test suite "Physics"
Leaving test suite "Suites"

*** No errors detected

Step 6: Fixtures

To add common setup and teardown code around your cases, Boost.Test supports fixtures. These take advantage of C++’s own mechanism for setup and teardown – construction and destruction. Indeed, you can easily add a “fixture” to a test case be just defining a type with the appropriate constructor and destructor and allocating one on the stack at the start of the case. This is repetitious, however, and not terribly explicit. From my experience the nicest way is to organise your tests into suites so that you can use one fixture per suite, and then just use BOOST_FIXTURE_TEST_SUITE in place of BOOST_AUTO_TEST_SUITE:

#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MODULE Fixtures
#include <boost/test/unit_test.hpp>

struct Massive
{
    int m;

    Massive() : m(2)
    {
        BOOST_TEST_MESSAGE("setup mass");
    }

    ~Massive()
    {
        BOOST_TEST_MESSAGE("teardown mass");
    }
};

BOOST_FIXTURE_TEST_SUITE(Physics, Massive)

BOOST_AUTO_TEST_CASE(specialTheory)
{
    int e = 32;
    int c = 4;

    BOOST_CHECK(e == m * c * c);
}

BOOST_AUTO_TEST_CASE(newton2)
{
    int f = 10;
    int a = 5;

    BOOST_CHECK(f == m * a);
}

BOOST_AUTO_TEST_SUITE_END()

Note that the test cases can refer directly to the public “m” member of the fixture type — in the background inheritance is at work, so protected members are also directly accessible. If you run this with logging, you can see that the fixture runs for each case:

$ ./fixtures --log_level=test_suite
Running 2 test cases...
Entering test suite "Fixtures"
Entering test suite "Physics"
Entering test case "specialTheory"
setup mass
teardown mass
Leaving test case "specialTheory"
Entering test case "newton2"
setup mass
teardown mass
Leaving test case "newton2"
Leaving test suite "Physics"
Leaving test suite "Fixtures"

*** No errors detected

Conclusion

I hope that gives you a decent starting point for using Boost.Test. As I mentioned in the beginning, there is plenty more documentation available at the source — it’s just daunting due to its size. Happy testing!

Update

Sample code is now available at GitHub.


1 – It is also possible to define free functions containing your tests and register them manually, although I find the automatic method simpler.

Liked this post? Share it!

33 Responses to “C++ Unit Testing With Boost.Test”

  1. April 7th, 2009 at 8:26 am

    a little madness » Boost.Test XML Reports for Continuous Integration says:

    [...] a little madness A man needs a little madness, or else he never dares cut the rope and be free. -Nikos Kazantzakis « C++ Unit Testing With Boost.Test [...]

  2. May 4th, 2009 at 5:19 am

    Brian Tyler says:

    Thanks, that’s a much better introduction to the boost testing framework than the one given in the boost documentation. Cheers.

  3. June 18th, 2009 at 6:40 am

    a little madness » Blog Archive » Boost.Test Tutorial Sample Code says:

    [...] the fact that people seem to have found my short Boost.Test tutorial, I thought they might appreciate having access to the sample code. So I have cleaned up the code, [...]

  4. September 6th, 2009 at 4:29 pm

    blair says:

    Thanks, that was a lot more useful than the formal documentation.

  5. October 1st, 2009 at 4:58 pm

    Michael says:

    On Ubuntu 9.10 I could not get this to work till I linked against boost_unit_test_framework-mt I don’t know why this would be the case? Any ideas?

  6. December 3rd, 2009 at 6:08 am

    Albob says:

    @Michael: I think that you have to link against a lib and provide its exact name because there are a lot of configurations available and G++ doesn’t know how to choose them. Somehow this is all automatic with Visual Studio.

    By the way, thank you for this article, it gave me the BOOST_TEST_DYN_LINK that I missed on Ubuntu 9.04 (I had an “undefined reference to: ‘main'”).

  7. January 19th, 2010 at 5:30 pm

    test says:

    Thank you so much. This is THE MOST useful tutorial I have found so far on boost testing. It is easy to understand and very well written. Thanks so much once again.

  8. January 31st, 2010 at 6:11 am

    Rakesh says:

    Thankyou very much. I was struggling a lot to find the boost testing framework and this is the only tutorial that helped me ( in all respects) so far.

  9. February 19th, 2010 at 8:50 pm

    Joe says:

    Thanks so much for writing this up! This was way more helpful than the official Boost docs.

  10. February 27th, 2010 at 12:18 pm

    Essex says:

    This little introduction is better than any of the official Boost.Test documentation. I’m actually using it now, rather than getting more and more baffled in a mountain of technical specifications.

  11. March 10th, 2010 at 1:04 am

    Gui says:

    I may be stupid but is the parameter “-lboost_unit_test_framework” supposed to work while compiling hello.cpp ? I get this outpur error :
    /usr/bin/ld: cannot find -lboost_unit_test_framework …

  12. March 10th, 2010 at 1:11 am

    Ulrik Gammelby says:

    I agree with the other posters, thanks for this gentle and useful introduction to boost tests. The official documentation is WAY too heavy on (irrelevant) words, such as general chatter about why testing is a good thing and long elaborations on how things looked earlier in the framework, confusing the picture if you are just scanning for a specific piece of information.

  13. March 10th, 2010 at 1:20 am

    Gui says:

    Did not see the previous answer. Seems that on ubuntu boost is multi-threading enable. Following line solve the problem :
    g++ -ohello -lboost_unit_test_framework-mt hello.cpp

  14. March 10th, 2010 at 8:45 pm

    Gui says:

    Does anyone know if it is possible to use boost.test without any other boost library ?

  15. March 14th, 2010 at 7:17 pm

    Jason says:

    Hi Gui,

    AFAIK Boost.Test does not require any of the other Boost libraries. I haven’t tried installing just Boost.Test to verify this, though.

  16. March 17th, 2010 at 9:27 pm

    Gui says:

    Thanks Jason, I tried to cross compile Boost.Test through bjam. It is pretty easy to use and automatize. Using Bjam with the “–with-test” parameter will only compile Boost.Test.

  17. May 27th, 2010 at 6:38 pm

    Robert says:

    Is there a way to put separate test suites in different files such that I can either compile and run them individually or all together? Sometimes I only want to run the tests for a single module I’m working on, other times I want to run all the tests for the entire project.

  18. March 19th, 2011 at 9:22 am

    Very nice and concise blog post about Boost.Test | The Lone C++ Coder's Blog says:

    [...] a little madness » Blog Archive » C++ Unit Testing With Boost.Test This entry was posted in C++ and tagged c++, unit testing. Bookmark the permalink. ← Emacs 23.3 released [...]

  19. August 2nd, 2011 at 4:22 am

    Marco Di Bartolomeo says:

    A very nice Boost.Test tutorial! Thank you.

  20. October 9th, 2011 at 8:04 pm

    KEPS says:

    Thank goodness for some clarity on this! Thanks very much for your nice tutorial.

  21. January 28th, 2012 at 1:42 pm

    emc testing lab says:

    emc testing lab…

    [...]a little madness » Blog Archive » C++ Unit Testing With Boost.Test[...]…

  22. December 9th, 2012 at 4:38 pm

    Fraser says:

    Thanks!!!!

    You actually explained how to compile and use test cases – unlike the official boost website. This is what should be on their tutorial!

    Fraser

  23. January 18th, 2013 at 12:18 pm

    Dez says:

    Better than the boost docs on this. Thanks! First time I was able to get boost test to work for me.

  24. January 31st, 2013 at 10:35 am

    Adam says:

    Thanks so much – this has been a great help!

  25. March 24th, 2013 at 7:32 pm

    Journeyer says:

    Thanks this is very much easy to understand about Boost.Test for me.
    The official document is not that kind like this.

  26. April 16th, 2013 at 2:58 am

    jarrett says:

    I wish I could upvote this article :)

  27. April 17th, 2013 at 12:50 am

    Testing C++ Skip Lists | Partial Lattice says:

    [...] details, let’s start with how to get Boost.Test up and running. I would recommend you start here. Sometime in between the Boost 1.34.1 he used and Boost 1.48, there was a change to the Boost unit [...]

  28. July 9th, 2013 at 10:17 pm

    kaz says:

    I d echo what others are saying. Much better intro then original boost document. All the way A+++

  29. September 29th, 2013 at 5:43 am

    Gert-Jan de Vos says:

    There is now a GUI runner for boost.test on Windows. See http://boosttestui.wordpress.com for details. boosttestui runs boost.test, Google Test and NUnit based unit tests.

  30. October 6th, 2013 at 9:27 am

    tester says:

    if you get undefined reference errors try putting the -lboost_unit_test_framework after hello.cpp (not before):

    http://stackoverflow.com/questions/13767699/boost-test-undefined-reference-errors

  31. April 12th, 2014 at 2:02 am

    auro says:

    Excellent intro! Got the examples to work on my IoT device! Agree with others that the “official” boost unit-test framework tutorial could do a better job. I spent two hours on that guide before I found this one.

  32. June 5th, 2014 at 10:01 pm

    Hao Liu says:

    Great tutorial for boost test. Thanks a lot for this excellent work. It did help me out.

  33. October 1st, 2014 at 4:25 am

    Chris DUrso says:

    Kudos! Nice article.

    I did run into a snag with OSX using clang, your example would generate

    signal 11 [segmentation fault]

    Rebuilding boost with the following flags resolved problem.

    # boost_1_56_0
    ./bootstrap.sh –with-toolset=clang –prefix=/usr/local
    sudo ./b2 install toolset=clang cxxflags=”-std=c++11 -stdlib=libc++” linkflags=”-stdlib=libc++” threading=multi

    Thanks

Leave a Reply