Boost.Test XML Reports with Boost.Build
My previous post Using Boost.Test with Boost.Build illustrated how to build and run Boost.Tests tests with the Boost.Build build system. For my own purposes I wanted to take this one step further by integrating Boost.Test results with continuous integration builds in Pulse.
To do this, I needed to get Boost.Test to produce XML output, at the right level of detail, which can be read by Pulse. This is another topic I have covered to some extent before: the key part being to pass the arguments “-log_format=XML -log_level=test_suite” to the Boost.Test binaries. The missing link is how to achieve this using Boost.Build’s run task. Recall that the syntax for the run task is as follows:
rule run (
sources + :
args * :
input-files * :
requirements * :
target-name ? :
default-build * )
Notice in particular that you can pass arguments just after the sources. So I updated my Jamfile to the following:
using testing ;
lib boost_unit_test_framework ;
run NumberTest.cpp /libs/number//number boost_unit_test_framework
: --log_format=XML --log_level=test_suite
;
and lo, the test output was now in XML format:
$ cat bin/NumberTest.test/gcc-4.4.1/debug/NumberTest.output <TestLog><TestSuite name="Number"><TestSuite name="NumberSuite"><TestCase name="checkPass"><TestingTime>0</TestingTime></TestCase><TestCase name="checkFailure"><Error file="NumberTest.cpp" line="15">check Number(2).add(2) == Number(5) failed [4 != 5]</Error><TestingTime>0</TestingTime></TestCase></TestSuite></TestSuite></TestLog> *** 1 failure detected in test suite "Number" EXIT STATUS: 201
The output will not exactly win awards: it has no <?xml …?> declaration, no formatting, and thanks to Boost.Test contains trailing junk. We’ve made sure that the processing in Pulse 2.1 takes care of this, though.
If you are a Pulse user looking to integrate Pulse and Boost.Test, you might also be interested in a new Cookbook article that I’ve written up on this topic.
This entry was posted on Thursday, December 10th, 2009 at 2:24 am and is filed under Agile, Build, C++, Continuous Integration, Technology, Testing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
July 24th, 2010 at 6:00 am
Shivesh says:It seems that you are using Boost and Pulse to do CI. May I ask what’s your strategy for integrating Boost library with your CI? The Boost library library (artifact) is huge. Do you check it in to your repository? Does your CI check it out everytime it needs to do a build? Keep up the good blog!
July 29th, 2010 at 7:15 pm
Hi Shivesh,
I don’t currently work on a project that uses boost in production: my interest stemmed from adding support to Pulse for Boost.Test reports. However, the problem of large dependencies is a pretty common one. Opinions differ on the best way to handle such things.
Some people prefer to have their project as self-contained as possible, in which case they would check a dependency like boost in with their code. The obvious downside is clean checkouts become slow – so for continuous builds people often use incremental updates.
At the other end of the spectrum, you could require boost to be installed on any machine that runs the build. This keeps your codebase small, but has the disadvantage of needing to keep all the boost versions in sync (and risking using the wrong version by accident sometimes).
Somewhere in between the two, you could consider a dependency management tool. These are prevalent in the Java world, but not so much for C++. In our case we use Ivy. We store versioned binary dependencies in a repository (just files on a web server) and our projects declare what dependencies they need. The first time a project is built on a machine, Ivy pulls down the dependencies and caches them. Subsequent builds are fast as their dependencies are already in a local cache.