Since I have previously stated my admiration for UnitTest++ as a unit testing framework for C++, I guess it was high time I added direct support for it in Pulse. Well, as of Pulse 1.2.24, there is now a UnitTest++ post-processor that will slurp your UnitTest++ test results directly into Pulse. If you’re looking to do continuous integration for C++, then Pulse + UnitTest++ is a killer combo ;).
In my previous post, UnitTest++: The New Choice for C++ Unit Testing?, I gave a basic introduction to UnitTest++. Moving beyond the basics, you will often want to integrate UnitTest++ into your development process. A key to this is being able to generate test reports in a format amenable to integration. The default output of UnitTest++ is a simple, human-readable summary of the test results:
This is fine for the developer, but not so easy to process in code for integration purposes. Luckily, the reporting mechanism is not fixed. The test runner accepts an instance of type TestReporter to use for reporting results. The default reporter, TestReporterStdout, produces the developer-friendly output shown above. A second reporter is also included in the UnitTest++ distribution: XmlTestReporter. As its name suggests, this reporter outputs results in XML format, which is much easier to digest in code.
Using the XmlTestReporter is easy. Just construct one with an output stream and pass it to the test runner when executing the tests:
int main(int, char const *[])
{
std::ofstream f(“tests.xml”);
UnitTest::XmlTestReporter reporter(f);
return UnitTest::RunAllTests(reporter,
UnitTest::Test::GetTestList(),
NULL,
0);
}
The results in this case are saved to a file named “tests.xml”. An example XML report is shown below:
The report is easily interpreted, and can be easily parsed for integration in to other systems. A prime use case would be integration with a continuous integration server, such as Pulse, which is why I am looking into the reports myself :).
Finally, if the XML format is not suitable for your purposes, you can also create your own test reporters. The interface is compact and easily implemented.
In an earlier post on C++ Unit Testing Frameworks, I came across a relatively new framework by the name of UnitTest++. At first glance, this framework appealed to me for a couple of reasons:
So, I’ve decided to take a closer look. I’ll start in this post with the basics: how do we write tests, fixtures and suites in UnitTest++? These are the fundamentals of a unit testing library, and should be very simple to use.
First, we need the UnitTest++ distribution. It is available as a simple tarball from SourceForge. Exploding the tarball gives a basic structure with build files at the top level, and child docs and src directories. To build the library itself, on Linux at least, requires a simple make:
The primary output is libUnitTest++.a at the top level. This, along with the header files under src (excluding src/test), forms the redistributables needed to build against UnitTest++ in your own project. It is a little awkward that no binary distributions, nor a “dist” or similar Makefile target are available. However, the source tree is so simple that it is not hard to extract what you need.
Armed with the library, the next step is to create out first test case, and run it. UnitTest++ makes use of macros to simplify creating a new test case. It could hardly get an easier:
TEST(MyTest)
{
CHECK(true);
}
int main(int, char const *[])
{
return UnitTest::RunAllTests();
}
A test case is created using the TEST macro, which takes the case name as an argument. The macro adds the test case to a global list of cases automatically. The body of the test utilises the CHECK macro to assert conditions under test. Various CHECK* macros are available for common cases. Finally, to actually run the test, we call UnitTest::RunAllTests(). This runs all cases using a default reporter that prints a result summary to standard output:
RunAllTests returns the number of failed cases, so using this as the program exit code works well. If we change the check to CHECK(false), we get a failure report:
The next step is to create a test fixture, which allows us to surround our test cases with shared setup/teardown code. This is achieved in UnitTest++ by building upon standard C++ construction/destruction semantics. To create a fixture, you just create a standard C++ struct. The setup and teardown code go in the struct constructor and destructor respectively. Let’s illustrate how this works:
struct MyFixture
{
std::string testData;
MyFixture() :
testData(“my test data”)
{
std::cout << “my setup” << std::endl;
}
~MyFixture()
{
std::cout << “my teardown” << std::endl;
}
};
TEST_FIXTURE(MyFixture, MyTestCase)
{
std::cout << testData << std::endl;
}
int main(int, char const *[])
{
return UnitTest::RunAllTests();
}
Instead of the TEST macro, we use TEXT_FIXTURE to create a test case that uses the fixture struct. The example is artificial, but serves to illustrate the order in which the functions are called. Also of interest is how members of the fixture struct are referenced directly by name within the test case. Under the covers, the TEST_FIXTURE macro derives a type from MyTestFixture, making this possible. Running this new program gives the following:
The setup and teardown wrap execution of the test case, which has simple access to the data in the fixture. By leveraging construction/destruction, the fixture code is both familiar and concise.
The final step is to organise test cases into suites. UnitTest++ again uses macros to simplify the creation of suites. You simply wrap the tests in the SUITE macro:
SUITE(SuiteOne)
{
TEST(TestOne)
{
std::cout << “SuiteOne::TestOne” << std::endl;
}
TEST(TestTwo)
{
std::cout << “SuiteOne::TestTwo” << std::endl;
}
}
SUITE(SuiteTwo)
{
TEST(TestOne)
{
std::cout << “SuiteTwo:TestOne” << std::endl;
}
}
int main(int, char const *[])
{
return UnitTest::RunAllTests();
}
As shown above, it is possible to have two tests of the same name in different suites. This illustrates the first function of suites: namespacing. Running the above gives:
Suites also have another function: they allow you to easily run a group of related tests. We can change our main function to only run SuiteOne (note we also need to include TestReporterStdout.h):
Running this new main will only execute SuiteOne:
So there you have it, a taste of the basics in UnitTest++. The most appealing thing about this library is simplicity: you can tell that the authors have made an effort to keep construction of cases, fixtures and suites as easy as possible. This lets you get on with writing the actual test code. In this overview I have not explored all of the details, most notably the various CHECK macros that test for equality, exceptions and so on. However, as it stands UnitTest++ is quite a simple framework, and there is not a whole lot more to it. Although you may need more features than you currently get out of the box, UnitTest++ is young and thus I expect still growing. The simplicity also makes it an easy target for customisation, which is important given the diversity of C++ environments. I’ll be keeping an eye on UnitTest++ as it evolves, and recommend you take a look yourself.
I like to keep an eye on various build and testing tools, for potential integration with Pulse. As such, I’ve started to amass some links to unit testing tools/resources for C++, where there are many competing options:
That’s all I have gathered so far. I have to say, there are a lot of players out there, but little action. A few interesting ideas, but no framework seems to be a clear leader. I hope to get some time to play in depth a bit more, in which case I will flesh out more details.
I have been immersed in Java for a while now, but having worked in C++ for years before, there is one big thing I miss: destructors. Especially in a language with exceptions, destructors are a massive time and error saver for resource management.
Having garbage collection is nice and all, but the fact is that we deal a multitude of resources and need to collect them all. How do we do this in Java? The Hard Way: we need to know that streams, database connections etc need to be closed, and we need to explicitly close them:
Of course, with exceptions it gets worse. We need to guarantee that the stream is closed even if an exception is thrown, leading to the oft-seen pattern:
The noise is just incredible. A common way to reduce the noise is to use a utility function to do the null check and close, but noise still remains. Repeating the same try/finally pattern everywhere is also mind-numbing, and it can be easily forgotten leading to incorrect code.
In C++, this problem is solved elegantly using the Resource Acquisition Is Initialisation (RAII) pattern. This pattern dictates that resources should be acquired in a constructor and disposed of in the corresponding destructor. Combined with the deterministic destruction semantics for objects placed on the stack, this pattern removes the need for manual cleanup and with it the possbility of mistakes:
Where has all the cleanup gone? It is where it should be: in the destructor for std::ifstream. The destructor is called automatically when the object goes out of scope (even if the block is exited due to an uncaught exception). The ability to create value types and place them on the stack is a more general advantage of C++, but Java can close the gap with smarter compilers1.
Interestingly, C# comes in half way between Java and C++ on this matter. In C#, you can employ a using statement to ensure cleanup occurs:
In this case the resource type must implement System.IDisposable, and IDispose is guaranteed to be called on the object at the end of the using statement. The using statement in C# is pure syntactic sugar for the try/finally pattern we bash out in Java every day.
What’s the answer for Java?2 Well, something similar to using would be a good start, but I do feel like we should be able to do better. If we’re going to add sugar why not let us define our own with a full-blown macro system? Difficult yes, but perhaps easier than always playing catch up? An alternative is to try and retrofit destructors into the language3. It is possible to mix both garbage collection and destructors, as shown in C++/CLI4. However, I don’t see an elegant way to do so that improves upon what using brings. If you do, then let us all know!
—
1 it appears that Mustang already has some of the smarts such as escape analysis.
2 if you’re the one down the back who shouted “finalizers”: you can leave anytime you want as long as it’s now!
3 I said NOW!
4See also Herb Suttor’s excellent post on the topic Destructors vs. GC? Destructors + GC!.
A little madness is the blog of Daniel Ostermeier and Jason Sankey. We are the founders of zutubi, a company based in Sydney, Australia that makes software for software developers.
You are currently browsing the archives for the C++ category.