a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis

Zutubi

Archive for April, 2009

Concurrency: Are We Focusing On the Wrong Thing?

It is fairly widely accepted that concurrent programming is becoming more important, and that this trend will only accelerate over coming years. This can be traced back to Herb Sutter’s excellent article The Free Lunch Is Over, which elaborates on how we hit the wall in terms of CPU clock speed around 2003, and further CPU performance has concentrated on hyperthreading and multicore ever since. The figure used in that article is such a dramatic illustration that I’ve seen it used dozens of times in talks and articles since:

Intel CPU Introductions (sources: Intel, Wikipedia)

Intel CPU Introductions (sources: Intel, Wikipedia)

But something bothers me in all the talk about concurrency, and I think it derives from focusing too much on speed. It is true that CPU-bound applications no longer benefit from the “free lunch” of increasing clock speeds. However, I would contend that the vast majority of applications are not CPU bound. Further, many applications that do need a lot of CPU power are already naturally parallelisable: such as the fast-growing category of multi-user web applications.

So although clock speeds have stagnated, for many programmers it won’t have any real performance impact. Most desktop applications will get along just fine, still bound on I/O or a squishy, organic, slow-moving user. Web applications will naturally take advantage of more cores as they appear. The only programmers likely to notice the performance impact are those working in areas where performance has always been at the bleeding edge, which frankly is a small minority.

This doesn’t mean we’re all off the hook, though. Even though few of us write CPU-bound code, many more of us write multithreaded code. Badly. The only reason our code works is because it isn’t run on truly multi-processor systems, and certainly not on systems with dozens of CPUs (or more). Whole classes of concurrency bugs don’t show up until code is stressed on highly parallelised hardware. And it just gets worse as more CPUs or cores are added. We’ve got away with this for a while, but when every desktop has dozens of cores, ignorance will cease to be bliss.

Sutter’s article covers these points, but like most other discussions I think it both underplays them relative to the performance issues, and underestimates how difficult it is to learn concurrent programming. This is a much greater challenge than learning a design technique like OO — concurrent programs, at least those using current locking techniques, are insanely hard to reason about. The true solution, in the opinion of many (myself included), is to move away from shared memory and explicit locking as much as possible. This is a key reason why pure functional programming is seeing a resurgence.

Sutter was right: the free lunch is over. Only it’s not performance we’ll have to pay for: it’s correctness.

Boost.Test XML Reports for Continuous Integration

Following on from my Boost.Test primer, the key goal for me was test result reporting in a continuous integration server1. To support this, I needed to produce output from Boost.Test which I could easily consume in a plugin. As it happens, Boost.Test has built in support for producing XML reports, which are easy to parse and therefore integrate with other tools.

What was not obvious, though, was exactly how to produce output with the right level of detail. The most promising parameters in the documentation were report_format and report_level, and indeed these can be used to produce XML — but even the detailed version does not output the reason when a test case fails. It turns out that assertion failures are reported in log output, and the log format can be tweaked to produce XML (reformatted for readability):

$ ./main --log_format=XML
<TestLog>
    <Error file="main.cpp" line="20">check add(2, 2) == 5 failed</Error>
    <Error file="main.cpp" line="25">check add(2, 2) == 1 failed</Error>
</TestLog>

To get output for passing tests, and for test suites, I also had to adjust the log_level to test_suite

$ ./main --log_format=XML --log_level=test_suite
<TestLog>
  <TestSuite name="PulseTest">
    <TestSuite name="VariantsSuite">
      <TestCase name="simplePass">
        <TestingTime>0</TestingTime>
      </TestCase>
      <TestCase name="checkFailure">
        <Error file="main.cpp" line="20">check add(2, 2) == 5 failed</Error>
        <TestingTime>0</TestingTime>
      </TestCase>
...
    </TestSuite>
  </TestSuite>
</TestLog>

Bingo! Now the XML output has all the details required to render the test results nicely in a continuous integration server, provided the server has a plugin to read the XML.


1 In my case obviously Pulse, but XML reports are likely to be the easiest way to integrate with other CI servers too.