a little madness

The headline new feature in Pulse 1.2 is personal builds. A personal build is a build of the current state of your working copy on the Pulse server. This allows you to test your changes before you commit them to version control. The most obvious advantage of this is that you don’t have to taint your shared source base with untested code to get a CI build. You test first, then commit when you are happy. However, there are also some less obvious advantages:

• Multiplatform testing: Pulse supports distributing a build across multiple agents in parallel. This allows you to easily test on multiple platforms. Couple this with personal builds, and you can easily test code on platforms other than your preferred development platform while you develop. No need to struggle with slow builds over a networked file system, or to manually move the code about to test.
• More efficient resource usage: you can submit a personal build to your Pulse build farm and still have your development machine free for other tasks. You can also make use of free developer machines as Pulse agents, but that is for another post .
• Full reporting: just like every other build, Pulse extracts and reports interesting information for personal builds via a rich web interface. This beats digging through hundreds of lines of build logs and scratching up test reports to find the relevant info. Pulse can also be configured to send you notifications when personal builds complete, just like a CI build. You can even customise both the information Pulse extracts from the logs and the format of the notification messages.
• Build history: Pulse will remember your recent personal builds (as many as you choose) and keep the results available for browsing. This makes it possible to refer back to an earlier issue without scrolling frantically in your console buffer (if you’re lucky enough to have the output at all!).

Personal builds are just one way in which we are making Pulse do more than your regular continuous integration server. From the start, we have seen Pulse as not just a server that sits on the sideline, but as a tool that you can leverage during every day development. This is why we have a strong focus on the developer in Pulse: every developer has their own account with a configurable dashboard and very flexible notification settings. Adding personal builds to the mix expands on what you can do with Pulse as you develop, and it is just one of a suite of tools we have or plan to add to Pulse in the near future.

Anyhow, I hope I’ve piqued your interest in the idea. If so, check out the Early Access Page for Pulse 1.2, and enjoy!

Phew, it’s been a busy time, but finally we have the first milestone build of Pulse 1.2 ready to go! The headline feature for this release is the ability to run personal builds. A personal build takes your local changes and applies them to a pulse™ build without them being submitted to your SCM first. This allows you to test your changes before submitting them to version control.

Other major features in this release include:

• Reports: each pulse™ project now has its own “reports” page, which displays build data for the project visually. Currently, the reports show trends over time for:
  • Build results
  • Tests run per build
  • Build time
  • Stage execution time
• Windows System Tray Notification: a new Pulse client, Stethoscope, sits in your system tray allowing you to see your project health at a glance. You can configure Stethoscope to monitor both personal builds and project builds for your selected projects. If you like, Stethoscope will pop up a message whenever a build completes.
• Customisable Notifications: don’t like the format of your notification emails or instant messages? In pulse™ 1.2, the notification templates can be customised using FreeMarker.
• Automatic Agent Upgrades: we go to great effort to make pulse™ easy to install, upgrade and maintain. That is why in pulse™ 1.2 we have made the upgrade process even simpler by adding automatic upgrades for agent machines. Now, after you upgrade your main pulse™ server, your agents will be automatically upgraded for you!
• Resource Configuration Wizard: on the same theme of keeping things simple, we have also added a new resource configuration wizard. This wizard makes it easy for you to configure common build dependencies, such as Java Development Kits and build tools (ant, make, etc). We have also improved the resource auto-discovery code to detect resource versions for you: in many cases you won’t even need the wizard!
• Anonymous Signup: you can now optionally allow users to sign up to pulse™ themselves. This lessens the burden on the pulse™ administrator by removing the need for them to create accounts. It is also perfect for public-facing servers (e.g. open source projects) where interested parties can sign up for read-only access but with their own dashboard and preferences.

Grab a milestone build now from our Early Access Program page and try it out!

Recently I’ve been cooking up some new acceptance-level tests for Pulse. The sort of tests that need to take a Pulse release package, unpack it, start the server and poke it with a stick. This is the sort of automation that I would typically use bash for: since a lot of the work involves executing and managing subprocess. However, I had learned from earlier testing that process control from bash is not very portable. More specifically, the behaviour of Cygwin bash on Windows differs considerably from *nix bash. At the time, I worked around the problem by doing some process control using Ant, but there had to be a better way.

Enter Python. Or, more specifically, the new(ish) subprocess module (introduced in Python 2.4). With this module Python finally has a concise but powerful way to launch and control subprocesses. It also includes convenient ways to handle subprocess I/O, including setting up pipelines. What’s more, I have so far found it to be quite portable between Linux and Windows.

Starting with something simple, let’s say you just want to launch a command synchronously (much as you would in a shell script):

Simple. Want to set up a pipeline like `dmesg | grep hda` (capturing the final output)? Here’s an example straight from the doco:

For my specific case, I used the ability to launch a Pulse server processes in the background and capture the output extensively. When done, I shut down the server using the standard shutdown script (testing of said script being a nice side-effect). An asynchronous child is started simply by creating a Popen object:

I have simplified this example a bit, but the real work is all there, and it’s pleasantly straightforward. This combined well with Python’s simple XML-RPC API, which I used to manipulate the running server.

So far the switch to from bash/Ant to Python has been a great success. If you need a cross-platform way to automate your acceptance testing, give it a go.

I like to keep an eye on various build and testing tools, for potential integration with Pulse. As such, I’ve started to amass some links to unit testing tools/resources for C++, where there are many competing options:

• Exploring the C++ Unit Testing Framework Jungle: the most comprehensive article I’ve found on the topic, even though it is getting old.
• CppUnit: probably the best know xUnit port, supported in Pulse already. Does a decent job.
• Boost.Test: part of the well known set of Boost libraries. Naturally has dependencies on Boost.
• CppUnitLite: a minimalistic rewrite of CppUnit, intended to be simpler and more portable. May be appropriate if you don’t mind getting your hands dirty to add the features you need.
• Nano Cpp Unit: more an exercise in illustrating the barest testing framework than a usable framework itself.
• Unit++: pitched as a more “C++ like” xUnit port. Documentation is thin on the ground, and I don’t have any practical experience with it.
• CxxTest: takes the novel approach of using Perl to generate a test runner, simplifying the code. Also relatively portable, although of course you will need Perl.
• TUT: a simple, template-based framework distributed as a single header file. Reasonable portablility (given a modern compiler). Lacking some features of other frameworks, but without any dependencies.
• cutee: another framework aimed at simplicity. Looks to have test case creation down to its simplest form. Documentation is thin.
• CppTest: you guessed it: another framework aimed to be simple and portable. Supports a few output formats out of the box, including HTML reports.
• UnitTest++: co-authored by the author of the article above, this framework hopes to combine the best ideas from others. Documentation is non-existant, but on the plus side it is one of the more modern frameworks (developed this year, not 2004!).
• QtUnit: probably a good option if you were using Qt, but is officially unmaintained. Mind you, most of the other frameworks are also dormant.

That’s all I have gathered so far. I have to say, there are a lot of players out there, but little action. A few interesting ideas, but no framework seems to be a clear leader. I hope to get some time to play in depth a bit more, in which case I will flesh out more details.

Each day, every developer on a software team interacts with the build system several times. Despite this, many teams underplay the importance of an effective build system. Is your build system working for you, or do you merely tolerate its existence? In this article, we will journey through build system requirements starting from the most basic through to those that make the whole development process more efficient.

Read the The Road To Build Enlightenment at zutubi.com.

Earlier today I read Cedric Beust’s post Agile People Still Don’t Get It. In that post Cedric makes some fair criticisms of misguided agile evangelism. I find it particulalry ironic that the closing sentence is:

So here is my advice to Agilists: get real now, or you will become irrelevant soon.

Pragmatism should be a cornerstone argument for agile development, so those evangelists that don’t emphasise it surely are doing more harm than good¹.

Having said all that, there is an interesting undertone in some of Cedric’s comments that got me thinking. In his defense of pragmatism, Cedric makes repeated assertions that sometimes we need to cut corners to make a real-world deadline. One corner to be cut is testing:

But the truth is: at times, I don’t do TDD because implementing a feature quickly is more important than a fuzzy feeling.

This is what really got me thinking. Note that we are talking short-term here: in the long term, I think most would agree that cutting corners will catch up with us eventually. But why do we assume that cutting this corner will save time in the short term? I think the assumption is based purely on a shallow evaluation: writing less code this second means I can hit the looming deadline more easliy. This seems valid enough: writing the extra code to perform the testing takes non-zero time after all. But the analysis is missing something: can we really implement a feature without testing it? We can try, but how will we know if it works at all? A broken feature is worse than no feature at all, so there is no way to invoke the “customer value” argument here.

I think most people agree that some level of testing is required: the risk of zero testing is just too great. A temptation, then, is to test the feature by hand, because that will be quicker than writing code to test it. Now the assumption is getting even shakier. Maybe a manual test is quicker as a once-off, but what about when you find a problem? Now you need to fix the code, then manually test again. You will be wishing you had just written test code in the first place, so you could repeat the test with very little extra effort. So, counter-intuitively, it may be quicker to code the feature and tests rather than just the feature. Not in all cases, for sure², but probably more frequently than we realise.

In fairness to Cedric, I have taken this off on a tangent of my own. His post just got me thinking about a natural assumption that we tend to make which may actually be false. I agree with the assertion that sometimes a pragmatist has to make compromises to meet a short term goal, I just wonder about the short term effects of this particular compromise.

—
¹ I suppose this is what happens when people become true believers in a methodology: they start to drift farther and farther from reality.

² For example, when the feature is very hard to test in an automated fashion. Usually this can be remedied in the long term, when you can afford the extra time to refactor for testability.