a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis


Archive for September, 2006

Automate Your Acceptance Tests With Python

Recently I’ve been cooking up some new acceptance-level tests for Pulse. The sort of tests that need to take a Pulse release package, unpack it, start the server and poke it with a stick. This is the sort of automation that I would typically use bash for: since a lot of the work involves executing and managing subprocess. However, I had learned from earlier testing that process control from bash is not very portable. More specifically, the behaviour of Cygwin bash on Windows differs considerably from *nix bash. At the time, I worked around the problem by doing some process control using Ant, but there had to be a better way.

Enter Python. Or, more specifically, the new(ish) subprocess module (introduced in Python 2.4). With this module Python finally has a concise but powerful way to launch and control subprocesses. It also includes convenient ways to handle subprocess I/O, including setting up pipelines. What’s more, I have so far found it to be quite portable between Linux and Windows.

Starting with something simple, let’s say you just want to launch a command synchronously (much as you would in a shell script):

exitcode = subprocess.call(["cat", "foo.txt"])

Simple. Want to set up a pipeline like `dmesg | grep hda` (capturing the final output)? Here’s an example straight from the doco:

from subprocess import Popen

p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
output = p2.communicate()[0]

For my specific case, I used the ability to launch a Pulse server processes in the background and capture the output extensively. When done, I shut down the server using the standard shutdown script (testing of said script being a nice side-effect). An asynchronous child is started simply by creating a Popen object:

from subprocess import *

# Launch process, capturing output
out = open("stdout.txt", "w")
err = open("stderr.txt", "w")
server = Popen(["startup.sh"], stdout=out, stderr=err)

# Do some stuff with the process…

# Trigger a shutdown and wait for the process to exit
exitcode = server.wait()

I have simplified this example a bit, but the real work is all there, and it’s pleasantly straightforward. This combined well with Python’s simple XML-RPC API, which I used to manipulate the running server.

So far the switch to from bash/Ant to Python has been a great success. If you need a cross-platform way to automate your acceptance testing, give it a go.

C++ Unit Testing Frameworks

I like to keep an eye on various build and testing tools, for potential integration with Pulse. As such, I’ve started to amass some links to unit testing tools/resources for C++, where there are many competing options:

  • Exploring the C++ Unit Testing Framework Jungle: the most comprehensive article I’ve found on the topic, even though it is getting old.
  • CppUnit: probably the best know xUnit port, supported in Pulse already. Does a decent job.
  • Boost.Test: part of the well known set of Boost libraries. Naturally has dependencies on Boost.
  • CppUnitLite: a minimalistic rewrite of CppUnit, intended to be simpler and more portable. May be appropriate if you don’t mind getting your hands dirty to add the features you need.
  • Nano Cpp Unit: more an exercise in illustrating the barest testing framework than a usable framework itself.
  • Unit++: pitched as a more “C++ like” xUnit port. Documentation is thin on the ground, and I don’t have any practical experience with it.
  • CxxTest: takes the novel approach of using Perl to generate a test runner, simplifying the code. Also relatively portable, although of course you will need Perl.
  • TUT: a simple, template-based framework distributed as a single header file. Reasonable portablility (given a modern compiler). Lacking some features of other frameworks, but without any dependencies.
  • cutee: another framework aimed at simplicity. Looks to have test case creation down to its simplest form. Documentation is thin.
  • CppTest: you guessed it: another framework aimed to be simple and portable. Supports a few output formats out of the box, including HTML reports.
  • UnitTest++: co-authored by the author of the article above, this framework hopes to combine the best ideas from others. Documentation is non-existant, but on the plus side it is one of the more modern frameworks (developed this year, not 2004!).
  • QtUnit: probably a good option if you were using Qt, but is officially unmaintained. Mind you, most of the other frameworks are also dormant.

That’s all I have gathered so far. I have to say, there are a lot of players out there, but little action. A few interesting ideas, but no framework seems to be a clear leader. I hope to get some time to play in depth a bit more, in which case I will flesh out more details.

10 Golden Rules for Software Support

Most software companies know that support is important. Many wax lyrical about the importance of the customer. Few deliver. If only they’d abide by some basic rules for support:

  1. Be Honest: don’t try to pull one over your customer. If you made a mistake, or they have found a bug, acknowledge it.
  2. Believe: no matter how far-fetched the user’s problem/claim, take it at face value. Think you’re certain it’s user error? Test it anyway. Nothing is worse than being fobbed off as a stupid user when you’ve found a real problem.
  3. Respect: your customers are not idiots (they chose your excellent product, right?). There is no such thing as a stupid question: we’ve all asked “obvious” things at some time. Treat the request and the person with respect.
  4. Respond: monitor support requests frequently, and respond as soon as possible. If a full answer will take a significant amount of time, respond to let the customer know you are on to it. Nobody likes to feel ignored!
  5. Track: never, ever allow yourself to lose track of a support request. Use a proper tracking system to ensure all problems are stored and dealt with. Lose the request and you lose the customer.
  6. Learn: don’t just solve the short term problem. Use support requests as a guide to improve your software and documentation. Getting asked a “simple” question a lot? Add an FAQ, or just improve the interface. Support is not just a time drain: it’s a valuable source of feedback.
  7. Apologise: your customers shouldn’t have to bother with support requests, they just want things to work. Let them know you’re sorry that you’ve wasted their time. No need to grovel, keep it genuine.
  8. Thank: many customers that encounter a problem will just walk away (probably to the competition). When customers take the effort to lodge a support request, thank them for their feedback. It’s just polite.
  9. Follow Through: don’t pass the support buck from one employee to another. If you respond to the request follow through. Get advice where you need to, but take the issue to completion.
  10. Be Certain: when you answer a support request, you must know what you are talking about. All but trivial requests will require real product knowledge. All employees (e.g. developers) should be available to help answer support requests.

Really, support is mostly about attitude: care about your customers. Understand that support, though time consuming, is helping you make your product better.