a little madness

Many developers seem to look upon software build tools (Ant/Make/Maven/Rake/SCons/etc) with disdain. They see them as painful creatures put on the earth to gnaw at their patience. The developer just wants to get on with Real Work (coding), and not have to deal with such annoying details. This attitude arises for a few reasons:

• Someone Else’s Problem: some don’t see writing/maintaining build scripts as part of their job. They’re programmers, not build/release engineers.
• Yet Another Language: build tools (frequently) use a custom language for their scripts. Some resist having to learn Yet Another Language.
• Real Pain: writing, and particularly debugging build scripts can be a right pain. The language might be restrictive, debugging support limited, the behaviour arcane.
• It’s Good Enough: at the start of the project someone slapped together a copy-pasted build script that just, feebly, manages to output something resembling a binary. Nobody will ever bother looking at the build system again.
• Lack Of Time: improving the build system is not a customer requirement. Hence, nobody will ever get around to doing it.

I would argue that this attitude is harmful. I could even refute each point above, one by one. But it doesn’t matter. Even if all these points were valid, there is a single overriding factor: the build system is dog food you have to eat. If you don’t like the taste, why are you putting up with it? So: learn your build tool, and learn it well. Invest a day in improving the build system, and reap the benefits every day thereafter. Keep the knowledge with you, and refactor your build scripts just like your code to make everyone’s life easier.

I can remember many a time that I put off making such improvements for an age. When I finally make the change, the first thought is always: I rock! The next thought is: I’m a fool for not doing this sooner!

In this article, we look into automating the release process. We begin by reviewing the benefits of automated releases. We then take a look at common steps involved in the automation process, and some of the challenges they may present.

Read the full article at zutubi.com.

Recently I’ve been cooking up some new acceptance-level tests for Pulse. The sort of tests that need to take a Pulse release package, unpack it, start the server and poke it with a stick. This is the sort of automation that I would typically use bash for: since a lot of the work involves executing and managing subprocess. However, I had learned from earlier testing that process control from bash is not very portable. More specifically, the behaviour of Cygwin bash on Windows differs considerably from *nix bash. At the time, I worked around the problem by doing some process control using Ant, but there had to be a better way.

Enter Python. Or, more specifically, the new(ish) subprocess module (introduced in Python 2.4). With this module Python finally has a concise but powerful way to launch and control subprocesses. It also includes convenient ways to handle subprocess I/O, including setting up pipelines. What’s more, I have so far found it to be quite portable between Linux and Windows.

Starting with something simple, let’s say you just want to launch a command synchronously (much as you would in a shell script):

Simple. Want to set up a pipeline like `dmesg | grep hda` (capturing the final output)? Here’s an example straight from the doco:

For my specific case, I used the ability to launch a Pulse server processes in the background and capture the output extensively. When done, I shut down the server using the standard shutdown script (testing of said script being a nice side-effect). An asynchronous child is started simply by creating a Popen object:

I have simplified this example a bit, but the real work is all there, and it’s pleasantly straightforward. This combined well with Python’s simple XML-RPC API, which I used to manipulate the running server.

So far the switch to from bash/Ant to Python has been a great success. If you need a cross-platform way to automate your acceptance testing, give it a go.

Each day, every developer on a software team interacts with the build system several times. Despite this, many teams underplay the importance of an effective build system. Is your build system working for you, or do you merely tolerate its existence? In this article, we will journey through build system requirements starting from the most basic through to those that make the whole development process more efficient.

Read the The Road To Build Enlightenment at zutubi.com.

Well, if they try to do anything useful anyway. Last week I was working on improvements to the Windows service installation for Pulse. The existing service installation was handled by a pretty simple batch file wrapping the service executable. However, this method had some major limitations, mostly related to location of a compatible JVM. I needed to expand the logic in the batch script considerably as part of the solution to these limitations.

That’s when I realised: logic and batch scripts don’t really go together. In fact, scripting and batch scripts don’t really go together. Even something as simple as reading the output of an executed command is bizarre in batch (there is an obscure for loop syntax to do it). Fed up with the archaic nature and limitations of batch files, I went looking for an alternative. I had heard of powershell (aka Monad), but of course it is not yet widely installed. So I turned to Windows Script Host, which has been around since Windows 98. I hadn’t used it before, but I discovered you could write scripts in JScript (Javascript-like) and easily access the filesystem and registry, so it seemed like a good fit.

In fact, apart from the pain of navigating MSDN documentation online, my foray into WSH started quite promisingly. Then, just as I was really starting to get somewhere, Norton Antivirus chimed in. Apparently, this script that I was running was trying to do something “malicious”. Whenever I tried to do something moderately useful in the script, like access file or registry information, Norton would warn me in no uncertain terms. Brilliant. No matter that I was running the script directly, just as I could run any executable that could do equally “malicious” things. I suppose Symantec doesn’t care about false positives; that might take effort. Instead, WSH has been rendered useless to software distributors, except for those willing to have their customers warned that their software is “malicious” by a virus scanner.

In the end I implemented the solution as a standalone C++ program, because at least that way I knew I could get it to work. It’s a sad state of affairs when I am forced to write a few hundred lines of C++ when a 50 line script should do. That’s what happens when a platform is completely devoid of useful scripting options.

Well, it’s finally here! The first public beta of pulse 1.1. This is a huge milestone for us, just take a look at what’s new to see what I mean. The biggest single feature is powerful distributed building, but as you’ll see the list is long!

Thanks so much to our private pre-release testers, your feedback has been invaluable. Now for the rest of you: bring it on!

Pulse version 1.0.6 has been released. This release fixes some minor issues in 1.0.5, see the release notes for details.

Zutubi is proud to announce the availability of the Pulse automated build (or continuous integration) server for sale from today. This is the culmination of many months of development and beta testing. We would like to thank all of our beta testers for their feedback during this period.

If you haven’t tried it yet, download pulse today and let us know what you think!

When your bash script is going haywire, and you’re having trouble sorting out why, one of the simplest ways to see what is going on is to use:

set -x

This turns on the xtrace shell option, which prints the commands the shell is executing with expanded arguments, as they are executed. From the man page:

This shows you exactly what is going on as your script executes. Take the following, contrived shell script:

When executed, the script will print the following:

Note that the trace lines are prefixed with ‘+’ characters (in fact, whatever the value of $PS4 is), and regular output appears as normal. You can also see that the for loop and if conditions are displayed. This is all very handy for tracking down obscure problems.

One issue with tracing is the output can be very verbose, making the useful bits hard to find. If you have a rough idea of where the problem is, you can alleviate this problem by turning tracing on and off selectively around the interesting parts of the script (set +x turns it off). Happy scripting!

—
Automation crazy? Need a serious automated build server? Try pulse.

Every good continuous integration server provides some form of notification of build results. The most basic (and one of the best) forms of notification is via email. Email is ubiquitous, and thus there is nothing extra to set up or teach before you can use email notifications. The better continuous integration servers also allow filtering of notifications based on the build result. This is simply about increasing the signal/noise ratio: if you get emails after every build, and a large percentage of builds are successful, it is easy to lose the useful notifications in the deluge.

Given this flexibility to choose when to receive notifications, it is tempting to start filtering out aggresively. It is a matter of personal preference, but there are many people who want to keep their email traffic to an absolute minimum. This is fair enough, but in my opinion there is a minimum level of notification that every developer should subscribe to: all failed builds. I don’t care who caused it, if a build of a project you are working on fails, you should know about it. I can imagine some complaints already:

• But that’s too many emails, it’s annoying!: Well, then your project spends too much of its time broken, and you deserve to be annoyed . Failed builds should be the exception, not the rule. Receiving the deluge of emails will hopefully motivate your team to keep things working.
• But it’s not my fault, somebody else broke the build!: You are all one big happy team, remember? In this case, it is still useful for you (and other team members) to know something is broken. You may help fix the build, or hold off on other changes. Further, the threat of public shame will make your teammates more careful .
• But once it fails, it keeps failing, and the emails are piling up!: Then fix it faster! Seriously! OK, in some cases where you know the fix will take a while, you may want to pause the server. But often the fix is quick, and should be applied ASAP.

Of course, this is all just one developer’s opinion, and no continuous integration server should enforce this. However, I believe that if all developers receive notifications of all failures (at least), failures will become less frequent as a matter of course. Otherwise that pile of emails will just get too annoying!

—
Into continuous integration? Want to be? Try pulse. You can choose when you want to be notified using arbitrary boolean expressions.