a little madness

Every good continuous integration server provides some form of notification of build results. The most basic (and one of the best) forms of notification is via email. Email is ubiquitous, and thus there is nothing extra to set up or teach before you can use email notifications. The better continuous integration servers also allow filtering of notifications based on the build result. This is simply about increasing the signal/noise ratio: if you get emails after every build, and a large percentage of builds are successful, it is easy to lose the useful notifications in the deluge.

Given this flexibility to choose when to receive notifications, it is tempting to start filtering out aggresively. It is a matter of personal preference, but there are many people who want to keep their email traffic to an absolute minimum. This is fair enough, but in my opinion there is a minimum level of notification that every developer should subscribe to: all failed builds. I don’t care who caused it, if a build of a project you are working on fails, you should know about it. I can imagine some complaints already:

• But that’s too many emails, it’s annoying!: Well, then your project spends too much of its time broken, and you deserve to be annoyed . Failed builds should be the exception, not the rule. Receiving the deluge of emails will hopefully motivate your team to keep things working.
• But it’s not my fault, somebody else broke the build!: You are all one big happy team, remember? In this case, it is still useful for you (and other team members) to know something is broken. You may help fix the build, or hold off on other changes. Further, the threat of public shame will make your teammates more careful .
• But once it fails, it keeps failing, and the emails are piling up!: Then fix it faster! Seriously! OK, in some cases where you know the fix will take a while, you may want to pause the server. But often the fix is quick, and should be applied ASAP.

Of course, this is all just one developer’s opinion, and no continuous integration server should enforce this. However, I believe that if all developers receive notifications of all failures (at least), failures will become less frequent as a matter of course. Otherwise that pile of emails will just get too annoying!

—
Into continuous integration? Want to be? Try pulse. You can choose when you want to be notified using arbitrary boolean expressions.

Brain 1.0 has a fantastic feature: auto-correction of common spelling and grammar mistakes. This feature makes it easier to process unreliable input by skipping over repeated words, ignoring trivial typos, and generally smoothing out kinks. It also reduces the effort required to process typical input via prediction, meaning I don’t have to waste time actually reading all of the input. This feature is, no doubt, a great time saver.

However, there are times when auto-correct actually gets in my way. A prime example is proof-reading. In this case I am particularly interested in those small mistakes that Brain is auto-correcting for me. Annoyingly, there does not seem to be an easy way to turn auto-correct off. This is surely an oversight in Brain’s design that should be fixed in a future version. Perhaps this option will be added to Brain 2.0, if there is room in the roadmap alongside social networking.

—
Into continuous integration? Want to be? Try pulse.

Local build is a unique feature of pulse that allows you to execute the pulse build engine in a local checkout of your project’s source code. Why would you want to do this? First of all, let me be clear that this is not a replacement for your regular build script (e.g. build.xml, Makefile): pulse is designed to be layered on top of an existing build system. There are three main use cases for local builds:

1. Debugging your pulse file: the pulse file describes to pulse how to build your project. You can choose to either configure the build via the pulse web interface (great for simple projects where you don’t want to mess with text configuration) or by writing the file by hand. In the latter case you can also store the configuration in your SCM where it is versioned with your project source code. This is a powerful way to manage the configuration, but it suffers from a problem: when you need to modify the pulse file, you have a lengthy debug cycle. You need to make the edits, check the file in, trigger a build and wait for the pulse server to get back to you. This is painful, especially for debugging simple typos. Local build allows you to avoid the problem completely by enabling you to run the build in your local checkout of the project’s source code. Simple errors are caught immediately, and you can fine tune your changes to your heart’s content before committing them.
2. Taking advantage of pulse’s post-processing model to make sense of your build output. Pulse uses post-processing as a simple but powerful way to extract useful information from a build. All build artifacts, including the output of build commands, may be processed using one of several built-in processors to find features (errors, warnings, test failures). You can even add your own post-processors based on regular expressions. By executing your build scripts via pulse local build, you get simple summaries of each command executed, with any useful features listed in the build result. There is no need to scan through line after line of build output: let pulse local build do it for you!
3. Reproducing the build: imagine a continuous integration build fails in a mysterious way on your pulse server and you are having a hard time reproducing the failure. A key to reproduction is to execute everything exactly the same way as the pulse server. Although you should be able to reproduce almost exactly the same build as the pulse server using just your regular build scripts, local build allows you to go one better.

You can use local build even if you configure your project via the web interface (rather than writing the pulse file by hand): the pulse server generates a pulse file for every build and makes them available for download.

Pulse local build is available as a separate download to the pulse server package. Try it today: local build is free!

We are happy to announce version 1.0.4 of pulse. Highlights in this release include:

• Support for CppUnit
• More flexible notification conditions: you can now write arbitrary boolean expressions to configure when you get notified!
• The ability to clone projects: simplifying setup for multiple similar projects.
• Improvements to the test results view for large test suites: expand/collapse suites, show/hide successful tests.

Full release notes are available. Thanks to the beta testers!

Every time that I have upgraded my monitor, I have marvelled at how much more screen realestate is available. Each time I have gone through a period of adjustment where I adjusted the layouts of my applications to take maximum advantage of the extra space. My latest move from a 19inch to one of the nice Dell 24inch screens has allowed me to move to a three column layout in IDEA with the project view on the left, the structure on the right, and still plenty of room for 120 char lines in the editor. Awesome. Moving on to Thunderbird has yielded a similarly pleasing outcome with its nice three column vertical layout. Nice. Even JEdit supports a configurable layout that allows me to make good use of the screen.

Unfortunately my amusement came to an end when I opened up my browser. Either one, it matters not. Where ever I go, whether it’s my favourite news or community website, any one of the blogs I try to keep up with, I end up with one of two things. Either a fixed width page with half my screen empty on either side (but all of the content nicely squeezed into the middle), or a 100% width page where the lines are so wide that they become difficult to read and large patches of whitespace spread around the page.

Now I understand that large screens are only just becoming affordable, and so I do not expect websites to suddenly rearrange their content to suit me and my dislike of unused pixels. It does however bring to attention one aspect of building web applications that can be very frustrating. The page layout. Now certainly some people are doing nice things with the tools that are available, namely css, but that can only go so far. What I would really like to see now is a good old fashioned trustworthy layout manager.

Only just last week I was trying to achieve a relatively simple task. Opening a popup window that is an appropriate size for the content it will display. Well, I had hoped that it would be a relatively simple task. After loading the content into an iframe, doing my best to work out what its visible width and height would be and then dropping it into a popup, I am still no closer to a solution that works in both Firefox and IE. With a slightly better API for handling page layouts, this type of task would become dead simple. Speaking of layout managers, those guys over at qooxdoo are doing some pretty interesting things. Now, if only it didn’t look so much like windows.

Back in my post Your Next Programming Language I mentioned I would post occassional tips about bash scripting. As soon as I started writing my next script, it occured to me: the first thing I always do when writing a new bash script is set the errexit option:

This option makes your script bail out when it detects an error (a command exiting with a non-zero exit code). Without this option the script will plough on, and mayhem often ensues. In all the noise generated it can be a pain to found the root cause of the problem. So I make it a rule to set this option and fail as early as possible.

—
Into continuous integration? Want to be? Try pulse.

A typical continuous integration server will be set up to build each time a change is detected in source control. This achieves fast feedback for changes by testing them as soon as they are available. However, I believe many projects should be built even more frequently than this: they should be built AFAP (As Frequently As Possible). Why build when there are no new changes to the code? Simple: more testing. Not all bugs show up every time you test. In fact, the nastiest bugs are those that only show up every once in a while. These are the bugs you really want to catch before you ship. If you think it’s nasty debugging race conditions in your cozy little cube, wait until your customers start reporting intermittent problems .

Since building AFAP can have very little cost (you have a dedicated build machine, right?), many teams have no reason not to do it. The only significant cost that comes to mind is delaying feedback when a change does occur, as the continuous integration server may be busy performing an AFAP build. On average a build with a change is delayed by half the time for a full build cycle. For projects with long build times, this may be a real issue (indeed, one of many issues: get those build times down!). For those projects, it may be better to stick to building on change throughout the day. Even so, there is no need to waste those overnight and weekend hours! Schedule your AFAP builds to run throughout the night, every night and all day over the weekend. These are also good opportunities to run longer build/test cycles with your most thorough test suites.

——-
Into continuous integration? Want to be? Try pulse.

Recently, a beta tester of pulse asked a fair question:

I can see the benefits of automated build software for large software teams. Would pulse considerably benefit a development team of 1-5?

Now, I am surely biased, but my answer is most certainly yes. Firstly, there are many benefits that apply to both, such as frequent testing, testing outside of the developers’ environments, early detection of integration problems, isolating changes that introduce problems, and so on.

It is also fair to say that some of the benefits for large teams do not apply to the same degree for a team of 1-5. For example, as the team grows the chance of an integration problem increases even more rapidly due to potentail semantic conflicts between changes committed by various developers. Even when the developers are all disciplined enough to update and run tests locally (which we all are, right? ), there is still a greater chance of a submit race. For large teams the continuous integration server can also be a great way to communicate recent project activity, as developers are notified of builds, and the web UI may allow you to see recent project changes.

Smaller teams do not have such problems with submit races and typically find communication much easier. However, some of the benefits of continuous integration are more important to small teams than to large ones. A prime example is the increased frequency of testing. Without continuous integration, tests are only run as frequently as the developers run them. The fewer developers, the less frequent the testing. Problems take longer to show up, especially intermittent bugs (those nasty blighters that only show up once every 364 test runs). If your tests run every fifteen minutes on your continuous integration server, even those intermittent bugs can’t hide for long. Further, small teams necessarily have a smaller number of environments that they are naturally testing on during development. Adding another one, even a single continuous integration server, is significant. Lastly, a smaller team, with naturally limited resources, needs to apply automation even more aggresively than a large team. Dedicating one or more engineers to release and integration management may not seem like a big deal to a large team, but for a small team that may be a third of the manpower!

Although they benefit in different ways, I wouldn’t say a large team needs CI more than a small one, or vice-versa. Experience has taught me that once a team employs continuous integration, whether it be 2 or 200 strong, the engineers will never want to do without it again.

——-
Into continuous integration? Want to be? Try pulse.

Many people talk about how, as software developers, we should learn new programming languages frequently. I couldn’t agree more: the broader perspective improves your skills and opens your eyes to the dark corners of the language you are currently using. It strikes me, however, that many developers are missing out on a class of languages that are extremely useful every day. People learn high-level languages like Java and C++, and often a scripting language or two like Perl or Python. Maybe they will even dabble in a functional language to get a really different take on the world. But for me, the single programming language I use most frequently day-to-day, alongside my primary language, is bash scripting. Yep, plain old hackish shell scripts.

Why? Because like most programmers, I’m lazy. I don’t like to do anything I can make a computer do for me, and there are a whole raft of such things that are easily achieved via a shell script. Often it will just be a one-liner to perform a batch operation on a bunch of files. A find/exec/sed sure beats the pants off changing 200 files by hand, and is even quicker than writing a Perl script. Shell scripting is also a boon for project automation. Is packaging your project a headache? Need to pull in a bunch of resources, munge a few files, run some tests and squeeze it all together? Build tools such as Ant or make may get you part of the way, but they are not designed to write scripts. I often use a script to do all the gathering and munging, and call out to those scripts from my build file.

So, no excuses! Even those of you more inclined to the Windows way of life have easy access to bash (and other shells) via Cygwin. Get a taste and you won’t look back. There’s something quite gratifying about replacing an arduous, multi-step task with a script that you can run without breaking a sweat. You’ll never have to work again!

——-
Into continuous integration? Want to be? Try pulse.

After a solid week of beta testing and feedback (thanks to our beta testers!) pulse version 1.0.2 is ready! You can download the new release at the pulse downloads page. Highlights in this release:

• Much improved RSS support
• Support for maven 2 projects
• Improved “latest changes by me” view
• Several other improvements and bug fixes

See the full release notes for further details. Thanks again to our beta testers, keep on stretching it!