a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis


Archive for the ‘Zutubi’ Category

Pulse Sample and Community Plugins

Since version 2.0, Pulse has supported plugins to extend integration with external tools. Whenever customers have asked for help with plugin implementation, we’ve always provided samples as the easiest starting point. Seeing a complete plugin project, with build and packaging support, is the easiest way to get a new plugin up and running.

Based on this we’ve decided to start maintaining a repository of open source Pulse plugins. These will act both as samples and in some cases as early versions of plugins that may migrate into the core Pulse distribution. The sample plugins are hosted in a Mercurial repository which has an hgweb interface:


So far we have provided a sample JUnit test post-processor (simplified from the version shipped with Pulse), and the beginnings of a Subversion SCM plugin (wrapping the svn command line). Provided you have Mercurial installed, you can easily clone the samples over HTTP, e.g.:

$ hg clone http://hg.zutubi.com/com.zutubi.pulse.core.postprocessors.junits

The samples have README files to get you started.

You may notice there is a third sample: a plugin that provides basic SCM integration with Bazaar by wrapping the bzr command-line. I’m happy to report that this sample was generously donated by a member of the community, Michiel van Slobbe. Michiel is the first to try to get Pulse and Bazaar working together and we appreciate the effort! Hopefully it will provide inspiration for other community members.

Although we are happy to provide hosting for such community plugins, you may also choose your own path. Another member of the community, Orvid, is using Pulse for C#/Mono projects. Orvid has taken the initiative of writing a command plugin to integrate XBuild, the Mono equivalent of Microsoft’s MSBuild, with Pulse. You can find the XBuild plugin on GitHub:


You might also notice that Orvid has been working on a .Net wrapper for the Pulse remote API:


These efforts are a great contribution that we are most thankful for!

Pulse Continuous Integration Server 2.4 Released!

Happy days: we’ve now released Pulse 2.4! Thanks to all those that provided feedback during the Pulse 2.4 beta period. Here’s a recap of the major updates in this release:

  • Mercurial support: in the form of a new plugin.
  • Maven 3 support: including a command, post-processor and resource discovery.
  • Agents page updates: with graphical status and more convenient navigation.
  • Reworked agent status tab: with more build links and efficient live updates.
  • New agent history tab: quickly browse all builds that involved an agent.
  • Reworked server activity tab: showing build stages nested under active builds.
  • Pause server: admins can pause the build queue, so all triggers are ignored.
  • New server history tab: showing build history across all projects.
  • Restyled info and messages tabs: for both the agents and server sections.
  • Improved process termination: to make terminating builds more reliable.
  • Kill build action: for quicker build termination forgoing full clean up.
  • Improved changelist views: these views have been reworked in the new style.
  • Pinned builds: mark builds that should never be deleted or cleaned.
  • Templated field actions: easily find or revert to an inherited value.
  • Introduce parent refactoring: adjust your template hierarchy over time.
  • Pluggable resource discovery: automatically locate build tools and libraries.
  • Subversion changelist support: easily submit a changelist as a personal build.
  • … and more: extra UI touches, improved performance, more plugin support implementations and more.

The new in 2.4 page on our website has more details and a few screenshots. Or you can simply download and try Pulse 2.4 for free.

Pulse 2.4 Release Candidate

After a few iterations of Pulse 2.4 beta builds, we’ve finally reached a stable enough state to declare our first release candidate. Since the original 2.4 beta post, we haven’t just been squishing bugs, but have come up with several more improvements:

  • New agent history tab: quickly browse all builds that involved an agent.
  • Pause server: admins can pause the build queue, so all triggers are ignored.
  • Improved changelist views: these views have been reworked in the new style.
  • Pinned builds: mark builds that should never be deleted or cleaned.
  • Subversion changelist support: easily submit a changelist as a personal build.
  • Faster browse view: most of these optimisations also appear in Pulse 2.3.
  • Improved process termination: to make terminating builds more reliable.

We know from feedback that these changes will be popular! Check them out yourself: release candidate builds are available from the Beta Program page.

Pulse Continuous Integration Server 2.2!

Big news today: Pulse 2.2 has graduated to stable! This release includes a stack of new features and improvements, including:

  • Build UI overhaul: all tabs improved and restyled.
  • New logs tab: making it easier to access stage logs.
  • Featured artifacts: choose which artifacts should appear prominently.
  • Build navigator: easily move forward and backward through history.
  • Working copy browser: view working copies for in progress builds.
  • Move refactoring: move projects and agents in the template hierarchy.
  • Template navigation: navigate directly up and down a hierarchy.
  • Subscription by label: subscribe to notifications by project groups.
  • Agent executing stages: see what all agents are building at a glance.
  • Subversion exports: for faster and smaller builds.
  • Performance improvements: key for larger installations.

See the new in 2.2 page for full details.

In conjunction with this release, we’ve also given our website a complete overhaul. The new site has a fresher look, and communicates the key features of Pulse more directly. The updates also include some new features:

  • RSS feeds for news items.
  • Links to our latest blog posts on the front page.
  • An improved buying process, allowing multiple licenses to be purchased in one transaction.
  • Self-service renewal payments – just enter your current license key and go!
  • A more user-friendly downloads page.

We hope you enjoy the new release, and the new site. And please, let us know what you think!

Pulse Continuous Integration Server 2.2 Beta!

Great news: today the latest incarnation of Pulse, version 2.2, went beta! In this release we’ve focused primarily on usability, largely in the build reporting UI. A new build navigation widget allows you to easily step forwards and backwards in your build history – while sticking to the same build tab. All of the build tabs themselves have been overhauled with new styling and layout. Here’s a sneak peak at the artifacts tab, for example:

Artifacts Tab

Artifacts Tab

It not only shows additional information, with greater clarity, but also allows you to sort and filter artifacts so you can find the file you are after. Other UI changes go beyond style too – for example the new build summary tab shows related links and featured artifacts for the build. More information, and screenshots, are available on the new in 2.2 page.

We’ve also squeezed in some less obvious updates, such as:

  • The much-requested ability to move projects and agents in the template hierarchy.
  • Convenient navigation up and down the template hierarchy.
  • The ability to subscribe to projects by label.
  • An option to use subversion exports for smaller and faster builds.
  • Improved cleanup of persistent working directories (when requesting a clean build).
  • Performance improvements for large configuration sets.

The first beta build, Pulse 2.2.0, is available for download now. We’d love you to give it a spin and let us know what you think!

Zero To Continuous Integration in Three Minutes

A large part of our focus with Pulse revolves around saving time. We started Pulse with the belief that it shouldn’t be so hard to set up a continuous integration server, nor should it take so much effort to maintain. With that in mind, I’ve highlighted the main ways we achieve simplicity and maintainability in Pulse in two new demo videos:

  • Getting Started With Pulse: in which I start from scratch, installing Pulse, adding a new project and running a first build in under three minutes. By watching the video, you’ll see that it is unabridged, and I did nothing but follow the simple steps laid out in front of me.
  • Templated Configuration: in which I demonstrate how Pulse’s unique templated configuration system saves you time configuring and (especially) maintaining your continuous integration server. Templates make CI DRY.

We focus on saving time simply because it adds a lot of value to Pulse. Our customers tell us that simplicity, maintainability and dedicated support are the main reasons they chose Pulse to manage their builds. Give it a go yourself: you can get started in no time ;).

Pulse 2.1.11: Get More From Your Build Agents

The latest Pulse 2.1 beta build, 2.1.11, has just been freshly baked. This build includes several new features and improvements. Prominent among them is a new “statistics” tab for agents. This tab lists various figures such as the number of recipes the agent executes each day and how long the average recipe keeps the agent busy. Statistics are also shown for agent utilisation, including a pie chart that makes it easy to visualise:

agent utilisation chart

This allows you to see if you are getting the most out of your agent machines. If you do notice a machine is underutilised, another new feature could help identify the cause: compatibility information for projects and agents. Pulse matches builds to agents by considering if the resources required for the project are all available on the agent. Now when you configure requirements, Pulse shows you which agents those requirements are compatible with. On the flip side, when configuring an agent’s available resources, Pulse shows you which projects those resources satisfy.

Other highlights in this build:

  • Optional compression of large build logs (on by default).
  • Visual indicators of which users are logged in, and last access times for all users.
  • Support for Subversion 1.6 working copies for personal builds.
  • Actions can now be performed on all descendants of a project or agent template (e.g. disable all agents with one click).
  • New options to terminate a build early if a critical stage or number of stages have already failed.
  • The system/agent info tabs now show the Pulse process environment (visible to administrators only).
  • Use of bare git repositories on the Pulse master to save disk space.

Yes, we have been busy :). Get over to our website and download the beta now — it’s free to try, and a free upgrade for customers with current support contracts!

Pulse 2.1 Beta Rolls On

We’ve reached another significant milestone in the Pulse 2.1 beta: the release of 2.1.9. This latest build rolls up a stack of fixes, improvements and new features. Some of the much-anticipated improvements include:

  • Support for NAnt in the form of a command and post-processor.
  • Support for reading NUnit XML reports.
  • Support for reading QTestlib XML reports.
  • The ability to mark unstable tests as “expected” failures: they still look ugly (so fix them!) but won’t fail your build.
  • Better visibility of what is currently building on an agent.
  • New refactoring actions to “pull up” and “push down” configuration in the template hierarchy.
  • The ability to specify Perforce client views directly in Pulse.

I’ll expand upon some of these in later posts. In addition we’ve made great progress on the new project dependencies support, which should be both easier to use and more reliable in this build.

We’d love you to download Pulse 2.1 and let us know what you think!

CITCON Paris 2009: Mocks, CI Servers and Acceptance Testing

Following up on my previous post about CITCON Paris, I thought I’d post a few points about each of the other sessions I attended.

Mock Objects

I went along to this session as a chance to hear about mock objects from the perspective of someone involved in their development, Steve Freeman. If you’ve read my Four Simple Rules for Mocking, you’ll know I’m not too keen on setting expectations, or even on verification. I mainly use mocking libraries for stubbing. Martin Fowler’s article Mocks Aren’t Stubs had make me think that Steve would hold the opposite view:

The classical TDD style is to use real objects if possible and a double if it’s awkward to use the real thing. So a classical TDDer would use a real warehouse and a double for the mail service. The kind of double doesn’t really matter that much.

A mockist TDD practitioner, however, will always use a mock for any object with interesting behavior. In this case for both the warehouse and the mail service.

So my biggest takeaway from this topic was that Steve’s view was more balanced and pragmatic than Fowler’s quote suggests. At a high level he explained well how his approach to design and implementation leads to the use of expectations in his tests. I still have my reservations, but was convinced that I should at least take a look at Steve’s new book (which is free online, so I can try a chapter or two before opting for a dead tree version).

A few more concrete pointers can be found in the session notes. A key one for me is to not mock what you don’t own, but to define your own interfaces for interacting with external systems (and then mock those interfaces).

The Future of CI Servers

I wasn’t too keen on this topic, but since it is my business, I felt compelled. I actually proposed a similar topic at my first CITCON back in Sydney and found it a disappointing session then, so my expectations were low. Apart from the less interesting probing of features on the market already, conversation did wander onto the more interesting challenge of scaling development teams.

The agile movement recognises the two main challenges (and opportunities) in software development are people and change. So it was interesting to hear this recast as wanting to return to our “hacker roots” — where we could code away in a room without the challenges of communication, integration and so on. Ideas such as using information radiators to bring a “small team” feel to large and/or distributed teams were mentioned. A less tangible thought was some kind of frequent but subtle feedback of potential integration issues. Most of the time you could code away happily, but in the background your tools would be constantly keeping an eye out for potential problems. What I like about this is the subtlety angle: given the benefits it’s easy to think that more feedback is always better, without thinking of the cost (e.g. interruption of flow).

Acceptance Testing

This year it seemed like every other session involved acceptance testing somehow. Not terribly surprising I guess since it is a very challenging area both technically and culturally. As I missed most of these sessions, they are probably better captured by other posts:

One idea I would call attention to is growing a custom, targeted solution for your project. I believe it was Steve Freeman that drew attention to an example in the Eclipse MyFoundation Portal project. If you drill down you can see use cases represented in a custom swim lane layout.

Water Cooler Discussions

Of course a great aspect of the conference is the random discussions you fall into with other attendees. One particular discussion (with JtF) has given me a much-needed kick up the backside. We were talking about the problems with trying to use acceptance tests to make up for a lack of unit testing. This is a tempting approach on projects that don’t have a testable design and infrastructure in place — it’s just easier to start throwing tests on top of your external UI.

Even though I knew all the drawbacks of this approach, I had to confess that this is essentially what has happened with the JavaScript code in Pulse. We started adding AJAX to the Pulse UI in bits and pieces without putting the infrastructure in place to test this code in isolation. Fast forward to today and we have a considerable amount of JavaScript code which is primarily tested via Selenium. So we’re now going to get serious about unit testing this code, which will simultaneously improve our coverage and reduce our build times.


To wrap up, after returning from Paris I plan to:

  1. Give expectations a fair hearing, by reading Steve’s book.
  2. Look for ways to improve our own information radiators to help connect Zutubi Sydney and London.
  3. Get serious about unit testing our JavaScript code.
  4. Get PJ and JtF to swap the dates for CITCON Asia/Pacific and Europe next year so I can get to both instead of neither! 😉

If I succeed at 4 (sadly not likely!) then I’ll certainly be back next year!

CITCON Paris 2009

As mentioned Daniel and I both attended CITCON Paris the weekend before last. I’ve not had a chance to post a follow up yet as we also took the opportunity to eat the legs off every duck in France (well, we tried).

Firstly a huge thanks to PJ, Jeff, Eric and all the other volunteers for another great conference. Thanks again to Eric and Guillaume for acting as local guides on Saturday night. As always, the open spaces format and mix of attendees delivered a great day. It was also great to see a few familiar faces from the year before in Amsterdam (and a familiar shirt thanks to Ivan 🙂 ).

This year I proposed and facilitated a single topic: Distributed SCM in the Corporate World. I finally added a full write-up on the conference wiki earlier in the week for those who are interested. For the impatient, here are my take-aways from the session:

  1. Of the distributed SCMs, there is not much traction in the corporate world just yet, although git appears to have gained a foothold. (Obviously our sample size is small, but I also expect CITCON attendees to be closer to the edge than the average team.)
  2. Where distributed SCMs are used, the topology is still like the centralised model. However, the ability to easily clone and move changes between repositories presents opportunities to work around issues like painful networks (contrast this to special proxy servers which are needed in similar scenarios with centralised SCMs).
  3. The people using git liked it primarily for its more flexible workflow and better merging. It’s conceivable to have this in the centralised model too, but no single centralised contender was mentioned.
  4. So far the use of distributed SCMs didn’t seem to have practical implications for CI – probably due to the use of a centralised topology.

Looks like we’re still waiting to see more creative use of distributed SCMs in corporate projects – perhaps it is something worth revisiting in future conferences. I hope to post on some of the other sessions I attended at a later date.