a little madness

Having just made the third milestone build of the Pulse 2.1 Early Access Program, it occurred to me that it’s about time we started telling people about it! We’re baking two major new features into 2.1:

• Project Dependency Support: until now Pulse has had only limited support for dependencies, and we’ve recommended the use of tools like Apach Ivy for the delivery of artifacts between dependent builds. In Pulse 2.1, however, we’ve embedded Ivy in Pulse — both an artifact repository and the configuration of dependencies. Now you can simply declare what projects you depend on and Pulse will deliver artifacts between your builds. We’re also adding support for smart dependency triggering.
• Flexible Projects Without the XML: the build engine of Pulse has always been flexible, allowing you to define multiple build recipes each with an arbitrary number of commands, captured outputs and post-processing. However, to access the full flexibility you needed to write an XML Pulse file by hand. Well, no longer. In Pulse 2.1, the full flexibility of the engine is configurable via the Pulse web UI! These changes also allow support for new build tools to be plugged in more easily.

We’ve still got plenty of work to do, such as figuring out the trickier questions around dependencies and improving the usability of the new features. On top of this we also plan plenty of minor improvements based on feedback from our users. If you want to be part of the feedback process, just head over to the EAP page, download a build and tell us what you think!

This particular myth was one I hadn’t anticipated, but I’ve seen variations repeated often enough to convince me that there is some confusion about the relationship between continuous integration and nightly (or indeed daily) builds.

The first point to clarify is that having a build that runs daily is not the same as practicing continuous integration. Daily is just not frequent enough — indeed the goal should be a new build for every change, or continual builds if the former is impossible. Add on top of that the fact that having a build server is not the same as CI, and it’s clear that we are talking about different beasts.

Where things get more interesting is when people start to ask questions like “Is there any value in a nightly build when you are already practicing CI?”. It may seem, if you’re building twenty times a day, that there’s little point building a twenty-first time overnight. But far from making a nightly build redundant, a CI setup can include nightly builds as a complement to the normal continuous builds. Let’s look at some common examples:

• Fast Build/Slow Build: larger projects can often develop a test suite that takes hours to run. Although the goal should be to cut this time down mercilessly, the reality is some forms of testing are slow. A common workaround is to split the slower tests out of the continuous build (to keep it fast), and run a nightly build with the full test suite.
• Nightly Releases: a build can be used to release a full package nightly based on the last known good revision. This is a good example of how the CI build complements the nightly — by identifying the most recent revision that passed all tests, making the nightly releases more reliable.
• Resource Intensive Builds: the idle hours in the night are a great time to run tests that require a lot of resources, as the build can make use of hardware that would normally be tied up during the day.

So, nightly builds are not CI, but at the same time a CI setup should not neglect nightlies altogether.

It’s been a little while, but my theme hasn’t finished yet. In this case I am revisiting an idea I posted about some time ago. Due to the clear benefits of continuous integration for large teams, sometimes people forget the ways in which it can be useful for small teams. Even teams as small as one.

For small teams, the benefits are less about communication (although that is still an important factor as soon as you have more than one developer), and more about automation. Calling out the points made in my last post, these benefits include:

• Build and release automation: nobody should waste time on manual build and release processes, least of all teams already short on development resources. Setting up a continuous integration process enforces a fully automated build, which naturally extends to automated releases.
• Building across multiple environments: as a small team with a correspondingly small number of development environments, it’s unlikely your code is naturally exercised on all the platforms your support. A continuous integration server that supports distributed builds can make it easy for you to test across platforms.
• Testing frequency: a small team of developers will naturally have a lower overall frequency of testing than a large team, simply because there are less people exercising the code as they work. A continuous build can help close the gap, helping to uncover Heisenbugs that might otherwise escape into the wild.

Perhaps the largest benefit is a shift to the continuous integration mindset of working — coding in short iterations and checking in at least daily. The solo developer, with nobody else waiting for their changes, can easily bash away at the keyboard for weeks between checkins. Sometimes this works fine, but other times you end up losing perspective and sometimes motivation as you keep digging and digging away. Nothing keeps development on track better than taking baby steps and constant assessment of progress, and the very feeling of progress feeds your motivation.

This is what continuous integration is all about — baby steps, constant feedback.

—
Small team looking for a CI solution? I’ve got good news: we offer small team licenses for free, with no obligation whatsoever!

We are excited to announce the latest release of our continuous integration server: Pulse 2.0! This is by far our biggest release yet, hence the bump to 2.0. It combines two huge new features: templated configuration and plugins, which we have deliberately added together to allow them to combine seamlessly. The new configuration system has a very visible impact, including a whole-new configuration UI, but more importantly makes extending Pulse much easier. This applies not just to us but all plugin authors: we’ve spent significant effort making the plugin system simple!

For those wanting more details on what’s new, here goes:

• Templated Configuration: this allows you to manage projects with similar configurations intuitively and avoid maintainance headaches (keep your config DRY).
• New Configuration UI: a new, AJAX-powered interface which improves usability and responsiveness.
• Plugins: a standards-based OSGi plugin system designed so that plugin authors never need to worry about boring details (configuration CRUD is handled automagically).
• Fine-grained Security: use granular project, agent and server permissions to take full control of your installation.
• Improved Reporting UI: both the dashboard and browse views have been overhauled to fit more information in less space.
• Predictable URLs: Pulse web interface URLs are now human-readable and guessable, and allow easy linking to things like the latest successful build.
• Flexible Build Hooks: you can now trigger hooks both pre- and post-build or manually as part of your custom workflow.
• Integrated Help: the new configuration interface now has full help built-in — you need never RTFM again.
• Configuration API: configuration still not simple enough for you? Now you can automate it using XML-RPC!
• External Database Support: migrating to external databases is now supported with a simple wizard, and upgrades are easier.
• Automatic Configuration Backups: out-of-the-box Pulse will take care of your configuration for you by backing it up daily.
• Git Support: basic Git support has been added via a new plugin, to be extended soon.
• Much more: literally dozens of smaller features and improvements that didn’t make this list.

As always, Pulse 2.0 is free to download and evaluate, and free forever for small teams and open source projects. Why not try it today, you can set it up in minutes!

Being in the continuous integration game, it’s part of my job to keep an eye on build tools and technologies. Occasionally I hear of something interesting enough to try out, so when I next start a small project I’ll give it a go.

This time it is the turn of Gradle. From the Gradle website:

Gradle is a build system which provides:

• A very flexible general purpose build tool like Ant.
• Switchable, build-by-convention frameworks a la Maven (for Java and Groovy projects). But we never lock you
  in!
• Powerful support for multi-project builds.
• Powerful dependency management (based on Apache Ivy).
• Full support for your existing Maven or Ivy repository infrastructure.
• Support for transitive dependency management without the need for remote repositories and pom.xml
  or ivy.xml files (optional).
• Ant tasks as first class citizens.
• Groovy build scripts.

Build tools that leverage scripting languages such as Groovy, Ruby and Python are all the rage. This is undoubtedly a useful feature, but so common these days that it is not a differentiating factor. After all, just adding a more concise way to write procedural build scripts is not a big win. The focus needs to be on making builds as declarative as possible.

The current king of declarative builds in the Java world is undoubtedly Maven. However, as I have said in a previous post, the current implementation of Maven leaves a lot to be desired. Still, the Maven idea of build-by-convention is still a good one if it can be achieved in a flexible way. This, then, is what attracts me to Gradle — its specific goal of providing build-by-convention without the lock-in.

Installation

To begin, I set myself the lofty goal of writing a “Hello, World” build script. This gets me to the point where I have a working gradle installation. As I already had a JDK installed (the only external dependency), installation was as simple as:

Gradle build files are named “build.gradle”, so next I created a trivial example as follows:

The above is normal Groovy source code, executed in an environment provided by gradle. It defines a single task which is gradle’s equivalent of a target (almost like the combination of an Ant task and target). Executing this task gives:

Note that the -q flag suppresses some gradle output.

A Simple Java Project

To test gradle’s claims of build-by-convention, I next put it to work on a simple Java project. Gradle’s build-by-convention support is implemented as “plugins” for different languages, with Java and Groovy plugins provided out of the box. The project to be built is a JavaDoc doclet, so the build just needs to compile Java source files into a single jar. To make life a little interesting, the project does not fit gradle’s default conventions in two ways:

1. It should not be named after the containing directory.
2. The source is located under “src/java”, not “src/main/java”.

These are truly simple customisations — so you would expect them to be easily configured. And indeed they are, as shown in the build.gradle file:

The first line customises the project and jar file names, and the last line is used to override the default location for Java source files. With this build file in place, I can build the jar as follows:

That was pleasantly simple! The Java plugin also gives me a bunch of other tasks for free:

Conclusion

So far, gradle looks very promising. The example above is too simple to judge how it would fare on a larger, more challenging build, but it shows the basics are right: at least a simple case is simple. I hope to give it a try on a larger code base soon.

At the moment the gradle project is still in its infancy, so I’ll be keeping a keen eye on its development. Indeed, if it can achieve its stated goals it will become a build tool to reckon with, and a tough competitor for Maven.