a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis

Zutubi

Archive for the ‘Build’ Category

Android: Attaching Sources to Libraries in Eclipse

Since ADT r17, Android projects in Eclipse have had a nice property: all jars in your libs directory have been automatically picked up by Eclipse projects under a magic “Android Dependencies” entry in the build path. Ant builds use the same conventions, so adding a library is as simple as dropping it into the right directory. Unfortunately, though, this magic came with a major limitation: there was no way to attach sources to these libraries (see issue 27940). This led me to duplicate build path entries manually just to get source attachments.

Good news: in the just-released r20 version of ADT, Google have provided a solution to this problem. It’s not well advertised, but if you check out comment 21 on the aforementioned issue, you see that by creating a properties file for each library, you can tell ADT where to find the sources. So, for example, if you have a jar named:

some-library-1.0.jar

you can create a properties file alongside it named:

some-library-1.0.jar.properties

Within the properties file you can add a src property set to the relative (or absolute, but that’s version-control-unfriendly) path of the source jar, zip or directory. Say, like me, you put it in the libs/src subdirectory, your properties file might look like:

src=src/some-library-1.0-sources.jar

Once you’ve added the properties file, refresh the project in Eclipse and voilĂ : the sources will be attached! A similar property named doc is supported for javadoc attachments.

Although this new feature is very welcome, I feel like it could use a bit more fine tuning. The main issue is the need to create a separate properties file for each jar. I keep my source jars in predictable locations, with conventional names, so I should be able to configure the convention once and have everything Just Work from then on. Heck, I’m not sure why there isn’t a default convention, which I could just follow with no further configuration!

Pulse Continuous Integration Server 2.5 Released!

After a smooth beta period, we’re excited to announce the release of Pulse 2.5! Massive credit for the new feature and improvement ideas in this release go to the Pulse community, whose feedback was invaluable. Here’s the brief list of what’s new:

  • Browse and dashboard view filtering (by project health).
  • A new MSTest post-processor plugin.
  • Increased visibility of build (and other) comments.
  • Comments on agents, to communicate with other users.
  • Properties on agents, for simpler parameterisation.
  • The ability to reference properties in SCM configuration.
  • The ability to configure SCMs using resources.
  • A resource repository on the Pulse master.
  • A new option to attach build logs to notification emails.
  • Upstream change reporting (changes via dependencies).
  • Downstream build notifications (again via dependencies).
  • Simpler delivery of directory trees via dependencies.
  • Pre-stage hooks, which run just before the stage is dispatched to the agent.
  • Stage terminate hooks, allowing you to run custom cleanup.
  • SCM inclusion filters, complementing existing exclusion filters.
  • Inverse resource requirements (i.e. requiring the absence of a resource).
  • Smart label renaming.
  • A distinct warnings status for UI and API consistency.
  • A separate permission for the ability to clean up build directories.
  • Support for git submodules.
  • Support for Subversion 1.7 in personal builds.
  • Support for Perforce streams.

For more details, see the new in 2.5 page on our website. Or, why not just download Pulse and try it free today?

Pulse Continuous Integration Server 2.5 Beta!

I’m happy to report that the Pulse 2.5 release stream has been upgraded to beta! This marks completion of the 2.5 feature set. Since the alpha release, we’ve powered through more features and improvements; examples include:

  • A new warning status, making it easier to work with builds with warnings.
  • The ability to reference properties in SCM configuration.
  • Another new stage hook points on stage termination.
  • Upstream change reporting to show how dependencies influenced a build.
  • Dependency notifications to show how your changes affected downstream builds.
  • Artifact archives, allowing easy delivery of directory trees between projects.
  • Flexible bootstrapping allows control over how your builds are prepared.
  • Subversion 1.7 working copy format support.
  • Git submodule support.

Our plan now is to focus on performance and stability for the next few builds before pushing 2.5 as the stable release. In the mean time you can find out more, and even try beta builds, by visiting our beta program page. Happy building!

Pulse Continuous Integration Server 2.5 Alpha!

Cobwebs may be creeping over the blog, but not over Pulse: we’ve just published the first Pulse 2.5 alpha build! The Pulse 2.5 series is focused (even more so than usual) on customer feedback. We’ve got plenty of great ideas from the Pulse community, and we’re folding in some of the best ones in this release series. Updates so far include:

  • Browse and dashboard view filtering (by project health).
  • A new MSTest post-processor plugin.
  • Increased visibility of build (and other) comments.
  • Comments on agents, to communicate with other users.
  • Properties on agents, for simpler parameterisation.
  • The ability to configure SCMs using resources.
  • A resource repository on the Pulse master.
  • A new option to attach build logs to notification emails.
  • Pre-stage hooks, which run just before the stage is dispatched to the agent.
  • SCM inclusion filters, complementing existing exclusion filters.
  • Inverse resource requirements (i.e. requiring the absence of a resource).
  • Smart label renaming.
  • A separate permission for the ability to clean up build directories.
  • Support for Perforce streams.

There’s plenty more to come in subsequent alpha builds, too. In fact, we’ve started the groundwork for some other larger changes already in 2.5.0.

To grab yourself a fresh-from-the-oven 2.5 alpha build, or just to find out more, head over to our alpha program page.

Pulse Continuous Integration Server 2.4 Released!

Happy days: we’ve now released Pulse 2.4! Thanks to all those that provided feedback during the Pulse 2.4 beta period. Here’s a recap of the major updates in this release:

  • Mercurial support: in the form of a new plugin.
  • Maven 3 support: including a command, post-processor and resource discovery.
  • Agents page updates: with graphical status and more convenient navigation.
  • Reworked agent status tab: with more build links and efficient live updates.
  • New agent history tab: quickly browse all builds that involved an agent.
  • Reworked server activity tab: showing build stages nested under active builds.
  • Pause server: admins can pause the build queue, so all triggers are ignored.
  • New server history tab: showing build history across all projects.
  • Restyled info and messages tabs: for both the agents and server sections.
  • Improved process termination: to make terminating builds more reliable.
  • Kill build action: for quicker build termination forgoing full clean up.
  • Improved changelist views: these views have been reworked in the new style.
  • Pinned builds: mark builds that should never be deleted or cleaned.
  • Templated field actions: easily find or revert to an inherited value.
  • Introduce parent refactoring: adjust your template hierarchy over time.
  • Pluggable resource discovery: automatically locate build tools and libraries.
  • Subversion changelist support: easily submit a changelist as a personal build.
  • … and more: extra UI touches, improved performance, more plugin support implementations and more.

The new in 2.4 page on our website has more details and a few screenshots. Or you can simply download and try Pulse 2.4 for free.

Pulse 2.4 Release Candidate

After a few iterations of Pulse 2.4 beta builds, we’ve finally reached a stable enough state to declare our first release candidate. Since the original 2.4 beta post, we haven’t just been squishing bugs, but have come up with several more improvements:

  • New agent history tab: quickly browse all builds that involved an agent.
  • Pause server: admins can pause the build queue, so all triggers are ignored.
  • Improved changelist views: these views have been reworked in the new style.
  • Pinned builds: mark builds that should never be deleted or cleaned.
  • Subversion changelist support: easily submit a changelist as a personal build.
  • Faster browse view: most of these optimisations also appear in Pulse 2.3.
  • Improved process termination: to make terminating builds more reliable.

We know from feedback that these changes will be popular! Check them out yourself: release candidate builds are available from the Beta Program page.

Android: Automated Build Versioning

The Android SDK documentation offers a guide for Versioning Your Applications. In short, applications have two version fields:

  1. versionCode: a simple integer that you should increase with each new release. Being an integer, the code can be processed in software (e.g. other applications) if required.
  2. versionName: a free-form string you can use as the customer-facing version for your application. A typical format would be dotted-decimal (e.g. 1.2.0), although any form is allowed.

This version scheme is simple and allows you the flexibility to choose your own naming scheme. One annoyance, however, is that the version is specified in the AndroidManifest.xml file:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
        package="com.zutubi.android.example"
        android:versionCode="1"
        android:versionName="1.0">
    <application android:name=".ExampleApplication">
        ...
    </application>
</manifest>

The problem here is that the manifest is a version controlled file that also contains other vital information about the application. This makes it inconvenient to automate stamping your application with a new version for each release.

This motivated me to create a couple of Ant tasks which make it easy to set the version of your Android applications from your Ant build. I’ve released them as part of an initial version of the zutubi-android-ant project. So far there are two tasks: setversion and bumpversion.

setversion

This task allows you to explicitly set the version code and/or name for your application to specified values. Typically these values would be specified indirectly via Ant properties. The properties would be set by some other process (e.g. automatic generation based on the date, an input prompt, a property from your build server, or an read from an unversioned properties file).

For example, to set the versions based on Ant properties, prompting if those properties are not yet defined:

<target name="set-version">
    <input addproperty="version.code" message="Version Code:"/>
    <input addproperty="version.name" message="Version Name:"/>
    <zaa:setversion code="${version.code}" name="${version.name}"/>
</target>

bumpversion

Perhaps you simply want to increment the version code and point number for your application each time you make a build. (You may also change the major.minor version at times, but that is a manual choice.) You can do this simply with the bumpversion task. By default, this task will just increment the existing version code. If you set the optional attribute bumpname to true, the last element of the version name will also be incremented. For example:

<target name="bump-version">
    <zaa:bumpversion bumpname="true"/>
</target>

If your existing manifest has a version code of 3 and a version name of 1.0.3, then after bumping you will have a code of 4 and a name of 1.0.4. Simple!

Try It Out

The zutubi-android-ant GitHub page has a full README and simple examples showing in more detail how you can use these tasks. Happy versioning!

Android JUnit XML Reports: Now With Test Durations

I’ve been planning to add test case durations to the XML reports generated by android-junit-report for some time. This morning, however, the magic of open source caught up with me. I received a pull request from another GitHub user who had implemented durations already!

So, with thanks to Tim from todoroo, there is a new release of android-junit-report which outputs the duration (in seconds) for each test case. This matches the output produced by the regular Ant JUnit report task, and thus should be compatible with all tools that read the format. Durations are not added to test suites, for the same reason the test case counts are not: it would require buffering. For my own use this is no big deal, because Pulse will sum the times of all cases to give an idea of the total suite time when it is not provided directly.

You can grab this new version of (1.1, build 4) from the GitHub downloads page. Or, as always, you can access the latest release build directly from the build server (click on “jar” in the “featured artifacts” table on the right of the page).

Pulse Continuous Integration Server 2.2!

Big news today: Pulse 2.2 has graduated to stable! This release includes a stack of new features and improvements, including:

  • Build UI overhaul: all tabs improved and restyled.
  • New logs tab: making it easier to access stage logs.
  • Featured artifacts: choose which artifacts should appear prominently.
  • Build navigator: easily move forward and backward through history.
  • Working copy browser: view working copies for in progress builds.
  • Move refactoring: move projects and agents in the template hierarchy.
  • Template navigation: navigate directly up and down a hierarchy.
  • Subscription by label: subscribe to notifications by project groups.
  • Agent executing stages: see what all agents are building at a glance.
  • Subversion exports: for faster and smaller builds.
  • Performance improvements: key for larger installations.

See the new in 2.2 page for full details.

In conjunction with this release, we’ve also given our website a complete overhaul. The new site has a fresher look, and communicates the key features of Pulse more directly. The updates also include some new features:

  • RSS feeds for news items.
  • Links to our latest blog posts on the front page.
  • An improved buying process, allowing multiple licenses to be purchased in one transaction.
  • Self-service renewal payments – just enter your current license key and go!
  • A more user-friendly downloads page.

We hope you enjoy the new release, and the new site. And please, let us know what you think!

Simpler Ant Builds With the Ant Script Library

Introduction

Ant may be unfashionable these days, but it still has its advantages. Key among these are familiarity and simplicity: most Java developers have worked with Ant, and with an Ant build what you get is what you see. A major disadvantage, though, is that Ant provides very little out-of-the-box. When you start a new project, you’ve got a lot of grunt work to endure just to get your code compiled, packaged, and tested. An all-too-common solution, in the grand tradition of make, is to copy a build file from an existing project as an easy starting point.

Over the years, though, Ant has gradually expanded support for creating reusable build file snippets. On top of this a few projects have emerged which aim to simplify and standardise your Ant builds, including:

Today I’ve taken my first proper look at the latter, and so far I like what I see.

The Ant Script Library

In the author Joe Schmetzer’s own words:

The Ant Script Library (ASL) is a collection of re-usable Ant scripts that can be imported into your own projects. The ASL provides a number of pre-defined targets that simplify setting up build scripts for a new project, bringing re-use and consistency to your own Ant scripts.

ASL consists of several Ant XML files, each of which provides a group of related functionality via predefined targets. For example, the asl-java-build.xml file defines targets for compiling and packaging Java code. The asl-java-test.xml file extends this with the ability to run JUnit tests, and so on. Essentially, ASL packages up all the grunt work, allowing you to concentrate on the small tweaks and extra targets unique to your project. The modular structure of ASL, combined with the fact that it is just Ant properties and targets, makes it easy to take what you like and leave the rest.

An Example

Allow me to illustrate with a simple project I have been playing with. This project has a straightforward directory structure:

  • <project root>
    • asl/ – the Ant Script Library
    • build.xml – Ant build file
    • lib/ – Jar file depedencies
    • src/ – Java source files
    • test/ – JUnit-based test source files

To add ASL to my project, I simply downloaded it from the project download page and unpacked it in the asl/ subdirectory of my project1. Then I can start with a very simple build file that supports building my code and running the tests:

<?xml version="1.0" encoding="utf-8"?>
<project name="zutubi-android-ant" default="dist">
    <property name="java-build.src-dir" location="src"/>
    <property name="java-test.src-dir" location="test"/>
    <property name="java-build.lib-dir" location="libs"/>
	
    <property name="asl.dir" value="asl"/>

    <import file="${asl.dir}/asl-java-build.xml"/>
    <import file="${asl.dir}/asl-java-test.xml"/>
</project>

Notice that I am using non-standard source locations, but that is easily tweaked using properties which are fully documented. With this tiny build file, let’s see what targets ASL provides for me:

$ ant -p
Buildfile: build.xml

Main targets:

 clean                 Deletes files generated by the build
 compile               Compiles the java source
 copy-resources        Copies resources in preparation to be packaged in jar
 dist                  Create a distributable for this java project
 generate              Generates source code
 jar                   Create a jar for this java project
 test-all              Runs all tests
 test-integration      Runs integration tests
 test-run-integration  Runs the integration tests
 test-run-unit         Runs the unit tests
 test-unit             Runs unit tests
Default target: dist

It’s delightfully simple!

Adding Reports

It gets better: ASL also provides reporting with tools like Cobertura for coverage, FindBugs for static analysis and so on via its asl-java-report.xml module. The full range of supported reports can be seen in the report-all target:

<target name="report-all"
        depends="report-javadoc, report-tests, report-cobertura, report-jdepend, report-pmd, report-cpd, report-checkstyle, report-findbugs" 
        description="Runs all reports"/>

Having support for several tools out-of-the-box is great. For my project, however, I’d like to keep my dependencies down and I don’t feel that I need all of the reporting. Although the choice of reports is not something that is parameterised by a property, it is still trivial to override by providing your own report-all target. This shows the advantage of everything being plain Ant targets:

<?xml version="1.0" encoding="utf-8"?>
<project name="zutubi-android-ant" default="dist">
    <property name="java-build.src-dir" location="src"/>
    <property name="java-test.src-dir" location="test"/>
    <property name="java-build.lib-dir" location="libs"/>
	
    <property name="asl.dir" value="asl"/>

    <import file="${asl.dir}/asl-java-build.xml"/>
    <import file="${asl.dir}/asl-java-test.xml"/>
    <import file="${asl.dir}/asl-java-report.xml"/>
    
    <target name="report-all"
            depends="report-javadoc, report-tests, report-cobertura, report-pmd, report-checkstyle" 
            description="Runs all reports"/>
</project>

Here I’ve included the java-report module, but defined my own report-all target that depends on just the reports I want. This keeps things simple, and allows me to trim out a bunch of ASL dependencies I don’t need.

Conclusion

I’ve known of ASL and such projects for a while, but this is the first time I’ve actually given one a go. Getting started was pleasantly simple, as was applying the small tweaks I needed. So next time you’re tempted to copy an Ant build file, give ASL a shot: you won’t regret it!


1 In this case I downloaded the full tarball including dependencies, which seemed on the large side (21MB!) but in fact can be easily trimmed by removing the pieces you don’t need. Alternatively, you can start with the basic ASL install (sans dependencies) and it can pull down libraries for you. Sweet :).