For simplicity and efficiency, android-junit-report does not produce files in the exact format used by the Ant JUnit task. In the 1.1 release there are two main differences:

1. A single report file is produced containing all test suites.
2. Redundant information, such as the number of cases, failures etc is not added using attributes on the testsuite tag.

It turns out the first of these restrictions caused multiple users issues with tools accustomed to handling a single report file per suite. So in the latest 1.2 release I have added a new multiFile option. When this option is enabled, android-junit-report produces a separate output file for each test suite. This does mean that to retrieve the results from the device you will need to pull a whole directory.

To enable this option from an Ant build, you can override the default run-tests target as follows:

You can learn more about android-junit-report and/or download the new release using the links below:

• Original blog post
• Pre-built jar downloads
• Project home and documentation
• Build Server

Ant may be unfashionable these days, but it still has its advantages. Key among these are familiarity and simplicity: most Java developers have worked with Ant, and with an Ant build what you get is what you see. A major disadvantage, though, is that Ant provides very little out-of-the-box. When you start a new project, you’ve got a lot of grunt work to endure just to get your code compiled, packaged, and tested. An all-too-common solution, in the grand tradition of make, is to copy a build file from an existing project as an easy starting point.

Over the years, though, Ant has gradually expanded support for creating reusable build file snippets. On top of this a few projects have emerged which aim to simplify and standardise your Ant builds, including:

• EasyAnt
• Ant Master Build Scripts
• Ant Script Library

Today I’ve taken my first proper look at the latter, and so far I like what I see.

The Ant Script Library

In the author Joe Schmetzer’s own words:

The Ant Script Library (ASL) is a collection of re-usable Ant scripts that can be imported into your own projects. The ASL provides a number of pre-defined targets that simplify setting up build scripts for a new project, bringing re-use and consistency to your own Ant scripts.

ASL consists of several Ant XML files, each of which provides a group of related functionality via predefined targets. For example, the asl-java-build.xml file defines targets for compiling and packaging Java code. The asl-java-test.xml file extends this with the ability to run JUnit tests, and so on. Essentially, ASL packages up all the grunt work, allowing you to concentrate on the small tweaks and extra targets unique to your project. The modular structure of ASL, combined with the fact that it is just Ant properties and targets, makes it easy to take what you like and leave the rest.

An Example

Allow me to illustrate with a simple project I have been playing with. This project has a straightforward directory structure:

• <project root>
  • asl/ – the Ant Script Library
  • build.xml – Ant build file
  • lib/ – Jar file depedencies
  • src/ – Java source files
  • test/ – JUnit-based test source files

To add ASL to my project, I simply downloaded it from the project download page and unpacked it in the asl/ subdirectory of my project¹. Then I can start with a very simple build file that supports building my code and running the tests:

<?xml version="1.0" encoding="utf-8"?>
<project name="zutubi-android-ant" default="dist">
    <property name="java-build.src-dir" location="src"/>
    <property name="java-test.src-dir" location="test"/>
    <property name="java-build.lib-dir" location="libs"/>

    <property name="asl.dir" value="asl"/>

    <import file="${asl.dir}/asl-java-build.xml"/>
    <import file="${asl.dir}/asl-java-test.xml"/>
</project>

Notice that I am using non-standard source locations, but that is easily tweaked using properties which are fully documented. With this tiny build file, let’s see what targets ASL provides for me:

$ ant -p
Buildfile: build.xml

Main targets:

 clean                 Deletes files generated by the build
 compile               Compiles the java source
 copy-resources        Copies resources in preparation to be packaged in jar
 dist                  Create a distributable for this java project
 generate              Generates source code
 jar                   Create a jar for this java project
 test-all              Runs all tests
 test-integration      Runs integration tests
 test-run-integration  Runs the integration tests
 test-run-unit         Runs the unit tests
 test-unit             Runs unit tests
Default target: dist

It’s delightfully simple!

Adding Reports

It gets better: ASL also provides reporting with tools like Cobertura for coverage, FindBugs for static analysis and so on via its asl-java-report.xml module. The full range of supported reports can be seen in the report-all target:

<target name="report-all"
        depends="report-javadoc, report-tests, report-cobertura, report-jdepend, report-pmd, report-cpd, report-checkstyle, report-findbugs"
        description="Runs all reports"/>

Having support for several tools out-of-the-box is great. For my project, however, I’d like to keep my dependencies down and I don’t feel that I need all of the reporting. Although the choice of reports is not something that is parameterised by a property, it is still trivial to override by providing your own report-all target. This shows the advantage of everything being plain Ant targets:

<?xml version="1.0" encoding="utf-8"?>
<project name="zutubi-android-ant" default="dist">
    <property name="java-build.src-dir" location="src"/>
    <property name="java-test.src-dir" location="test"/>
    <property name="java-build.lib-dir" location="libs"/>

    <property name="asl.dir" value="asl"/>

    <import file="${asl.dir}/asl-java-build.xml"/>
    <import file="${asl.dir}/asl-java-test.xml"/>
    <import file="${asl.dir}/asl-java-report.xml"/>

    <target name="report-all"
            depends="report-javadoc, report-tests, report-cobertura, report-pmd, report-checkstyle"
            description="Runs all reports"/>
</project>

Here I’ve included the java-report module, but defined my own report-all target that depends on just the reports I want. This keeps things simple, and allows me to trim out a bunch of ASL dependencies I don’t need.

Conclusion

I’ve known of ASL and such projects for a while, but this is the first time I’ve actually given one a go. Getting started was pleasantly simple, as was applying the small tweaks I needed. So next time you’re tempted to copy an Ant build file, give ASL a shot: you won’t regret it!

–
¹ In this case I downloaded the full tarball including dependencies, which seemed on the large side (21MB!) but in fact can be easily trimmed by removing the pieces you don’t need. Alternatively, you can start with the basic ASL install (sans dependencies) and it can pull down libraries for you. Sweet .

This post introduces the Android JUnit Report Test Runner, a custom instrumentation test runner for Android that produces XML test reports. Using this runner you can integrate your Android test results with tools that understand the Ant JUnit task XML format, e.g. the Pulse Continuous Integration Server.

The motivation and details of the runner are discussed below. For the impatient: simply head on over to the project home page on GitHub and check out the README.

Introduction

If you’ve been following my recent posts you’ll know that I’ve been figuring out the practical aspects of testing Android applications. And if you’ve been following for longer, you might know that my day job is development of the Pulse Continuous Integration Server. So it should come as no surprise that in my latest foray into the world of Android testing I sought to bring the two together .

Status Quo

Out of the box, the Android SDK supports running functional tests on a device or emulator via instrumentation. Running within Eclipse, you get nice integrated feedback. Unfortunately, though, there are no real options for integrating with other tools such as continuous integration servers. Test output from the standard Ant builds is designed for human consumption, and lacks the level of detail I’d like to see in my build reports.

The Solution

On the upside, having access to the Android source makes it possible to examine how the current instrumentation works, and therefore how it can be customised. I found that the default InstrumentationTestRunner may be fairly easily extended to hook in extra test listeners. So I’ve implemented a custom JUnitReportTestRunner that does just that, with a listener that generates a test report in XML format. The format is designed to be largely compatible with the output of the Ant JUnit task’s XML formatter — the most widely supported format in the Java world. Tools like Pulse can read in this format to give rich test reporting.

How It Works

As mentioned, the JUnitReportTestRunner extends the default InstrumentationTestRunner, so it can act as a drop-in replacement. The custom runner acts identically to the default, with the added side-effect of producing an XML report.

For consistency with the SDK’s support for generating coverage reports, the XML report is generated in the file storage area of the target application. The default report location is something like:

/data/data/<tested application package>/files/junit-report.xml

on the device. To retrieve the report, you can use adb pull, typically as part of your scripted build.

Using the Runner

Full details on using the runner are provided in the README on the project home page. Briefly:

• Add the android-junit-report-<version>.jar to the libraries for your test application.
• Replace all occurrences of android.test.InstrumentationTestRunner with com.zutubi.android.junitreport.JUnitReportTestRunner:
  • In the android:name attribute of the instrumentation tag in you test application’s AndroidManifest.xml.
  • In the test.runner property in the Ant build for your test application (before calling the Android setup task).
  • In the Instrumentation runner field of all Android JUnit Run Configurations in your Eclipse project.
• Add logic to your Ant build to run adb pull to retrieve the report after the tests are run.

As an example for retrieving the report in your Ant build:

<target name="fetch-test-report">
    <echo>Downloading XML test report...</echo>
    <mkdir dir="${reports.dir}"/>
    <exec executable="${adb}" failonerror="true">
        <arg line="${adb.device.arg}"/>
        <arg value="pull" />
        <arg value="/data/data/${tested.manifest.package}/files/junit-report.xml" />
        <arg value="${reports.dir}/junit-report.xml" />
    </exec>
</target>

In the Wild

You can see a complete example of this in action in my simple DroidScope Android application. The custom runner is applied in the droidscope-test application in the test/ subdirectory. You can even see the test results being picked up by Pulse on our demo server. Note that some of the tests are pure unit tests, which are run on a regular JVM, whereas others are run with the custom runner on an emulator. It’s nice for all the results to be collected together!

Naturally, as I started working on an Android application, I tried to apply the same technique. Problems arose when I tried to combine DI with the Android SDK’s Testing and Instrumentation support. In particular, I am yet to find a suitable way to combine DI with functional testing of Android activities via ActivityInstrumentationTestCase2. When testing an activity using the instrumentation support, injection of dependencies is foiled by a couple of factors:

1. Constructor injection is impossible, as activities are constructed by the framework. I experimented with various ways of creating the Activity myself, but was unable to maintain a connection with the Android system for true functional testing.
2. Setter injection is fragile, as activities are started by the framework as soon as they are created. There is no time to set stub dependencies between the instantiation of the Activity and its activation.

Not ready to give DI away, I scoured the web for existing solutions to this problem. Although I did find some DI libraries with Android support (notably Guice no AOP and roboguice which builds upon it), the only testing support I found was restricted to unit tests. Although roboguice has support for Activities, it relies on being able to obtain a Guice Injector from somewhere — which just shifts the problem by one level of indirection.

Given how complex any DI solution was going to become (if indeed it is possible at all) I decided to step back and consider alternatives. A classic alternative to DI is the Service Locator pattern: where objects ask a central registry for their dependencies. Martin Fowler’s article Inversion of Control Containers and the Dependency Injection pattern compares and contrasts the two patterns in some detail. Most importantly: a Service Locator still allows you to substitute in different implementations of dependencies at test time. The main downside is each class is dependent on the central registry — which can make them harder to reuse. As I’m working with Activities that are unlikely to ever be reused outside of their current application, this is no big deal.

Implementation-wise, I went with the simplest registry that works for me. I found it convenient to use my project’s Application implementation as the registry. In production, the Application onCreate callback is used to create all of the standard dependency implementations. These dependencies are accessed via simple static getters. Static setters are exposed to allow tests to drop in whatever alternative dependencies they desire. A contrived example:

public class MyApplication extends Application
{
    private static IService service;
    private static ISettings settings;

    @Override
    public void onCreate()
    {
        super.onCreate();
        if (service == null)
        {
            service = new ServiceImpl();
        }

        if (settings == null)
        {
            SharedPreferences preferences = PreferenceManager.getDefaultSharedPreferences(getApplicationContext());
            settings = new PreferencesSettings(preferences);
        }
    }

    public static IService getService()
    {
        return service;
    }

    public static void setService(IService s)
    {
        service = s;
    }

    public static ISettings getSettings()
    {
        return settings;
    }

    public static void setSettings(ISettings s)
    {
        settings = s;
    }
}

I access the dependencies via the registry in my Activity’s onCreate callback:

public class MyActivity extends Activity
{
    private IService service;
    private ISettings settings;

    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);

        service = MyApplication.getService();
        settings = MyApplication.getSettings();

        setContentView(R.layout.main);
        // ...
    }

    // ...
}

And I wire in my fake implementations in my functional test setUp:

public class MyActivityTest extends ActivityInstrumentationTestCase2<MyActivity>
{
    private MyActivity activity;

    public MyActivityTest()
    {
        super("com.zutubi.android.example", MyActivity.class);
    }

    @Override
    protected void setUp() throws Exception
    {
        super.setUp();
        MyApplication.setService(new FakeService());
        MyApplication.setSettings(new FakeSettings());
        activity = getActivity();
    }

    public void testSomething() throws Throwable
    {
        // ...
    }

After all of the angst over DI, this solution is delightful in its simplicity. It also illustrates that static is not always a dirty word when it comes to testing!

The box in question runs Windows Vista, and I noticed while poking around that some disk operations were very slow. In fact, deleting even a handful of files via Explorer took so long that I gave up (we’re talking hours here). About this time I fired up the Reliability and Performance Manager that comes with Vista (Control Panel > Administrative Tools). I noticed that there was constant disk activity, and a lot of it centered around C:\$MFT — the NTFS Master File Table.

I had already pared back the background tasks on this machine: the Recycle Bin was disabled, Search Indexing was turned off and Defrag ran on a regular schedule. So why was my file system so dog slow? The answer came when I looked into the AppData\Local\Temp directory for the user running the Pulse agent. The directory was filled with tens of thousands of entries, many of which were directories that themselves contained many files.

The junk that had built up in this directory was quite astounding. Although some of it can be explained by tests that don’t clean up after themselves, I believe a lot of the junk came from tests that had to be killed forcefully without time to clean up. It was also evident that every second component we were using was part of the problem – Selenium, JFreechart, JNA, Ant and Ivy all joined the party.

So, how to resolve this? Of course any tests that don’t clean up after themselves should be fixed. But in reality this won’t always work — especially given the fact that Windows will not allow open files to be deleted. So the practical solution is to regularly clean out the temporary directory. In fact, it’s quite easy to set up a little Pulse project that will do just that, and let Pulse do the work of scheduling it via a cron trigger. With Pulse in control of the scheduling there is no risk the cleanup will overlap with another build.

A more general solution is to start with a guaranteed-clean environment in the first place. After all, acceptance tests have a habit of messing with a machine in other ways too. Re-imaging the machine after each build, or using a virtual machine that can be restored to a clean state, is a more reliable way to avoid the junk. Pulse is actually designed to allow reimaging/rebooting of agents to be done in a post-stage hook — the agent management code on the master allows for agents to go offline at this point, and not try to reuse them until their status can be confirmed by a later ping.

Do not use loops for list operations. Learning from functional languages, looping isn’t the best way to work on collections. Suppose we want to filter a list of persons to those who can drink beer. The loop versions looks like:

List<Person> beerDrinkers = new ArrayList<Person>();
for (Person p: persons) {
    if (p.getAge() > 16) {
            beerDrinkers.add(p);
    }
}

This can – even in Java – be rewritten to a more a functional programming style. For example using Google collections filter:

Predicate<HasAge> canDrinkBeer = new Predicate<HasAge>() {
    public boolean apply(HasAge hasAge) {
        return hasAge.getAge() > 16;
    }
};
List<Person> beerDrinkers = filter(persons, canDrinkBeer);

Then Cedric Beust countered with:

No loops

Java is not very well suited to a functional style, so I think the first example is more readable than the second one that uses Predicates. I’m guessing most Java programmers would agree, even those that are comfortable with comprehensions and closures and especially those that aren’t.

I guess this means I’m not “most Java programmers”. Although I find the verbosity of the anonymous Predicate in Stephan’s code lamentable, aside from that I think the functional style is far superior. And I don’t believe it should be seen as in any way at odds with “normal” Java style. There’s no magic here: it’s just a straightforward anonymous class and filtering is a very simple concept to understand. Conceptually it is no harder than the explicit loop: in fact it is easier as you know the higher purpose as soon as you see “filter”.

The higher level of abstraction is the key to its superiority. Every non-trivial Java project will include many instances of searching, filtering, folding and so on of collections. Repeating this most basic of logic time after time fills the code with mechanical details that hide its true purpose. Why not get those details right once (in a library) and then never have to see them again?

In fact, once you embrace this style, things get even better. Not only can you reuse basic operations like filtering, you can also:

1. Compose these operations to form even higher-level manipulations.
2. Build up a reusable set of functors (Predicates, Transforms, etc) instead of using anonymous classes.

The latter point is important, as it allows you to remove an even more damaging sort of repetition — that of your project’s domain-specific logic. Reduce Stephan’s example to:

and it is clearly superior.

Really, I think it is a shame to see Java programmers hesitant to use this style. The ideas may come from the functional world, but there is nothing but basic Java code at work here. The abstractions are both simple and powerful, which to me is the very definition of elegance.

Luckily, dragging my project into the present wasn’t difficult. Along the way I also added my essential testing libraries: Hamcrest for matchers and Mockito for mocking (well, stubbing, but that’s another story). For posterity’s sake, and for others that share my testing tastes, here’s how it’s done.

Requirements

I’m assuming that you have Maven 2 installed already. If not, it’s trivial to:

1. Download the latest (2.1.0 at time of writing); and
2. Install it according to the instructions provided.

You can check if you have Maven ready to go by running:

Bootstrap

With the lack of an available alternative, I found it easiest to start with the default archetype. To create a new project, run something like:

Remember that if you’ve just installed Maven it will take this opportunity to download the internet. Be patient. If you’re new to Maven, you might also want to check out the 5 minute guide which walks through this in more detail.

Check that your bare application has been created:

Add the Dependencies

Next you need to update the project POM to add the testing libraries as dependencies to your build. This involves three changes:

1. Removing the default dependency on JUnit.
2. Adding new dependencies for TestNG, Hamcrest and Mockito to the “test” scope.
3. Configuring the compiler to accept Java 5 source.

The last step is necessary as the Maven default setting assumes Java 1.3 source, which apart from being ancient doesn’t support goodies such as annotations that are required for the new testing libraries. Your updated pom.xml file should look something like:

I’ve used the latest available versions of each of the libraries in this example — tweak them to suit your current reality.

Update the Sample Test

Now you’re ready to try updating the sample test case to use the trinity of TestNG, Hamcrest and Mockito. The easiest way to do this is to get Maven to generate a project for your IDE, e.g.

or:

Fire up your chosen IDE, open the AppTest class, and edit it to exercise all of the dependencies:

What are you waiting for? Try it out:

If you got this far, then everything you need is in place. Now you just have to … implement your project!

Extra Credit

If you poke about a bit, you will also find that the maven surefire plugin, which manages the tests, generates some reports by default. Along with HTML output, it also produces a JUnit-like XML report at:

This report is ideal for integration with a continuous integration server (in my case Pulse, naturally, but many will support it).

Happy testing!

Modularity

The focus of JDK 7 will be a new module system for Java which will in turn be used to modularise the JDK itself. The plan is to develop the system in the open (no more JSR-277-style project management) as part of the recently-announced Project Jigsaw. I encourage you to read Mark Reinhold’s blog entry for details.

One neat point that stood out for me was the desire to make this fit well with OS packaging systems (think RPMs, .debs, etc) — so it will be trivial to create an OS package for a Java module. This should couple nicely with the easy availability of the Sun JDK on Linux these days (now it is open source), making Java software more integrated with the native platform.

A much more critical point, however, is the idea of versioning as an enabler to make incompatible changes. Even as the news of Java’s death continues to roll in, it appears Sun is looking to put a mechanism in place that will allow them to finally make incompatible changes. Once you have a modular JDK and the ability to express dependencies on particular versions of modules, it follows that it should be possible to make incompatible changes to those modules in new versions. Perhaps in time this will allow a much-needed cleanup of the dark corners of the platform.

This also raises big questions for both OSGi and Maven. I don’t expect either of these entrenched projects to disappear, but there is certainly overlap and a need to adapt. On the OSGi front Sun is aiming for interoperability, but they claim that the Jigsaw module system will go down to a lower level than OSGi is capable (Update: it appears that this claim is controversial, see the comments below.).

Dynamism

Stepping away from Java the language and looking at the JVM platform, Brian Goetz and Alex Buckley spoke about the move “Towards a dynamic VM”. The talk focused on the efforts in JSR-292 (aka the invokedynamic JSR) to make hosting dynamic languages on the JVM easier and more efficient. The key component of this JSR is a new invokedynamic bytecode which can be used for dynamic method dispatch (when the actual method to call depends on more than just the type of the instance it is being invoked on).

The cool thing about invokedynamic is it will allow the various dynamic language runtimes to effectively plug method lookup functionality into the JVM. This will allow each dynamic language implementation, many of which have very flexible method binding, to take full control of method selection. This is key to efficient implementation of these languages on the JVM.

So hey, even if Java has lost its mojo, that isn’t the point anymore. It’s the platform, stupid!

Of course the meat of the conference is the talks. Here’s a rundown of those that I attended:

• Keynote: JavaFX: I have not taken much interest in JavaFX so far, so it was interesting to see a bit of it in action. The most interesting part was the demo of dragging a JavaFX application out of a browser and having it run standalone. It’s high on cool factor, and could be quite useful (though I doubt any user will think themselves to try it, so…).
• Keynote: Java and RFID: IBM and partners managed to take an interesting idea, namely running a live project at the conference tracking attendees with RFID, and turn it into a yawn-worthy presentation. I kept wanting to see the actual software details behind it (not which RFID printers I can buy…). When they did get to some software stuff, it felt like seriously over-engineered ADD (Acronym Driven Development).
• From Concurrent to Parallel: Brian Goetz gave a polished overview of the fork-join framework. I’m into java.util.concurrent, so it’s good to get a look at the next logical step.
• Effective Pairing: The Good, the Bad and the Ugly: Dave Nicolette had previously done this talk with rehearsed players for various pairing scenarios. As the players weren’t available, Dave took the ambitious route of getting audience volunteers to ad-lib instead. It didn’t go too badly considering the obstacles (e.g. audio not set up for the job), but on balance it was probably too ambitious. At least the audience did get involved, though.
• The Feel of Scala: this was my favourite of the day. Bill Venners is a very clear speaker, and this talk was polished (perhaps done before). I enjoyed the focus on real code samples, which were presented in an easily-followed fashion. Bill’s own motivations for using Scala reminded me of my own desire to check it out, which I now plan to do more seriously than before.
• Filthy Rich Android Clients: I’m planning to get a G1 so thought I’d check out some eye candy. Sadly, this talk was much too heavy on details and not enough on illustration. Learning how things work is great, but packing slides full of points won’t do it – there needs to be more sample code/hands on work. And this topic was just crying out for some eye-catching demos, of which there were too few in my opinion.
• Jython: Jythonistas Jim Baker and Tobias Ivarsson made a sometimes-awkward pairing for this talk. They focused too much on Python and Django and not enough on Jython itself for my taste. These are fine topics and we even use Django for zutubi.com, but I was expecting more content about Jython and cool ways to leverage Python on the Java platform. They did cover some of this territory, and get full marks for live coding and real demos.

Oh, and free beer and frites is always a good way to end a day…

I’m keen to hook up with any Zutubi customers, competitors or plain old build/continuous integration geeks while I’m there, so drop me an email if you’ll be there…