a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis

Zutubi

Android: JUnit XML Reports with Gradle

The Android development tools project has seen big changes over the last year. The original Eclipse ADT development environment was superseded late last year by Android Studio — a new IDE based on Intellij. Under the hood Android Studio also uses a new command line build system based on Gradle, replacing the previous Ant-based system. I’ve been keen to find out how these changes impact the integration of Android test reports with continuous integration servers like Pulse.

Summary

  • Android JUnit Report is redundant.
  • Run on-device Android tests with:
    ./gradlew connectedAndroidTest
  • Collect reports from:
    app/build/outputs/androidTest-results/connected/*.xml

 

Details

The original Ant-based build system for Android didn’t produce XML test reports for instrumentation tests (i.e. those that run on-device), prompting me to create the Android JUnit Report project. Android JUnit Report produced XML output similar to the Ant JUnit task, making it compatible with most continuous integration servers. The good news is: Android JUnit Report is now redundant. The new Gradle-based build system produces sane XML test reports out of the box. In fact, they’re even more complete than those produced by Android JUnit Report, so should work with even more continuous integration servers.

The only downside is the documentation, which is a little confusing (while there are still documents for the old system about) and not very detailed. With a bit of experimentation and poking around I found how to run on-device (or emulator) tests and where the XML reports were stored. With a default project layout as created by Android Studio:

ASDemo.iml
app/
  app.iml
  build.gradle
  libs/
  proguard-rules.pro
  src/
    androidTest/
    main/
build.gradle
gradle
gradle.properties
gradlew
gradlew.bat
local.properties
settings.gradle

You get a built-in version of Gradle to use for building your project, launched via gradlew. To see available tasks, run:

$ ./gradlew tasks

(This will download a bunch of dependencies when first run.) Amongst plenty of output, take a look at the Verification Tasks section:

Verification tasks
------------------
check - Runs all checks.
connectedAndroidTest - Installs and runs the tests for Debug build on connected devices.
connectedCheck - Runs all device checks on currently connected devices.
deviceCheck - Runs all device checks using Device Providers and Test Servers.
lint - Runs lint on all variants.
lintDebug - Runs lint on the Debug build.
lintRelease - Runs lint on the Release build.
test - Run all unit tests.
testDebug - Run unit tests for the Debug build.
testRelease - Run unit tests for the Release build.

The main testing target test does not run on-device tests, only unit tests that run locally. For on-device tests you use the connectedAndroidTest task. Try it:

$ ./gradlew connectedAndroidTest
...
:app:compileDebugAndroidTestJava
:app:preDexDebugAndroidTest
:app:dexDebugAndroidTest
:app:processDebugAndroidTestJavaRes UP-TO-DATE
:app:packageDebugAndroidTest
:app:assembleDebugAndroidTest
:app:connectedAndroidTest
:app:connectedCheck

BUILD SUCCESSFUL

Total time: 33.372 secs

It’s not obvious, but this produces compatible XML reports under:

app/build/outputs/androidTest-results/connected

with names based on the application module and device. In your continuous integration setup you can just collect all *.xml files in this directory for reporting.

Although the new build system has killed the need for my little Android JUnit Report project, this is a welcome development. Now all Android developers get better test reporting without an external dependency. Perhaps it will even encourage a few more people to use continuous integration servers like Pulse to keep close tabs on their tests!

Liked this post? Share it!

Leave a Reply