a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis

Zutubi

Java 6: Using Python Via The New Scripting Engine

SUMMARY

Do you ever find yourself writing Java code that interacts with external processes and systems, but wish you could use a scripting language more suited to the task? If you have Java 6 available to you, then you are in luck.

MOTIVATION

Interacting with the file system and external processes, whilst possible, is not what I would consider to be one of Javas’ strong points. So, when thinking about acceptance testing for a product such as Pulse, it makes sense that much of the deployment / starting / stopping is handled by a scripting language (Python in this case).

This has served us well up until recently, when we added features such as database migration and automated backups to Pulse.

Let me explain why:

Each time we ran our acceptance tests on a new Pulse build, we took the deliverable artifact, deployed and started it using the above mentioned scripts. We then triggered our Java test suit which ran through the setup tests first (effectively setting up a fresh installation) followed by all of the tests that require a running installation.

This works well so long as the context of the acceptance tests is a running Pulse server. But now we need tests that run in the context of multiple invocations of the server (start pulse – back it up – make changes – stop pulse, restart pulse with the restore flag etc). One option would be to write these tests in Python (as we have done with our automated agent upgrade tests) which works well, but means that tests are now split amongst multiple languages. The other option is to continue to drive all the tests from Java and drive our external scripts from Java as well.

SOLUTION

The new Java 6 scripting engine support provides a very clean solution to the problem we faced by running our scripts directly within the JVM and allowing our Java code to access the script components.

Using our situation for the example, with the help of jython, we were able to wrap our Python script components behind Java interfaces, and using a simple factory method, we are able to run our scripts as if they were plain Java objects.

Here is the code.

Firstly, we have the python scripts, that define a PulsePackage and Pulse classes, with the functionality to extract a package and then start / stop an installation.

class PulsePackage:
   
  def __init__(self, packageFile):
    self.packageFile = packageFile
   
  def extractTo(self, destDir):
    if not os.path.exists(destDir):
      os.makedirs(destDir)
       
    var = {‘destDir’:destDir, ‘packageFile’:self.packageFile}
      if self.packageFile.endswith(‘.zip’):
        os.system("unzip -qd %(destDir)s %(packageFile)s" % var)
      else:
        os.system("tar -zxC %(destDir)s -f %(packageFile)s" % var)
       
    return Pulse(os.path.join(destDir, self.getBasename()))
       
class Pulse:

  def __init__(self, baseDir, port=8080):
    """ snipped """
       
  def start(self, wait=True, service=False):
    """ snipped """
       
  def stop(self, timeout=60, service=False):
    """ snipped """

Then we define the Java interfaces that represent these python classes.

public interface PulsePackage
{
  Pulse extractTo(String target);
}

public interface Pulse
{
  void start();
 
  void stop();
}

Next, we define the factory that gives us access to the Python objects from within our Java tests.

public class JythonPulsePackageFactory
  implements PulsePackageFactory
{
  private File scripts = null;
   
  private Invocable invocableEngine;

  public JythonPulsePackageFactory()
  {
    this.scripts = new File("scripts");

    ScriptEngine jythonEngine =
        new ScriptEngineManager().
            getEngineByName("python");
       
    // the packageFactory.py script
    // contains our python class
    // definitions, and is loaded
    // here into the script engine.
    Reader script = new FileReader(
      new File(scripts, "packageFactory.py"));
    jythonEngine.eval(script);

    // this invocable engine now
    // allows us access to execute
    // python defined methods.
    invocableEngine =
      (Invocable) jythonEngine;
  }

  public PulsePackage createPackage(File pkg)
    throws Exception
  {
    // the createPackage method is
    // implemented in python, and
    // returns an instance (handle)
    // to python implemented extension
    // of our PulsePackage interface.
    return (PulsePackage)
      invocableEngine.invokeFunction(
          "createPackage", pkg.getCanonicalPath());
  }
}

And finally, where the magic happens, we bridge the gap between the Java and the python instances by defining the create package method in the python script.

def createPackage(packageFile):
    return PulsePackage(packageFile)

and update the python class definitions, extending the java interfaces:

class Pulse(com.zutubi.pulse.acceptance.Pulse):
  """ <snipped> """

class PulsePackage(com.zutubi.pulse.acceptance.PulsePackage):
  """ <snipped> """

And this is how the resulting java code looks for deploying and starting a new pulse installation:

File pkg =  new File("testing-packages/pulse-2.0.0.zip");

JythonPulsePackageFactory factory =
    new JythonPulsePackageFactory();

PulsePackage pulsePackage = factory.createPackage(pkg);

File destDir = new File("tmp");
Pulse pulse = pulsePackage.extractTo(destDir.getCanonicalPath());
pulse.start();

Liked this post? Share it!

11 Responses to “Java 6: Using Python Via The New Scripting Engine”

  1. July 16th, 2008 at 11:28 pm

    baoilleach says:

    I’ve heard about the new scripting support in Java 6, but could you spell out for me exactly what you can do now that previously was not possible? I thought that even with Java 5, you could use Jython from inside Java programs.

  2. July 17th, 2008 at 1:27 am

    Daniel says:

    Yes, everything can be done using the existing jython interpretor.

    In fact, the implementation of the jython scripting support is a very thin layer around the existing jython interpretor.

    What the Java 6 scripting support gives you is a framework into which you can load other scripting engines, allowing you to run a variety of different scripts. By default, it will support at least javascript with the JDK. Jython, freemarker, groovy, jelly, jruby, ognl, velocity and others are all freely available.

  3. July 30th, 2008 at 3:01 pm

    Janarthanan says:

    Hi,
    Thanks for sharing the information. I have few clarifications to make. Why do we have to opt to use scripting language to perform these tasks that can be performed in Ant.

  4. July 30th, 2008 at 3:20 pm

    Daniel says:

    Hi Janarthanan,

    Certainly, these things can be done within Ant, but Ant really isnt the best place for such things. Some tasks require the flexibility offered by a programming language, which Ant is not.

  5. July 30th, 2008 at 3:31 pm

    Janarthanan says:

    Daniel,
    Thanks for the response. What kind of tasks would they be. Would I I able to classify them cleanly and make sure that box gets address with a clean bunch of python API’s, when ever we perform systemic tasks before we perform Regression or any other form testing few things happen outside the intended context(am i sensible ?),
    1)talk to Version control Management System
    2)Do Compilation on the Source Code
    3)Packing and Unpacking
    4)Moving Files from one place to other,
    5)Starting/Stopping Servers(Databases/Application Servers)
    6)Executing Remote Commands(Ant goes down pretty hard here, this is one place where I see Ant cant do certain things if, we have to go outside the box to do certain things ,like perform scp, or starting a mysql server in a remote machine)

    I think I am slowly getting the picture.
    Thanks for the article. May be I should come back with more clarity and ask few more questions :-)

    jana

  6. July 30th, 2008 at 3:41 pm

    Daniel says:

    Hi Jana,

    All of the things you mentioned can be done within Ant, if you feel comfortable doing so. In fact we do some of these things in ant ourselves, although some of the more complicated stuff we put in to scripts that we then trigger via ant.

    The reason I needed to use a scripting language was not because things could not be done in ant, but because I needed to start/stop servers as part of individual acceptance tests. For instance, one acceptance test checks that all of the Pulse startup options are correctly handled. It would not be practical to try to call out to ant from within a single acceptance test case.

    As you point out in your #6, Ant does have its limitations, limitations that would be solved by using a scripting language designed for the task.

    I hope that somewhat answers your question. :)

    -Daniel

  7. July 30th, 2008 at 6:08 pm

    Janarthanan says:

    Daniel,
    I think we are on the same boat AFA the last point is concerned. I realize, the potential to use the scripting engine with JVM seems far more deeper than what I initially thought. And thanks for freewheeling this discussion. As I see it, it is definitely worth a read for other people too because of your candid response. Thanks again.

    regards,
    jana

  8. September 24th, 2008 at 6:25 am

    Duncan says:

    Can you please post the full code. I am a little confused, and am strugaling to get it working

  9. July 20th, 2009 at 12:54 am

    Enlaces y noticias - Angel "Java" Lopez says:

    [...] Cada vez más lenguajes dinámicos sobre máquinas virtuales, ver Java 6: Using Python via the new Scripting Engine. [...]

  10. April 3rd, 2010 at 8:47 am

    Waldo Keywan says:

    Wow I never new that, thanks.

  11. December 11th, 2013 at 11:41 pm

    Jitendar says:

    Thanks for sharing the code, it totally saved my life..

Leave a Reply