>Functional Testing Builds

>The various forms of Testing your code and regression testing have in many ways become standardized practices. Items like Unit Testing, Functional Testing, and Integration Testing are pretty normal when you are doing Continuous Integration. Coders are encouraged to do this when they are writing their application code, however, what about Build engineers? Can you make a change and in a short amount of time know that you haven’t broken the build?

Recently, I found out for ANT based builds, there is AntUnit. It allows you to do functional testing of your targets, and tasks to make sure that you aren’t breaking things after making changes. Having a build that is self testing is as important as having the code that it is building be self testing. The same benefits that you get from application testing apply directly to build testing.

I have been working on some proposed refactorings for the Athena Common Builder. By using AntUnit I can do some test driven development and functional testing to make sure that the changes I make are going to work as expected. An AntUnit follows the same general pattern of a JUnit 3 test:

  • setUp – if this target exists it is run before any test target
  • tearDown – if this target exists it is run after any test target
  • test* – if a target has test in it’s name, it is run during the Unit Test.

A sample unit test for an Athena refactoring I implemented looks like the following:

<project xmlns:au="antlib:org.apache.ant.antunit" xmlns="antlib:org.apache.tools.ant" default="run">

<property name="relengCommonBuilderDir" location="../../" />
<taskdef uri="antlib:org.apache.ant.antunit" resource="org/apache/ant/antunit/antlib.xml">
<pathelement location="${relengCommonBuilderDir}/lib/ant-antunit.jar" />
<pathelement location="../lib/ant-antunit.jar" />
<pathelement location="./lib/ant-antunit.jar" />

<import file="${relengCommonBuilderDir}/tools/scripts/tasks/getant4eclipse.xml"/>

<!-- is called prior to the test -->
<target name="setUp">
<property name="downloadsDir" location="${java.io.tmpdir}/antunit/downloads"/>
<delete file="${relengCommonBuilderDir}/lib/ant4eclipse.jar"/>

<!-- is called after the test, even if that caused an error -->
<target name="tearDown">
<delete dir="${downloadsDir}" quiet="true" />

<!-- test the MacroDef for downloading Files -->
<target name="testGetAnt4EclipseZip">
<property name="file" value="ant4eclipse-0.5.0.rc1.zip"/>
<property name="thirdPartyDownloadLicenseAcceptance" value="I accept"/>
<au:assertFileDoesntExist file="${relengCommonBuilderDir}/lib/ant4eclipse.jar" />

<get-ant4eclipse downloadDir="${downloadsDir}"

<au:assertFileExists file="${relengCommonBuilderDir}/lib/ant4eclipse.jar" />

<target name="testThirdPartyDownloadRejected">
<property name="file" value="ant4eclipse-0.5.0.rc1.zip"/>
<get-ant4eclipse downloadDir="${downloadsDir}" commonBuilderDir="${relengCommonBuilderDir}"/>
<au:assertFileDoesntExist file="${relengCommonBuilderDir}/lib/ant4eclipse.jar" />

<target name="run">
<fileset dir="${basedir}" includes="*getant4eclipse.test.xml" />
<au:plainlistener />

The output which when this runs looks like the following:

[au:antunit] Build File: /home/dcarver/eclipse3.5/athena/org.eclipse.dash.commonbuilder.releng/tests/tasks/getant4eclipse.test.xml
[au:antunit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 17.972 sec
[au:antunit] Target: testThirdPartyDownloadRejected took 0.082 sec
[au:antunit] Target: testGetAnt4EclipseZip took 17.161 sec

The nice thing about having this as an ANT script itself, is that the build engineer does not need to know Java in order to write a functional test. The other thing is that it helps provide some documentation on how various tasks and targets are used within a build. So that others that may need to maintain the build can find a simple reference point.

Having tests does require you to rethink in some ways the way your build scripts are constructed. They need to be modularized and they have to be able to be tested in isolation. It helps enforce general good design principals that are sometimes relaxed because “it’s just a build script”.

This entry was posted in build, eclipse, release engineering, testing, xml. Bookmark the permalink.

One Response to >Functional Testing Builds

  1. SteveL says:

    >Ant was written primary to test ant tasks and types, with every test case trying some options of the build. 1. you can set it push out XML compatible with the junit task.2. you can make assertions about output, asserts that only work in targets run by antunit, when the output is captured3. your testSomething targets can have dependencies like normal; these get executed too.4. you can use all the asserts other than those about output in any normal build file, just stick them in where you need extra checks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s