>How to Build XSL Tools with Ant4Eclipse – Part III

>Testing:

In part two of this series, I showed how I assembled a working target for eclipse. This allows for some manual testing, but is also needed for automated testing. The eclipse platform provides a testing framework plugin that will allow running of your Junit Plugin-in tests. This requires launching eclipse in headless mode, but DOES NOT require running ant through eclipse.
The reason for this is that there are no special ant tasks that are eclipse specific to be run. The documentation that is currently available for the eclipse testing framework assumes you are launching ant through eclipse, which is not needed. The reason for the assumption is that there are a few property variables that are setup that the library.xml file provided by eclipse requires. These can be setup through a normal ant build, and do not require eclipse to set these up.

I did make a slight change to the java-test target, to include the “-dev bin” argument when launching eclipse. The reason for this, is that the unit tests have to be unjarred. You can not jar your unit tests as they will not run properly. The dev option puts the eclipse environment into development mode the same way PDE does when it launches eclipse from within eclipse for testing. The bin entry tells eclipse to look in the bin folder of all the plugins that have it for it’s class files that it needs. With out this, the runner for the unit tests can not find the tests. I spent several hours trying to figure this out and the above -dev option worked.


<target name="java-test">
<!--default vm args-->
<property name="vmargs" value=" -Xms40m -Xmx256m"/>

<!--set default jvm to use for testing-->
<property name="jvm" value="${java.home}/bin/java" />

<echo message="Running ${classname}. Result file: ${junit-report-output}/${classname}.xml."/>

<java fork="true" dir="." timeout="${timeout}" jvm="${jvm}" logError="true"
classname="org.eclipse.core.launcher.Main" output="${junit-report-output}/${classname}.txt">
<classpath>
<fileset dir="${eclipse-home}/plugins">
<include name="org.eclipse.equinox.launcher_*.jar"/>
</fileset>
</classpath>
<arg line="-application ${application}"/>
<arg line="-data ${data-dir}"/>
<arg line="formatter=${formatter},${test-output}"/>
<arg line="-testPluginName ${plugin-name}"/>
<arg line="-className ${classname}"/>
<arg line="-os ${os}"/>
<arg line="-ws ${ws}"/>
<arg line="-arch ${arch}"/>
<arg line="-consolelog"/>
<arg line="-dev bin"/>
<jvmarg line="${vmargs} ${extraVMargs}"/>
<sysproperty key="PLUGIN_PATH" value="${plugin-path}"/>
</java>
<antcall target="collect-results" />
</target>

Testing the Build.

I’ve tried to use an iterative development process when designing this build, and have tried to make various pieces so that they can be tested and run independently of each of there. For this purpose I have put the testing of the application into it’s own target.


<target name="test.plugins">
<copy todir="${pluginDestination}/target/plugins">
<fileset dir="${pluginDestination}/plugins">
<include name="**/*test*" />
</fileset>
</copy>

<ant target="ui-test" antfile="${library-file}">
<property name="os" value="linux" />
<property name="ws" value="gtk" />
<property name="arch" value="x86" />
<property name="eclipse-home" value="${pluginDestination}/target" />
<property name="data-dir" value="${pluginDestination}/data-folder" />
<property name="plugin-name" value="org.eclipse.wst.xsl.tests" />
<property name="classname" value="org.eclipse.wst.xsl.tests.AllTestsSuite" />
<property name="vmargs" value="-Dbaz=true" />
</ant>
</target>

The first step is to copy the unit tests into the target instance so that they can be run. Remember these have to be Plugin Tests, and they have to be unjarred.

The script above uses the library.xml script from the org.eclipse.test plugin. This contains the ui-test target which will launch the unit tests on the UI thread. You can also use the core-test target to run the tests with out a UI. As mentioned earlier there are several parameters that are passed to eclipse when it is launched to run the unit tests. These in particular are:

  • os – the name of the operating system, in my case linux
  • ws – the windowing system being used, gtk.
  • arch – the architecture of the cpu, x86
  • eclipse-home – where the target eclipse instance resides.

In addition there are some properties that have to be set regardless of if you are running ant through eclipse or using ant to launch eclipse for testing.

  • data-dir – the workspace to use when running the unit tests, this is not where you have your source files.
  • plugin-name – the id of the plugin that houses your unit test suite or test class.
  • classname – the fully resolved name of the the test suite or test class to run.
  • vmargs – any additional VM args that are to be passed.

There are additional items that can be passed and overriden, but this is the minimum that needs to be overriden and specified. Once the test completes, it will create a XML file in the eclipse-home location under a directory named results. This contains the results of the unit tests.

Note: This XML file appears to be different than what the standard junitreport ant task expects. This file can not be processed by this task. You must use a custom XSL file to process it.

Determining if the Tests Passed or Failed:

One issue I ran into is that by launching the tests this way, there is no direct report from the eclipse headless system back to Ant that the unit tests either passed or failed. All you have is the standard java return code. In order to help determine whether the tests passed or failed, I created the following XSL 1.0 stylesheet.


<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="text"/>
<xsl:template match="/">
<xsl:apply-templates select="testsuites/testsuite"/>
</xsl:template>
<xsl:template match="testsuite">
<xsl:choose>
<xsl:when test="@errors > 0 or @failures > 0">
<xsl:text>test.status=fail</xsl:text>
</xsl:when>
<xsl:otherwise>
<xsl:text>test.status=pass</xsl:text>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
</xsl:stylesheet>

The above stylesheet will process the XML file produced from the unit test run, and output a text file in a standard java properties file format, that indicates whether the tests pass or fail. This is then used by a custom target in the build.xml file to fail the test if it did not pass.


<target name="create.junitreport">
<xslt in="${pluginDestination}/target/results/org.eclipse.wst.xsl.tests.AllTestsSuite.xml" out="./report/html/results.html" style="junit.xsl"/>
<xslt in="${pluginDestination}/target/results/org.eclipse.wst.xsl.tests.AllTestsSuite.xml" out="./passFail.txt" style="passFail.xsl" />
<property file="passFail.txt" />
<echo message="${test.status}" />
<fail message="Unit Tests failed.">
<condition>
<equals arg1="fail" arg2="${test.status}" />
</condition>
</fail>
</target>

The unit test report is generated using the JUNIT.XSL stylesheet that comes with the eclipse testing plugin. The passFail.xsl is shown above, and generates a passFail.txt file. This file is used to create a test.status property. If the property contains the word fails, then the build is failed.

Wrapping this all up, putting a bow on it:

The main reason for this whole process was to create a build for XSL Tools that I could test locally and reproduce on multiple systems. There also was a couple of other side benefits.

  1. Hopefully it showed that one can indeed build working eclipse plugins, assemble, and test those plugins without having to run eclipse headless for the Ant tasks. Eclipse still needs to be run headless for testing…but it does not need the extra layer of overhead that is currently introduced through the platform build, pde build, and yes even the common builder.
  2. Continuous Integration can happen with eclipse builds. The above build with all unit tests takes about 2 to 3 minutes to run, not including the time to check out the code. I’m running my builds over cable Interent connection, so the check out process takes several minutes to complete.
  3. Building from Head is possible. Allowing for true continuous integration with other developers code.
  4. It’s important that a build is not tied to any particular platform. The same build should be able to be checked out, and launched from the server or a workspace, and produce the same results.

With that said, there are still plenty of reasons to use the build tools that have been developed by various eclipse projects. The build process described in these three entries does not do features, update sites, or products. The first two are planned to be addressed in future postings. The latter will depend on support being added to Ant4Eclipse. However, there may be alternative ways to address it.

Advertisements
This entry was posted in agile, eclipse, release engineering. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s