>Testing Launch Configurations

>XSL Tools allows for debugging and launching of XSLT Stylesheets through the existing eclipse Run/Debug framework. This same framework is used by other eclipse based plugins. For those those dealing with Java, Ruby, PHP, C/C++, etc that need to test and debug XSLT Stylesheets, having a familiar framework one can use is a great benefit. This is one of the short comings I have with the Oxygen XML eclipse plugin, is that it doesn’t leverage the common look and feel when it comes to launching and debugging stylesheets.

I recently had to debug some concurrency issues our tests were experiencing with the launching framework. Every once in a while the test would randomly fail. There were several things our tests were not doing correctly.

  • After refreshing a projects contents, not waiting until isSynchronized() returned true before proceeding on.
  • Not waiting and checking for isTerminated() to return true after exectuing a launch configuration. We were waiting a certain amount of time before proceeding, however launches can vary depending on system load so this was not an accurate way to handle this.
  • Not keeping our test data isolated to a particular test. In particular in this test it was not following good principles in setUp() and tearDown(). Keep everything isolated, start and end with a clean environement.

So I applied a bit of refactoring, and managed to create a cleaner more useable testing framework. Some of the ideas I leveraged from the existing debug tests from the platform. For the XSL Tools launching tests, most of the common code has been moved to an AbstractLaunchingTest class.

AbstractLaunchingTest:


public abstract class AbstractLaunchingTest extends TestCase {
private static final String XSL_TEST_PROJECT = "XSLTestProject";
protected static final String XSL_LAUNCH_SHORTCUT_ID = "org.eclipse.wst.xsl.debug.ui.launchshortcut";
protected static final String LAUNCHCONFIGS = "launchConfigs";
protected TestEnvironment env;
protected IProject testProject;
protected IFolder folder;

public AbstractLaunchingTest() {
super();
}

public AbstractLaunchingTest(String name) {
super(name);
}

@Override
protected void setUp() throws Exception {
super.setUp();
createProject();
createEmptyLaunchConfigsFolder();
deleteExistingLaunchConfigs();
}

private void createProject() throws CoreException {
env = new TestEnvironment();
testProject = env.createProject(XSL_TEST_PROJECT);
}

private void createEmptyLaunchConfigsFolder() throws CoreException {
IPath path = testProject.getFullPath();
folder = testProject.getFolder(LAUNCHCONFIGS);
if (folder.exists()) {
folder.delete(true, null);
}
folder.create(true, true, null);
}

private void deleteExistingLaunchConfigs() throws CoreException {
ILaunchConfiguration[] configs = getLaunchManager()
.getLaunchConfigurations();
for (int i = 0; i < configs.length; i++) {
configs[i].delete();
}
}

@Override
protected void tearDown() throws Exception {
super.tearDown();
testProject.delete(true, new NullProgressMonitor());
}

protected void copyConfigurationToWorkspace(IPath folder, String filename)
throws Exception {
URL url = Activator.getDefault().getBundle().getEntry(
"testFiles" + File.separator + filename);
String workspacePath = getWorkspacePath();

File target = new File(workspacePath + folder.toPortableString()
+ File.separator + filename);
copyFile(url, target);
}

private String getWorkspacePath() {
IWorkspace workspace = ResourcesPlugin.getWorkspace();
IWorkspaceRoot root = workspace.getRoot();
IPath rootPath = root.getLocation();
String workspacePath = rootPath.toPortableString();
return workspacePath;
}

private void copyFile(URL src, File target) throws Exception {
BufferedOutputStream bos = new BufferedOutputStream(
new FileOutputStream(target));
BufferedInputStream bis = new BufferedInputStream(src.openStream());
try {
while (bis.available() > 0) {
int size = bis.available();
if (size > 1024)
size = 1024;
byte[] b = new byte[size];
bis.read(b, 0, b.length);
bos.write(b);
}
} finally {
if (bis != null) {
bis.close();
}
if (bos != null) {
bos.close();
}
}
}

protected LaunchShortcutExtension getLaunchShortcutExtension(String id) {
List exts = getLaunchConfigurationManager().getLaunchShortcuts();
LaunchShortcutExtension ext = null;
for (int i = 0; i < exts.size(); i++) {
ext = (LaunchShortcutExtension) exts.get(i);
if (ext.getId().equals(id)) {
return ext;
}
}
return null;
}

protected LaunchConfigurationManager getLaunchConfigurationManager() {
return DebugUIPlugin.getDefault().getLaunchConfigurationManager();
}

private ILaunchManager getLaunchManager() {
return DebugPlugin.getDefault().getLaunchManager();
}

protected ILaunch launch(String name) throws Exception {
ILaunchConfiguration configuration = getLaunchConfiguration(name);
ILaunch launch = configuration.launch(ILaunchManager.RUN_MODE,
new NullProgressMonitor());
return launch;
}

protected ILaunchConfiguration getLaunchConfiguration(String mainTypeName) throws Exception {
ILaunchManager mgr = DebugPlugin.getDefault().getLaunchManager();
IFile file = testProject.getProject().getFolder("launchConfigs")
.getFile(mainTypeName + ".launch");
ILaunchConfiguration mine = mgr.getLaunchConfiguration(file);
assertEquals("Wrong type found: ",
XSLLaunchConfigurationConstants.ID_LAUNCH_CONFIG_TYPE, mine
.getType().getIdentifier());
return mine;
}

protected void refreshProject() throws Exception {
testProject.refreshLocal(IResource.DEPTH_INFINITE,
new NullProgressMonitor());
while (testProject.isSynchronized(IResource.DEPTH_INFINITE) == false) {
Thread.sleep(1000);
}
}

protected void launchConfiguration(String launchConfigName) throws Exception {
ILaunch launch = launch(launchConfigName);
// Wait until the launch configuration terminates.
while (launch.isTerminated() == false) {
Thread.sleep(1000);
}
refreshProject();
}

protected String readFile(InputStream input) {
String str;
String finalString = "";
try {
BufferedReader in = new BufferedReader(new InputStreamReader(input));
while ((str = in.readLine()) != null) {
finalString = finalString + str + "\n";
}
in.close();
} catch (IOException e) {
}
return finalString;
}

protected Document parseXml(InputStream contents) throws Exception {
DocumentBuilderFactory builderFactory = DocumentBuilderFactory
.newInstance();
DocumentBuilder builder = builderFactory.newDocumentBuilder();
return builder.parse(contents);
}
}

The AbstractLaunchingTest class contains all of the commonly used methods that are currently known to be needed for the XSL Tools tests. This class is extened by the LaunchShortCutsTest and the XSLLaunchingTests classes. I’ll review what the XSLLaunchingTest class does next.

Testing XSL Launching:

The approach I have taken with launching is similar to how the platform has done it. A launch configuration is created using the appropriate Launch UI, and then the launch configuration is exported to a launch XML file. Now from a pure XML design point, this is not that nice of a file, but I’ll reserve that critique for another time. These launch files are stored along with the other files necessary for testing within the org.eclipse.wst.xsl.launching.tests plugin/bundle. The setup and teardown methods load and unload these into the workspace before and after each of the tests. This makes sure the workspace is clean for each test that needs to be run. Part of the issue I see with existing tests is that the setup and tear down is not done appropriately. Results of prior tests can mix in with other tests, and this can have unintended consequences.

Setting up launch configurations externally has advantages. It makes it very simple to add a new test, and you can easily control the various types of tests and processors to be used. In this case, all of the tests leverage the internal Xalan processor for the transformations.

XSLLaunchingTests:


public class XSLLaunchingTests extends AbstractLaunchingTest {

private static final String TRANSFORM_COMMENTS = "TransformComments";
private static final String SIMPLE_TRANSFORM = "SimpleTransform";

@Override
protected void setUp() throws Exception {
super.setUp();

IPath path = folder.getFullPath();
copyConfigurationToWorkspace(path, "SimpleTransform.launch");
copyConfigurationToWorkspace(path, "TransformComments.launch");
refreshProject();
}

@Override
protected void tearDown() throws Exception {
env.dispose();
super.tearDown();
}

public synchronized void testSimpleTransformation() throws Exception {
IPath folder = testProject.getFullPath();
env.addFileFromResource(folder, "1-input.xml", "1-input.xml");
env.addFileFromResource(folder, "1-transform.xsl", "1-transform.xsl");
refreshProject();

launchConfiguration(SIMPLE_TRANSFORM);
IFile output = testProject.getFile("1-input.out.xml");
Document doc = parseXml(output.getContents(true));
assertEquals("root-out", doc.getDocumentElement().getNodeName());
}

public synchronized void testTransformComments() throws Exception {
IPath folder = testProject.getFullPath();
env.addFileFromResource(folder, "testCommentInput.xml",
"testCommentInput.xml");
env.addFileFromResource(folder, "testComments.xsl", "testComments.xsl");
env.addFileFromResource(folder, "expected.xml",
"testCommentsExpected.xml");
refreshProject();

launchConfiguration(TRANSFORM_COMMENTS);
IFile output = testProject.getFile("testCommentInput.out.xml");
IFile expected = testProject.getFile("expected.xml");

String result = readFile(output.getContents());
String wanted = readFile(expected.getContents());

assertEquals("Unexpected results:", wanted, result);
}
}

These tests now consistently run successfully. In the past the testTransformComments() was periodically failing. Always complaining that it couldn’t find the output file that it expected as the test ran longer than expected. The above is no way perfect but for now does what needs to be done.

Comments on Testing Launch Configurations:

The Platform needs to separate out some of it’s tests better. Currently JDT is testing internal classes of the DebugUI and Debug plugins. Many of the internal classes are needed outside of the platform framework by other plugins to test the existance of shortcuts and that the definitions are defined as expected. I was unable to find a way to access and test this information for XSL Tools outside of using internal classes.

The JDT testing of various launch configuration is very robust, and anybody that needs some tips or clues how to test launch configurations, should mine those tests. However, I would really call what JDT has Functional Tests instead of unit tests, as it’s testing several classes or a combination of classes to return a result. However, this is the way most of eclipse currently has to be tested, as it was not really designed from a Test Driven Development standpoint.

Make sure that when testing launch configurations you check the isTermiated() value, this will let you know when the launch has been completed. It could terminate for a variety of reasons, but it has to be checked otherwise it’s difficult to know exactly when to proceed with the rest of the checks or steps for testing.

Hopefully, the above can give others some ideas on how they can setup tests for their own launch configuration support. XSL Tools still needs tests to cover the various debugging options, and several other launch configurations. However, this refactoring should make it much simpler to add new tests going forward.

I’m always looking for ways to improve my existing tests, so if there is a way to improve this code please let me know.

Advertisements
This entry was posted in agile, eclipse, testing, xml. Bookmark the permalink.

One Response to >Testing Launch Configurations

  1. >Very good observations, I ran into similar irregularities when writing tests for issue 254677.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s