>Eclipse Build Process – Packaging

>I wrote a while ago about how eclipse shouldn’t need eclipse to build eclipse, and said I’d report back findings here. Well, I took the easy piece first, which is packaging. Packaging in this case is just combining various plugins into a customized version of eclipse. To help with this, I’ve used a product for a while called Custom Eclipse Builder. It’s an ANT build script and a set of property files that will automate the packaging of a customized version of eclipse. You may control what plugins get incorporated and download them from multiple sources. The key is that they be available as an archive version.

The features I need in a custom eclipse installation are:

  1. Eclipse Platform.
  2. Eclipse CVS Plugin
  3. Subversion support (I happen to use Subclipse)
  4. XML Modeling support via Hypermodel by Dave Carlson
  5. Mylyn
  6. Mylyn Mantins Connector
  7. Topcased UML
  8. Web Standard Tools
  9. Secure FTP via JCraft SFTP plugin.
  10. Data Tools Platform.
  11. EMF, UML, and other required modeling libraries and plugins.
  12. X-Assist XSLT (now part of the XSL Tooling Incubator project)

I need versions to run on Windows, Linux, and Mac.

Doing this all manually can be a time consuming process. So I’d rather have it automated. A complete build that includes all three platforms takes between 10 and 20 minutes depending on download connection speed, and how many files are stored locally as compared to fetched from the Internet.

I setup the builds to be triggered by our build system, Luntbuild. I happen to like Luntbuild over Cruise Control because it helps separate the build process from the check out process. I can write one build file that works on multiple workspaces or locations with out worrying about how the files are going to be checked out. It also sends out notifications through multiple channels.

As for doing plugin builds, I’ll be experimenting over the next week with the Ant4Eclipse project’s builds to build individual plugins. The keys to a good build are that it meets the following requirements:

  1. It has to be able to run on any machine. In my case I need the same build to run on Linux as well as Windows. This requires some thought ahead of time, but it makes it much easier when working in a multi-developer/multi-platform environment.
  2. Be repeatable.
  3. Be easily maintainable by others.
  4. Easy to setup and execute.
  5. Support automate reporting (this can be done in the build or through the build system that manages the builds).

Most eclipse builds I’ve seen have been setup to be very platform specific (i.e. a build is usually targeted to run on a particular platform like Linux or Windows, but is difficult to get running on both). They also fail the above requirements. A build should be able to be executed out of the box regardless of what system it was originally written to run on. If a user has all necessary source and supporting libraries, it should be an execute and go configuration.

Packaging is the easy part, the more complicated part is to be tackled next. Building plugins with out a headless eclipse environment. That is where I hope Ant4Eclipse comes to the rescue. Results from that experiment will be another topic.

Note: There are other products out there like EasyEclipse and PluginBuilder that can help with some of this.

This entry was posted in agile, eclipse. Bookmark the permalink.

8 Responses to >Eclipse Build Process – Packaging

  1. >Just curious, why is it OK to use tools like Ant but not OK to use an headless Eclipse to build Eclipse?In Buckminster we took the approach of providing the latter. A fairly small set of plug-ins (about 6MB in total) is needed in order to build the Eclipse features, plug-ins, update sites, etc. unattended in headless mode. Since a lot of the pde-build environment is reused. we minimize the need of redundant declarations.IMO, the build environment should be able to reuse as much as possible of the declarations that the developer uses in his day to day work in the IDE and using the Eclipse build system is an excellent way of building Eclipse.

  2. David Carver says:

    >Thomas it comes down to a preference on how a build should be done. How was eclipse built before there was the ability to do headless builds with eclipse? A headless eclipse uses ANT anyways for eclipse. It also has to do with system resources, if you have a build machine that is doing multiple builds at the same time, launching eclipse headless just to build some plugins still takes up system resources.If you didn’t have eclipse available (think a total disaster situation) and you needed to rebuild how would it be done?ANT4Eclipse has the ability to read the eclipse configuration files, with out the need to have eclipse running. So with Ant 4 Eclipse you can still reuse all of the existing eclipse definitions in the plugin.xml, build.properties, etc that plugin developers use within eclipse.I’ve always just found it odd that eclipse needs eclipse to be running headless in order to build itself. Especially when most of the ant tasks if not all can be refactored so that they aren’t reliant on eclipse api to be used.Are there other development environments or applications that require a version of itself to build itself?

  3. >My personal opinion is that if a build environment is built and maintained by itself, that’s a sign of good health and normal evolution.I have a hard time seeing the disaster that would strike so selectively that it would make it possible to get to the source of everything and to some binaries of other tools (like ant in this case) but not to a binary of an existing release of the tool you want.That said, I did look into ant4eclipse a while back. What I could not figure out was how they maintain the really hard stuff that pde-build is doing such as defining a compile that resolves the classpath based on package imports/exports, that warns/bails out based on compiler preferences like “discouraged access”, or resolves classpath containers dynamically.I’m sure it can be done. I just don’t see the benefit of copying all the code that already does it and then maintain it. How will that make you less vulnerable?

  4. David Carver says:

    >Thomas, Ant4Eclipse contains the org.eclipse.osgi_3.2.0.v20060601.jar. They also recommend using the Eclipse Java Compiler to handle a lot of the error checking and references that are specific to plugin checking and conformance. I suspect that osgi state information is maintained and managed by the osgi api available from eclipse.Again, it comes down to personal preference on how you want builds to be done. I just don’t think that a system should require a running working copy of itself to build itself or any of it’s dependencies. The libraries should be available to build the plugins and jars needed, but actually running in eclipse headless just rubs me the wrong way from a build/design standpoint.

  5. >I’m missing the point here obviously. If ant4eclipse do use the Eclipse OSGi runtime and it’s recommended that the Eclipse Java compiler is used, how is that not using Eclipse to build Eclipse?Same question but in different words; how is using the Eclipse OSGi runtime to maintain state information different from running Eclipse headless?I suspect we’re talking past each other somehow.

  6. David Carver says:

    >Thomas the difference is in the way the build gets executed. In an eclipse headless build, you still launch eclipse to launch ant, so you can have access to all the built in Ant Tasks that eclipse has:D:\build\org.eclipse.releng.eclipsebuilder>java -cp %ECLIPSE_HOME%\startup.jar org.eclipse.core.launcher.Main -application org.eclipse.ant.core.antRunner -buildfile build.xml -Dcomponent=sdk.examples -Dconfigs=”*,*,*” -Dbaseos=win32 -Dbasews=win32 -Dbasearch=x86 -Djavacfailonerror=true -Dpde.build.scripts=%ECLIPSE_HOME%/plugins/org.eclipse.pde.build_3.0.1/scripts -DbaseLocation=%ECLIPSE_HOME%In a build that doesn’t require eclipse, the osgi jar file and the eclipse java compiler are run outside of eclipse. The osgi jar is used as a library by the Ant4Eclipse tasks. It doesn’t launch a stripped down version of the eclipse platform, to execute the build. Basically you get down to this statementD:\ant\ant.bat build.xml -Dcomponent=sdk.examples -Dconfigs=”*,*,*” -Dbaseos=win32 -Dbasews=win32 -Dbasearch=x86 -Djavacfailonerror=true -Dpde.build.scripts=%ECLIPSE_HOME%/plugins/org.eclipse.pde.build_3.0.1/scripts -DbaseLocation=%ECLIPSE_HOME%To do your build.

  7. David Carver says:

    >Ideally, you would have in your build, the ability to build the osgi jar file, before hand if needed, but if not, it’s stored as a binary. So technically you don’t have a fully functional eclipse system executing your build, you are using some of the library files though.

  8. >So it seems that both cases have similar prerequisites? They both need a selection of things from an Eclipse distribution (you refer to pde build scripts for instance) in order to perform the job. The built-in tasks cannot however not be used in the ant4eclipse case.A tool like Buckminster (a really stripped down version of the Eclipse runtime) makes the invocation really simple. For example:buckminster perform <name>#bundle.jarbuilds the jar file for the bundle with symbolic name <name>.No defines necessary. The configuration is defined using a template workspace that is initially created using an IDE. That template is then reused in unattended builds and can also be used whenever a developer starts off with a new workspace.So far, I haven’t seen any alarming resource consumption taking place as a result of this approach.The really nice thing about it is that the build is executed exactly as it will be when you run it from within the IDE, built-in tasks and all.In any case, I think similarities between what ant4eclipse does and what Buckminster does by far outweighs the differences. It would be very interesting to talk to the ant4eclipse people. Perhaps we could join forces somehow and create something that is twice as good as any of us have today?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s