>Continuous Integration speeding up builds

>The slowest part I have found in a continuous integration build can be the part that is most crucial to the build itself. Code checkout. This check out process can add 10s of minutes to the overall process. This is time that can be crucial in getting feedback to the developers about the build. The longer the build time, the less likely that a developer is going to remember what happened if the build fails.

There are two types of checkouts that can be done, depending on the need of your builds.

  • Clean – delete all existing source, and recheckout everything.
  • Incremental – only delete/checkout what has changed since the last build.

Both methods are needed, and both have their place. Which one you choose for a build is an important factor in its speed.

Incremental Checkouts:

Ideally these are used through continuous integration builds. As soon as code is checked in, a build is kicked off. The build only checks out the code that has changed since the last build. For especially large plugin-projects this can save 10 to 20 minutes off the checkout time, allowing the actual build to start much sooner. This simulates the role of a developer synchronizing their workspace and integrating with others code. The sooner these builds start the sooner the developer can get feedback to address issues before they get to the other types of builds.

Clean Checkouts:

I recommend these for the cross project integrations, nightly, I Builds, milestones, and release builds. These builds benefit from the certainty that a clean build will provide. It is building the system from a clean slate, bring in all the existing dependencies that may have changed as well. These types of builds are typically longer running, and run during off development hours. So the delay in feedback is not as critical, as long as the developers are notified whether the build passes or failed within a reasonable time period.

So, if the checkout process is one of the longest steps in your build process, then check to see if you can switch to an incremental checkout. On one process, I managed to save 9 minutes off the overal checkout period by switching from a clean checkout to an incremental checkout.

This entry was posted in build, eclipse, release engineering. Bookmark the permalink.

One Response to >Continuous Integration speeding up builds

  1. >There is something mid way actually. Do an incremental checkout, svn export it to another location and then build. This additional thing is way faster than a full checkout and is as good as a checkout.SWTBot does something similar, by using the ant “sync” task to sync the workspace to a location where the build executes.See the “copy-sources” target at http://github.com/ketan/swtbot/blob/master/org.eclipse.swtbot.releng/build.xml for more.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s