>Continuous Integration

>

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. — Martin Fowler, 2006-05-1, “Continuous Integration”


Continuous Integration is still something I don’t think a lot of projects fully get. While I realize that there are many different ways to do builds, and to verify their validity, I still think the idea of at least synchronizing daily with the repository is one of the key features. Especially when working on a multi-member, multi-national team. The code is always evolving, and constantly changing, waiting even a few days before synchronizing your code can be very time consuming.

First let me say that I’m very grateful for David Williams for setting up our build environment for the XSL Tooling project. It has saved us a lot of time and allowed us to quickly get started writing code. With that said, I think we can improve the build experience with time.

In XSL Tooling unfortunately, we don’t do what I would call continuous integration. Meaning that we don’t run a build as soon as code has been checked into CVS, and the unit tests (if any) aren’t run with them. The system is not built until it is tagged for “release”, which means that somebody has to use the Release Engineering tools to tag the files, and then a build will start and the unit tests will run. Ideally, I’d like to see this eventually changed so that instead of having to tag everything, we can get a true continuous build going based off the code that is currently in HEAD.

We also currently I believe, check out all the code (even if it hasn’t changed) and rebuild everything. Again, I would eventually like to see us migrate to something that does Incremental checkouts, and only builds what has changed since the last build was requested. We don’t have an overly large number of plugins or source files, but currently our process takes about 20 minutes to do a build. The full WTP build takes between 3 to 4 hours depending on build machine load. This is going to get a bit worse as we go along with development and add some more unit tests. So we may eventually need to get to a multi-stage build process.

The benefits one can realize from a continuous integration process are numerous. Continuous integration allows for detecting integration issues early when they can be more easily identified. If somebody checked in code within the last hour and the build just broke, it is more than likely from one of the pieces that was just checked in. However, if changes are batched, tagged, and then released….it could be any one of a number of changes that were just checked in. Making it much more difficult to find the real culprit for why a build broke.

Unit tests are a critical part to this endevor. I still hear from time to time, that the unit tests are broken, should they be removed from the build? If the build is breaking because a unit test isn’t passing, it means something is wrong with the build. It is either a bad unit test that needs to be corrected, or there is something wrong with the code that was just checked in. Continuous integration can help catch some of these earlier, rather than later.

There is still the issue of timing. In order for a continuous integration build to be of benefit, it should be completed entirely within a reasonable time. Some people say 10 minutes is reasonable, I tend to go for the hour threshold. Anything longer than hour, the build probably needs to be broken into two stages.

So what is my ideal situation?

  • Continuous Integration or at least Hourly Integration from Head. These run a limited number of critical tests, to check the stability of the integration.
  • Nightly Builds from Head. This is a longer running build, and runs a larger test suite.
  • Integration Builds. This is what we currently have, in that it is a tagged/released build and the same nightly build tests are run against the integration build.
  • Milestone/Stable builds – An Integrated build based on a specific tagged version, that is going to be released.

By spreading this out a bit, the developers working on the code can get feedback quicker and are able to address any build breakages sooner. Oh and if you haven’t noticed, I do like to challenge the status quo, and to get people to try and think outside the box a bit.

Advertisements
This entry was posted in agile, eclipse. Bookmark the permalink.

3 Responses to >Continuous Integration

  1. >Good post David. I’m not a fan of removing failing tests from the build unless it is confirmed that the test itself is broken and therefore isn’t flagging a problem with the build.I’ll also second your comment about being grateful to David Williams. David W. has done a tremendous amount of work to create a really solid build system for WTP. I think this type of work is often overlooked but it is critical to the success of the project. Thanks David!

  2. >Interesting. Breaking unit tests on head is something that ideally should not happen.This usually means developers not running tests locally or environment changes across different machines.All is not lost in cases like the one which you point out, where it’s outrageously expensive to run a build at every checkin.This is what we do at ThoughtWorks:1.Set up CruiseControl to run just the unit tests on head, this should be quite quick and provides far more value than running a full build everytime.2.A successful unit test triggers a set of smoke tests — things that test a few important flows of the application, end-to-end. These should also be ideally less than 15 minutes.3.Run the full build either nightly, or have a separate CruiseControl to run the huge build, split it across multiple cruises, this is the difficult part.Do drop me a line if you’d like to know more.

  3. >I agree, David deserves an Eclipse medal of honor for the work he has done!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s