Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. — Martin Fowler, 2006-05-1, “Continuous Integration”
Continuous Integration is still something I don’t think a lot of projects fully get. While I realize that there are many different ways to do builds, and to verify their validity, I still think the idea of at least synchronizing daily with the repository is one of the key features. Especially when working on a multi-member, multi-national team. The code is always evolving, and constantly changing, waiting even a few days before synchronizing your code can be very time consuming.
First let me say that I’m very grateful for David Williams for setting up our build environment for the XSL Tooling project. It has saved us a lot of time and allowed us to quickly get started writing code. With that said, I think we can improve the build experience with time.
In XSL Tooling unfortunately, we don’t do what I would call continuous integration. Meaning that we don’t run a build as soon as code has been checked into CVS, and the unit tests (if any) aren’t run with them. The system is not built until it is tagged for “release”, which means that somebody has to use the Release Engineering tools to tag the files, and then a build will start and the unit tests will run. Ideally, I’d like to see this eventually changed so that instead of having to tag everything, we can get a true continuous build going based off the code that is currently in HEAD.
We also currently I believe, check out all the code (even if it hasn’t changed) and rebuild everything. Again, I would eventually like to see us migrate to something that does Incremental checkouts, and only builds what has changed since the last build was requested. We don’t have an overly large number of plugins or source files, but currently our process takes about 20 minutes to do a build. The full WTP build takes between 3 to 4 hours depending on build machine load. This is going to get a bit worse as we go along with development and add some more unit tests. So we may eventually need to get to a multi-stage build process.
The benefits one can realize from a continuous integration process are numerous. Continuous integration allows for detecting integration issues early when they can be more easily identified. If somebody checked in code within the last hour and the build just broke, it is more than likely from one of the pieces that was just checked in. However, if changes are batched, tagged, and then released….it could be any one of a number of changes that were just checked in. Making it much more difficult to find the real culprit for why a build broke.
Unit tests are a critical part to this endevor. I still hear from time to time, that the unit tests are broken, should they be removed from the build? If the build is breaking because a unit test isn’t passing, it means something is wrong with the build. It is either a bad unit test that needs to be corrected, or there is something wrong with the code that was just checked in. Continuous integration can help catch some of these earlier, rather than later.
There is still the issue of timing. In order for a continuous integration build to be of benefit, it should be completed entirely within a reasonable time. Some people say 10 minutes is reasonable, I tend to go for the hour threshold. Anything longer than hour, the build probably needs to be broken into two stages.
So what is my ideal situation?
- Continuous Integration or at least Hourly Integration from Head. These run a limited number of critical tests, to check the stability of the integration.
- Nightly Builds from Head. This is a longer running build, and runs a larger test suite.
- Integration Builds. This is what we currently have, in that it is a tagged/released build and the same nightly build tests are run against the integration build.
- Milestone/Stable builds – An Integrated build based on a specific tagged version, that is going to be released.
By spreading this out a bit, the developers working on the code can get feedback quicker and are able to address any build breakages sooner. Oh and if you haven’t noticed, I do like to challenge the status quo, and to get people to try and think outside the box a bit.