The fault, dear Brutus, is not in our stars, But in ourselves…
Ed Merks gave a talk about Builds and Provisioning. How programmers hate and love them. While I agree with many aspects, I think some of the conclusions are wrong in where the problems with builds lay.
Ed is correct that programmers tend to shift the blame from themselves to that of the build when it is broken. After all if it works on their system it can’t possibly be broken. However, the reasons that Ed outline’s are only part of the problem (i.e. dependency on the IDE to produce the build which is in itself an Anti-Pattern of Continuous Integration.).
Many of us use IDEs, and most IDEs have some kind of build management process within them. However these files are always proprietary to the IDE and often fragile. Furthermore they need the IDE to work. It’s okay for IDE users set up their own project files and use them for individual development. However it’s essential to have a master build that is usable on a server and runnable from other scripts. So on a Java project we’re okay with having developers build in their IDE, but the master build uses Ant to ensure it can be run on the development server.Martin Fowler, Continuous Integration
In my experience working as both a programmer and a maintainer of build systems, 90% of the failures happen because of integration issues. Not necessarily integration dependencies from other projects, but integration issues within your own project. Many programmers do not follow good practices from Continuous Integration. Faults that happen:
- Code checked in with out running all the unit tests locally.
- Not synchronizing with the source code repository before checking code in.
- Not running integration tests with code that has been synchronized.
- Checking code in at the end of the day with out running tests.
- Not writing tests to make sure what we wrote is working.
These basic items will reveal problems that the build servers tend to catch, because the build server doesn’t forget to do these items. If following true continuous integration practices, then a broken build should be fixed in less than an hour. Reason is that the developer has gotten early feedback.
Overly Complicated Builds:
The problem in general with Eclipse based builds are legacy. They have made assumptions on how builds should be done that do not necessarily fit the more general practice. Regardless of the way the current eclipse ant tasks are implemented, there is no practical reason that you need to run Eclipse Headless to produce a build. I’ve done several builds outside of eclipse with very simple Ant scripts using the Ant4Eclipse project. This only depends on the eclipse OSGI runtime jar, and optionally the eclipse compiler. No need to launch eclipse headless to run ant scripts.
The key to making builds reproducible is to do The Simpliest Thing that Can Possibly Work. In many cases we tend to over architect the solutions. The recent work with P2 really has made the builds more complicated than they need to be. I’ve been working with some P2 provisioning for a Product build, and it really does add a lot of extra complications to the whole process. Instead of a simple feature build with an update site, now we have to add in steps for directors, provisioning of the repository, provisioning of the product, and other items. In my opinion P2 has unnecessarily complicated matters not made them easier.
Much of the excellent work that Nick Boldt has done with the Athena Common Builder is to work around the assumptions that are made by a PDE based build. Imagine if PDE would rework their builds what benefits and fewer work arounds that would have to be made.
Over complication is my same opinion of the eclipse MAP based builds. I have yet to run into a problem with builds that were built directly from HEAD. Allowing HUDSON or another build system do the tagging for a particular build is just as convient and workable solution as a map based build. The problem with MAPs is that they do not catch integration issues early, so the programmers blame the build for the failures instead of their own lack of discipline. They also do not match what a programmer is building from. A programmer is typically building from a copy of Head or a very close version of it. MAP based builds do NOT catch integration issues, they tend to hide them.
Ed is correct, unfortunately, in the Corporate world, if a team has a build engineer, it is usually the most junior programmer that has this responsibility. However, this is wrong, everybody should be responsible for the build. A good build can be reproduced by anybody with minimal effort. The corporate world tends to encourage knowledge silos, unfortunately on eclipse projects we can’t continue this practice. Everybody needs to wear multiple hats, including that of build engineer.
If your project consistently has bad builds, don’t necessarily blame the build engineer, or say it is because the builds are too complicated. Look in the mirror, it is probably your own fault that the build is broken.