Push Button Builds.

Kim Moir had a good post on busting some build myths, and continuing the “MythBusters” analogy, I think we need to revisit a one of the myths. First let me say that I totally agree with Kim, that being in charge of a build is not something a Monkey should be doing. It takes just as much programming skill and knowledge to construct a build and maintain it as it does with any software system. If it was easy, we wouldn’t complain about it so much.

There is really only one item I think that should be revisited:

Nobody needs to understand how the build works, they just need to push a button.

While the busting of this myth is sound, I do agree that people should understand how to maintain the build. I believe strongly that all developers on the team should be able to maintain the build, add to it, and be able to do releases. Where I disagree is the implied implication is that Builds can never just be “push a button and it goes.” Depending on the build technology being used, this can be easy or hard, but it can be done. Eclipse builds should be as simple as cloning the project or releng project, and just running at the command line:

ant build.xml

or if you are using maven:

mvn clean install

That is all they should have to do to build the complete project from source. This includes doing a release, deploying artifacts, creating a Product, etc. Yes it should be that simple to run. We don’t want to make our applications hard to use, as people will go elsewhere. Why should running a build be any different? So the goal should be that any developer, any community member, or any body period, should be able to build the complete product with one command. Doing this will make everybody’s lives easier. A push button build may also eliminate the initial bottle neck to a contributor being willing to enhance and maintain a build as well.

About these ads
This entry was posted in ant, craftsmanship, eclipse, maven, release engineering. Bookmark the permalink.

11 Responses to Push Button Builds.

  1. Kim Moir says:

    Dave I think you misunderstood my comment or perhaps I didn’t articulate myself clearly. I agree that push button builds are wonderful. The problem occurs when you push the button and the build explodes and you have no idea how to fix it. Repositories get stale or are moved, or dependencies change or perhaps source code repositories get migrated. The point I was trying to make was that there needs to be some understanding on the team of how the build works in the event that pushing the button doesn’t make the build magic happen :-)

  2. kingargyle says:

    Yep, that is where we agree. I think more accurately the problem is the fact that we let knowledge silos to occur on a team and people have specific roles. The way to address this to not have specific defined roles. In an open source project, this is a necessity. The problem comes from the the legacy corporate culture where people have said, you are a qa tester, that is all you do. I’m running into this problem dealing with corporate developers moving to an open source world.

  3. Pingback: Push Button Builds. | Eclipse | Syngu

  4. Ian bull says:

    While I would love a 1 line build, we often trivialize the hard parts. Let’s look at the Maven build in your example:

    1. Where do we get maven from, and what version?
    2. What version of maven do I grab to build Helios? (For a maintenance build, so I want to reuse what we used last year).
    3. Where did do I get the sources from?
    4. What platform does that build run on?
    5. How are the MacOS/Windows/Linux specific tests run and what do we have to do to setup those machines as slaves?
    6. What version of Eclipse are we using (I assume we are building using the Eclipse Compiler)
    7. What version of Java are we using to run the Eclipse Compiler (remember, we are re-building Helios).
    8. Is there a specific directory structure I need on disk? (java, eclipse, ant, etc….)

    Most builds are 1 cmd, but we gloss over the 8 other cmds you need to run before you run build.sh.

    • kingargyle says:

      Ian, Most of what you specify should be in a README file for that particular project being built. The README SHOULD specify what Version of Maven is required, any system pre-reqs. What should not be required by the end user is any Environment Variables, drive mappings, special socket settings, etc. I’ve delt with trying to run the Web Tools project’s ant/base builder, and there is a LOT of pre-reqs setup. I should be able to get up and running in 5 minutes or less.

      Item’s like ecipse dependencies (i.e.versions), java versions, directory structure, etc, should be taken care of by the build itself. If a specific version of Java is required, then that should be specified in a README. That README should be short and to the point.

      An example, is the README from the turmeric-soa project.

      https://github.com/ebayopensource/turmeric-runtime/blob/master/README.creole

      Now we can probably improve this, but after a person downloads Maven 3.0, and makes sure they have Java 1.6 installed. Then they should be able to follow the instructions there, and get going in about 5 minutes.

      Anything that is specific for specific OS environments should be noted, but the build should handle the necessary setup itself. So yes, it should be a push button build. The end user should have very little to do to get everything setup.

  5. Ian bull says:

    Dave, Re: a README, I completely agree. My point was that most people who tell me that they can build in one command often gloss over the setup steps. I strongly believe that any developer should be able to build on their machine. If this can be done in 3 commands (git clone releng-project, setup.sh, ant build.xml, GREAT!), but if it takes a few more commands, that’s ok too.

    However, I feel that completely reproducible builds trumps all. That is, we should be able to build any version of the software using the original setup (the setup that was used when the software was first built) — thus producing the exact same bytes. I don’t care how easy a build is to run, if it isn’t reproducible, then it’s useless.

    • kingargyle says:

      Which kinda brings us to a potential problems with p2 based repositories. You can’t guarantee that the build will be reproducible from any snapshot, because you can’t guarantee that the p2 repo that it was built from is going to be there, isn’t a composite repo, and isn’t missing artifacts or hasn’t been changed. It is one of the reasons that Maven Central was created, to help make sure that a builds dependencies are there.

      My understanding of p2 was that it was first and foremost designed with provisioning an application. i.e. installing new plugins and updating existing ones. It has since been used to help provision a build, but there is still this gap in guaranteeing that a build is reproducible if it is relying on a p2 repo. We need something like Maven Central, but for p2.

  6. I argue against README. README’s get out of date, aren’t read or can be plain wrong.
    What’s the point to have a README if you can have a script?
    For a good example, look at my ePen project. It has a setup.sh (http://epen.hg.sourceforge.net/hgweb/epen/ePen/file/b3eea57dbc8c/setup.sh) which lists dependencies at the head (ok, I was lazy: I could check those two dependencies and print nice error messages) but after that, the build is automatic:
    No maven? Download it, unpack it, use it.
    No source? Check it out.
    No compiled code? Compile the source.
    That’s what “push button build” is all about: Mac and Linux both support bash, so write bash scripts to do the initial setup.
    What about Windows? Well, there are two kinds of Windows users. The former kind “builds” applications from .msi files. The rest can read a bash script and follow it (or install cygwin/mingw).
    The main advantages of a script: It can be tested (run it every night on the CI server). Even if it fails, it contains exact instructions what it wanted to do so you have a starting point to fix the problem.

  7. Pingback: Push Button Builds « Dark Views

  8. Ian Bull says:

    Aaron,

    This is what I’ve done recently for a project I’m working on, but writing good scripts is not easy. For example, if I build last years code (for a hot fix to a customer), I don’t want it to download this years Maven. I need it to fetch the source from a branch (which needs to be a parameter). If you are using Eclipse as your build tool what architecture do you download — we can’t just download the latest. Also, which version of Eclipse do you run your tests on?

    I’ve found a solution that works well using Git and Submodules. I put all my build tools in Git, and you can fetch them from there. I use different submodules for different architectures so my repository doesn’t grow too big. Then the setup is simply git clone && git checkout && build.sh.

  9. digulla says:

    While writing good scripts isn’t easy, it’s easier to improve a bad script than a bad README. For one thing, a script can be tested.

    As for downloading the correct version: This is a big advantage of scripts! With a script, you can’t download just any version, you have to be specific. Most download sites offer a “nightly” download but the link changes every night – not good for a script. So you won’t download Eclipse-N201109021845 but eclipse-jee-indigo because that link is good for a couple of years.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s