>
Many of us use IDEs, and most IDEs have some kind of build management process within them. However these files are always proprietary to the IDE and often fragile. Furthermore they need the IDE to work. It’s okay for IDE users set up their own project files and use them for individual development. However it’s essential to have a master build that is usable on a server and runnable from other scripts. So on a Java project we’re okay with having developers build in their IDE, but the master build uses Ant to ensure it can be run on the development server.
Martin Fowler, Continuous Integration.
Sometimes when we are only focused on our portion of the universe it is difficult to image that there are other ways to do things. Eclipse hosted projects are in some ways unique. For the vast majority, they are all developing plugins that run on top of the IDE or as RCP applications. Which means they are going to be using Eclipse to develop things. However, step outside of the Eclipse microcosm, and things change dramatically. An example are the projects hosted at the Apache Foundation, GitHub, Sourceforge.net, and other places. Here there is a wide variety of ways people are contributing to open source projects. They are using Netbeans, IntelliJ, Vi, EMacs, XCode, etc. Each of the IDEs have their own build systems and their own way of building. Each of the IDEs provide their own spin on the build process. However, there appears to be some in the community that believe the IDE is the ideal way to build. That you should have to deploy the same set of plugins or IDE onto your build server to build your application. This works fine if you have a captured audience and everybody is going to use the exact same IDE, with the exact same configuration and plugins, however it starts to fall apart once you get outside.
The Master build that is the defacto standard needs to be IDE independent. Especially if you have a community that uses multiple ides. The build system really can be whatever you choose, but it should not be specific to a particular IDE. It just rubs me the wrong way. It is what bothers me about the concepts of Buckminster and B3 in general. If you are isolated to eclipse and eclipse plugins then there may not be a problem. But as soon as you step into the OSGI Runtime things start to change. It can cause more complications when you have to deal with a larger more disperse community. There you have people using systems like Bundle, PAX, Eclipse, and Tycho, and numerous other methods to create their bundles. Eclipse is just another player in that space.
Ideally, a build should be as easy as saying:
maven clean package
ant deploy
buildr clean test
gant run
Anybody should be able to run the build at any time with out necessarily having to have a specific IDE and specific version. The build should ideally be it’s own self testing entity.
So I’m all for modelling builds and making them easier to implement. But I’m more for making those models agnostic of the IDE. I still think it is a good thing that programmers SHOULD know how their software is built and deployed. But we do need to make it easier for those builds to be accomplished.
intellectualcramps 
>From purely "clean architecture" standpoint, there is certainly a lot to be said about keeping your build IDE-agnostic. However, it is important to understand why people want to have IDE-aware builds. If you erect barriers between IDE and the build, you are introducing significant additional costs to your development. You must configure everything once in the build environment and then again in your IDE. After initial configuration, every change must be made in IDE metadata and separately in the build environment. A lot of extra overhead that is pretty fundamental to IDE-agnostic build approach. Add to that the cost of chasing down problems when something builds IDE and doesn't build in the build system (or the other way around).So it really comes down to doing what's right for your project or product team. If you have a lot of diversity of contributors who want to use different tools, the extra overhead of an IDE-agnostic build is a necessity. If your team is already using a single IDE, using a build system that integrates with it and cuts you releng costs is just common sense.As is typical in engineering, no solution is perfect for all situations and I am skeptical of anyone claiming otherwise. 😉
>@Konstantin. Correct. There is no one solution that will fit all.
>Dave, by saying "IDE", are you really referring to the UI portion of Eclipse or other IDEs? Then I would fully agree with you.But how would the headless/core parts of any IDE differ from any other build system like Ant or Maven?They are all configured with a number of text files that can be edited with any text editor and they can all be executed by an automated runner on a server.So, what makes Eclipse's headless build mechanisms (JDT core, Buckminster, b3, …) worse than Ant, Maven etc?
>Dave, I think you are wrong. Using Buckminster or B3 is not anything different than using Maven with Tycho or any other highly sophisticated build system. Under the covers, both use IDE specific components (such as the JDT compiler or p2 for resolving dependencies).Frankly, a really good build system integrates with my IDE in a way that not requires me to synchronize any meta-data between my IDE and build-system specific configuration files. That's just a waste of my spare developer time and causes headaches. IMHO it's absolutely crucial to have the build system working 100% equal to the IDE build. For example, there are issues with Ant based builds because char set configurations aren't correctly read by PDE Build. After all, I set them up correctly in my IDE. So why the hell do I have to configure this again for some build system?It's never a build system that's at the center of a project. The developers make a project. I really don't want my guys wasting any time duplicating data between the primary IDE and build systems.Of course, things might get complicated if you can't agree on a primary IDE. But I guess in the case of an Eclipse project that's not an issue. 😉
>Konstantin probably summarized the main points the best. You have overhead when dealing with an agnostic build that doesn't necessarily depend on specific tasks that are only in one particular IDE. If these particular tasks or features are easily distributable (i.e. small and isolated) then they just become libraries that go on top of the build script.However none of the current build systems at eclipse fall into this category, with the exception of the upcoming Tycho project. All are tightly integrated into eclipse core and the way eclipse works. It makes it very difficult when working on projects outside of eclipse to collaborate effectively with others. We start to get the "well it works in my IDE syndrom." Which is why the IDE needs to be taken out of the situation. This may not be as much of a concern to those that only do Eclipse plugin development, but as you expand beyond Eclipse's OSGI view port into a more diversified OSGI world, it becomes more critical to take the various idiosyncrasies of how IDEs build out of the equation.If you have a group of people using Netbeans, Eclipse, and IntelliJ…whose build is the correct build if it is solely based on how the IDE builds the system.BTW, P2 is not specific to the IDE the P2 repos, and provisioning those repos are IDE independent. You can use P2 with out having to use the Eclipse IDE. It's not pretty and integration can and should be better with the Ant tasks it provides.
>As others have pointed out, your use of the term "IDE" as a synonym for "anything that depends on Eclipse" seems misguided at best. Is making use of Eclipse's workspace—a concept supported independent of the Eclipse IDE and one that runs very well on a server—really a bad thing? Why create alternative when you already have something that works well? You say p2 isn't IDE, while you brand Buckminster and b3 as IDE. That seems to fly in the face of facts given that Buckminster runs very well on servers ever day.In the end, none of us wants to manage more than one build system. In fact, none of us wants to manage a build system period, but we have to have at least one to work in our IDE; that's the only one we really must have. So it seems to me the goal of agnosticism is really just another zealous religion, no different from all the other religions to which it claims to be superior, and one with a heavy price to pay.
>Perhaps someone should build an IDE on top of the Maven plug-in framework. That would put things in perspective 🙂
>Let me see how I can clarify the Buckminster stance.Buckminster goes into and uses the internals of the eclipse platform and specifically the internals of PDE where necessary to drive it's build. PDE itself requires the eclipse platform and all that comes with it.The only real dependency for P2 is OSGI, Equinox and the P2 related plugins. Make Buckminster, B3, work the same way, and I'll take it out of the IDE category.Compared with technologies like Ant, Buildr, and Maven which are IDE agnostic builds (i.e. they don't have a dependency on any particular IDE to function), the other build systems in eclipse don't meet this criteria currently. B3 has the potentional, but we'll see how it evolves.Not wanting to manage a build system is like not wanting to write any code. Builds are programming, we can make it simpler but ultimately somebody has to tell whatever build engine what to do.
>There's no argument that Buckminster is built on the Eclipse platform. And sure, PDE does carry some baggage. But lets look at what that consists of and compare:1. The JDT compiler (of course).2. The PDE Manifest checker.3. The PDE Extension point checker.4. The Eclipse internal build system.5. The Equinox framework.6. The p2 resolver7. The resource framework.8. Ant.This is about 75 bundles.Buckminsters PDE support adds:7. Launchers for JUnit plug-in tests8. Out of the box support to build, sign, and pack p2 sites.9. Headless ant integration.Now Maven.The first time you make an attempt to build a bundle with Tycho, 90 jars are downloaded (it's 559 files in total but most of them are small poms or checksums). That's some baggage! Still, there is:1. No PDE Manifest checker.2. No PDE Extension point checker.3. No Eclipse internal build system.4. No resource framework.5. No Ant.But OK, you are no longer dependent on the Eclipse platform. On the other hand, you are now 100% dependent on another platform that is just as complex and require files of its own in each project. So where did the gain come from?I get back to my previous comment. Let's assume that someone builds and IDE on top of the Maven plug-in framework. Would that render the Maven build system IDE dependent?
This is pretty old but I thought I’d share a recent experience. I’m was an Eclipse user for years and about three years ago I decided to switch to Netbeans. Eclipse had just started to feel…well I’m not here to troll – it just didn’t suit me as much any more. I was surprised to find that Netbeans actually runs Maven projects natively. There is no “two sets of config info” – it just works out of the box, no hassle. And since my builds are automated on Jenkins and pushed out to a repository, with automatic versioning, etc – I’ve been very happy. I recently was given the task of mavenizing a plugin/feature/site build so that we could get it built as part of our continuous integration proceses; I used tycho and with almost no configuration, everything just worked. It’s been a real treat!
>If I can take the same build scripts unmodified, and work in Netbeans, InteliJ, or command line, then you are no longer IDE agnostic. If I can't then you are soley dependent on that ide.Maven, Ant, Buildr, Gant, and numerous other build systems are designed so they don't depend on a particular ide. This is a good thing, not a bad thing. Especially when you are dealing with a project that has people using multipel IDEs.B3 has the potential to fall into this same category of being IDE agnostic, but only if we try to make it so.
>What's stopping you from taking a Buckminster build and run it from the IntelliJ or Netbeans?
>Buckminster requires certain artifacts from an eclipse IDE and workspace to work correctly. So it becomes IDE agnostic in this sense. Those artifacts do not necessarily exist in other IDEs.Sure you can check them into the repository, but virtually all projects at Apache do not include IDE specific metada checked into the repository.Technically I can run it through some mechanism in another IDE, but I do I really need a full platform download to run Buckminster or any build system?We have different view points, and there are a growing number of people that think the IDE is the ultimate source point for a build, because it works in their IDE. However, whose IDE is the main IDE? Especially on a large team. If it works on one persons IDE but not another…whose is right and whose is wrong?
>Let's visit your criteria for an IDE agnostic build again:"If I can take the same build scripts unmodified, and work in Netbeans, InteliJ, or command line, then you are no longer IDE agnostic."Check. You can do that with Buckminster.Maven introduces a lot of Maven specific things. That's obviously OK by you. What I can't understand is why the same thing isn't true when we use the Eclipse platform instead of the Maven platform. Just because an IDE is built on top of the Eclipse platform doesn't mean that the platform itself is IDE specific.
>Thomas IF you can get everybody on the team to agree that Buckminster should be the one true source for a build system then I have no problem with it being the build system.My experience though working with projects outside of eclipse has been that nobody can agree that a build that depends on specific items from a particular IDE is the defacto build. An extreme case that is just as bad is having two different build systems. I've seen some Apache projects in which both a maven 2 and buildr are used. Ideally they need to get to one build system to ease maintenance.Konstantin really summarized this up well in the first comment. Do what is right for the particular project you are working on.My preference will always be to have a build that doesn't depend on particular tasks that are specific to a particular IDE. I just work with too many other open source projects where people use different IDEs.
>Dave, I can understand that you, in an attempt to mediate, would choose something that isn't better suited for one IDE than another. But it should be clear in that case, that you do this for political reasons. Not because there's a technical benefit.I think we totally agree that it is crucial to have a main build system that works for all. The main difference in our views is that I don't see a problem if that happens to coincide with the build used by one of the IDE's. Everyone using that IDE wins. Nobody looses.
>Frankly, Dave's position makes no sense at all to me. Eike's comment at the top went right to the heart of this post, which I think is based on some kind of misunderstanding, so I don't see why the discussion has carried on for so long.Everything – Buckminster, Maven, whatever – is based on something. In the case of Buckminster that "something" includes a subset of Eclipse platform technologies, on top of which many things could be/are built, not just an IDE. P2 also depends on a subset of Eclipse platform technologies (maybe a smaller subset, but so what?), and is in fact such a technology itself.So what is Dave saying? That because Buckminster is built on something that an IDE could also be built on that's somehow bad, as in "guilt by association." And it would be better if it were built on something on which other things could be built but an IDE could not be? If so, the guilty association is with Eclipse, not with and "IDE". That's just Eclipse self-hate!It's ironic that one of the key objectives of Buckminster was to AVOID dependency on an IDE (i.e. headless server builds), while also avoiding unnecessary competition between the IDE and build system when you want to build from the IDE. So, why don't we find something else to pick on?