Are Projects Outpacing the ability to Adapt and Respond?

After reading Ed Merk’s, post about trying to be fair and balanced, and the corresponding bug, in which Scott Lewis has a valid concern, I’m wondering if the number of projects that the Foundation has under it’s umberalla is outpacing it’s ability to adapt to the changing environment. To be honest, the Friends of Eclipse program sounds like a good thing; however, as soon as you put a committe in place to determine how and when that money is going to be used, it be comes a contentious issue.

I think in general the issue at hand is more than the distribution of funds from Friends of Eclipse, but a larger issue that has been growing, the Foundation’s ability to adapt to existing project requests and needs, while also having to adapt to the slowly growing economic recovery. The fact of the matter is that as resources get constrained, money gets diverted to other projects. Non profits and membership dues are either canceled or cut back. The less funds there are, the more the Foundation itself has to manage it’s own budget. Staff cuts occur, and positions are not filled after people leave. It’s just a fundamental fact when you have to depend mostly on membership dues to fund development.

Scott’s request for money to pay some students to do releng work is interesting, however, my personal opinion is that releng should be the responsibility of all committers. There is in general a fundamental problem with releng work, nobody really wants to do it. How complicated doing releng really depends on your project, and more particularly the process and tools you use. Sometimes we need to take a step back from our desire to keep writing new features and new functionality, and just look at what we have created. Are we doing things the most efficiently? Are we doing the simpliest thing that can possibly work, or have we over engineered things? Have our processes become a big giant ball of tangled spaghetti?

I work on several smaller projects and sub-projects, and maintain both code and releng scripts for both. My experience is that unless the committers themselves take the time to learn the release process, and to manage it themselves, it will always be this big black whole. Some of the newer technologies for doing eclipse releases really do take the pain out of things. If you are doing a PDE build based system, because that is what you started the project with. Maybe it’s time to take a step back and look at the other items out there? Will a B3, Buckminster, or Tycho build prove to be something easier to maintain, with the limited resources the team has?

The other issue comes down to the fact that many of the current jobs that are running on the Hudson build system at the Foundation are not playing well with other jobs. Jobs that take 6 to 8 hours will lock up Executors for that entire time. Maybe a staged build approach would be better? Break the build into smaller stages, or maybe look at refactoring the build to reduce the time it takes. Is a full build required every time? Do you need to run every unit test every time? Can the test be broken up so that you only run the most critical ones during a Continuous Integration build, and run longer running tests on another slave?  Why do projects have to jump through multiple hoops in order to do a release to downloads.eclipse.org? All projects need to keep in mind that their build is not the only one running on the build servers.

Foundation time and resources do the best they can at responding to projects needs, but my feeling is that maybe the Foundation needs to look at what is the ultimate size and number of projects it can safely and fairly administer. Are the number of projects it has now, putting a bigger burden on the foundation than it’s current resources can support? How can we address this fundamental issue?   I don’t have an answer, but willing to chat about it over a beer at EclipseCon.