>Is Quality Dead for eclipse code?

>Yes the title is ment to be a bit touchy, and controversial. Quality means different things to different people, and there tolerance of the deviation from that acceptable level varies. One measurement of quality is by how consistently your projects unit tests pass or don’t pass during your builds. A short run through the build status reports of projects on the release train, shows that for those projects that publish their unit test results, that the Modeling projects consistently have good I-builds that pass all their tests. There are more than a hand full of other projects that consistently have tests that fail during I-builds. Frightenly there are more than a few that have published Milestone builds with failing unit tests. A milestone should not be declared unless all it’s tests pass.

To me personally, unit tests are your first warning sign that your quality is on shakey ground if they conistently fail. Ignoring the tests, or deleting them to just get the build green does not address the issue. A unit test fails for a reason. Maybe the test itself is bad, but not addressing it still leaves the quality of the product produced in question.

If you are on the release train…how do you determine an acceptable quality for your product? Would you want to consume something from another project that was consistently failing tests? It could affect your product and how you consume that api? What about adopters? Shouldn’t they have stable (passing) I-Builds and Milestones to consume and develop with?

Note: The way I checked and determined the status was to visit each projects download page and for those that published their I-Build statuses, reviewed the results of their test suite runs. About one third of the projects only provided binaries for download, but not status information. Having the information available about the state of the builds is an important part of working in an open manner with the community. It helps adopters determine whether they want to consume that particular build or not.

Advertisements
This entry was posted in agile, eclipse, testing. Bookmark the permalink.

5 Responses to >Is Quality Dead for eclipse code?

  1. Eike Stepper says:

    >David, I fully agree! Test status should always be published. It’s like with food in a super market: You want to know what you will get and eat 😉

  2. Mickael says:

    >I think that Hudson also agrees with you. Let’s hope that more and more projects will be fully built with it, so that it becomes easy for anyone to know the status/”quality” of projects.

  3. David Carver says:

    >Hopefully as the Athena common builder starts to spread and more projects migrate to Hudson the visibility will be better. One of the big problems from a consumers perspective is that there is no consistency in how or what is displayed in regards to the builds.Every project displays the information differently.

  4. >It is truly a bad sign if we’re consistently not passing our junits for our ibuilds. I am aghast that a milestone is declared with a project not passing tests.Has it just gotten too complex (i.e. too many ways to easily bust something) or are just not caring as much?

  5. David Carver says:

    >@Kevin McGuire: There are a multi-tude of reasons. Not caring, builds always being broken so why look, not getting feedback quick enough, no true continuous integration, lack of understanding the value of unit testing, and not following the technical practices.The thing is, that Team needs to enforce the issue. I a team member fix a bug and refactored some code around, but forgot to move the unit tests. the build failed, so I reopended the bug. Bugs shouldn’t be resolved until the code is verified to work and all unit tests pass.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s