Enable Code Coverage for Integration and Unit Tests using Cobertura and Maven

On the turmeric project, we maintain a nightly dashboard.  On the dash board we collect statistics about the project, including code coverage, findbugs analysis and other metrics.   We had been using the Maven EMMA plugin to provide code coverage, but ran into a problem with EMMA.  It was causing test failures after the classes were instrumented.    So we disabled the code coverage, as we needed accurate test results during our builds.    However we still needed code coverage, more importantly we also need coverage for the existing test suite, which really an integration test suite instead of a unit test suite.

Cobertura and EMMA plugins both really are designed to work with unit tests.  So we have to work around the limitation.

  1. First we need to instrument the classes.
  2. Second we need to jar up the instrumented classes and have them used by the build later.
  3. Need to tell the Integration Tests to use the instrumented classes for it’s dependencies.
  4. Generate an XML report of the results.

I tried doing this without falling back to ant, but everytime I tried to use the maven-site-plugin and configure it to generate the reports, it would complain that cobertura:check wasn’t configured correctly.  In our case I didn’t need check to run, I just needed the reports generated.   So Ant and AntContrib to the rescue.   The following is the complete maven profile I came up with:

     <profile>
         <id>cobertura</id>
         <dependencies>
            <dependency>
               <groupId>net.sourceforge.cobertura</groupId>
               <artifactId>cobertura</artifactId>
               <optional>true</optional>
               <version>1.9.4.1</version>
            </dependency>
         </dependencies>
         <build>
            <plugins>
               <plugin>
                  <groupId>org.codehaus.mojo</groupId>
                  <artifactId>cobertura-maven-plugin</artifactId>
                  <configuration>
                     <instrumentation>
                        <excludes>
                           <exclude>org/ebayopensource/turmeric/test/**/*.class</exclude>
                           <exclude>org/ebayopensource/turmeric/common/v1/**/*.class</exclude>
                        </excludes>
                     </instrumentation>
                  </configuration>
                  <executions>
                     <execution>
                        <id>cobertura-instrument</id>
                        <phase>process-classes</phase>
                        <goals>
                           <goal>instrument</goal>
                        </goals>
                     </execution>
                  </executions>
               </plugin>
               <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-jar-plugin</artifactId>
                  <executions>
                     <execution>
                        <id>cobertura-jar</id>
                        <phase>post-integration-test</phase>
                        <goals>
                           <goal>jar</goal>
                        </goals>
                        <configuration>
                           <classifier>cobertura</classifier>
                           <classesDirectory>${basedir}/target/generated-classes/cobertura</classesDirectory>
                        </configuration>
                     </execution>
                  </executions>
               </plugin>
               <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-install-plugin</artifactId>
                  <version>2.3.1</version>
                  <executions>
                     <execution>
                        <id>cobertura-install</id>
                        <phase>install</phase>
                        <goals>
                           <goal>install</goal>
                        </goals>
                        <configuration>
                           <classifier>cobertura</classifier>
                        </configuration>
                     </execution>
                  </executions>
               </plugin>
               <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-antrun-plugin</artifactId>
                  <executions>
                     <execution>
                        <phase>verify</phase>
                        <configuration>
                           <tasks>
                              <taskdef classpathref="maven.runtime.classpath"
                                 resource="tasks.properties" />
                              <taskdef classpathref="maven.runtime.classpath"
                                 resource="net/sf/antcontrib/antcontrib.properties" />
                              <available
                                 file="${project.build.directory}/cobertura/cobertura.ser"
                                 property="ser.file.exists" />
                              <if>
                                 <equals arg1="${ser.file.exists}"
                                    arg2="true" />
                                 <then>
                                    <echo message="Executing cobertura report" />
                                    <mkdir
                                       dir="${project.build.directory}/site/cobertura" />
                                    <cobertura-report
                                       format="xml"
                                       destdir="${project.build.directory}/site/cobertura"
                                       datafile="${project.build.directory}/cobertura/cobertura.ser" />
                                 </then>
                                 <else>
                                    <echo message="No SER file found." />
                                 </else>
                              </if>
                           </tasks>
                        </configuration>
                        <goals>
                           <goal>run</goal>
                        </goals>
                     </execution>
                  </executions>
                  <dependencies>
                     <dependency>
                        <groupId>ant-contrib</groupId>
                        <artifactId>ant-contrib</artifactId>
                        <version>20020829</version>
                     </dependency>
                  </dependencies>
               </plugin>
            </plugins>
         </build>
      </profile>

Note: Do not use the cobertura:cobertura goal with this profile. It will fail the build because it will try to instrument the classes twice.

The use of Ant and AntContrib was a necessity because there is no cobertura:report goal, as it expects to run during the site generation phase. However, this causes the check goal to run as well, and we didn’t need that. So maybe, I’ll work up a patch to add a reporting goal just to run the report without having to run the site goal as well.
Hopefully, this helps some people, as I lost much hair working this out.

About these ads
This entry was posted in clean code, craftsmanship, eclipse, maven, turmeric. Bookmark the permalink.

6 Responses to Enable Code Coverage for Integration and Unit Tests using Cobertura and Maven

  1. Pingback: Dave Carver: Enable Code Coverage for Integration and Unit Tests using Cobertura and Maven

  2. brabenetz says:

    Hi,

    This post was a very important inspiration for me.
    I tried to get it work with the Maven Site Report.

    My Result was: Not to use the cobertura-maven-plugin (which want run instrument severel times…) at all.
    And run copertura ONLY with the maven-ant-plugin.

    First i removed all cobertura-maven-plugin entries.

    My Changes to the Maven-Ant-Plugin:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.3</version>
        <executions>
            <!-- Run Cobertura with ANT Run -->
            <!-- The cobertura-maven-plugin doesn't work right. -->
            <execution>
                <id>cobertura-instrument Classes</id>
                <phase>process-classes</phase>
                <configuration>
                    <tasks>
                        <taskdef classpathref="maven.runtime.classpath" resource="tasks.properties" />
                        <taskdef classpathref="maven.runtime.classpath" resource="net/sf/antcontrib/antcontrib.properties" />
    
                        <!-- Start instrument Classes -->
                        <echo message="Executing cobertura instrument Classes" />
                        <mkdir dir="${project.build.directory}/cobertura" />
                        <mkdir dir="${project.build.directory}/generated-classes/cobertura" />
                        <cobertura-instrument datafile="${project.build.directory}/cobertura/cobertura.ser" todir="${project.build.directory}/generated-classes/cobertura">
                            <fileset dir="${project.build.directory}/classes">
                                <include name="**/*.class" />
                                <!-- Maybe some excludes of generated Classes -->
                            </fileset>
                        </cobertura-instrument>
                        <!-- Copy remaining Files (e.g.: package-info.class). -->
                        <copy todir="${project.build.directory}/generated-classes/cobertura" overwrite="false">
                            <fileset dir="${project.build.directory}/classes" />
                        </copy>
                    </tasks>
                </configuration>
                <goals>
                    <goal>run</goal>
                </goals>
            </execution>
            <execution>
                <id>cobertura create report</id>
                <phase>verify</phase><!-- or phase 'pre-site' -->
                <configuration>
                    <tasks>
                        <taskdef classpathref="maven.runtime.classpath" resource="tasks.properties" />
                        <taskdef classpathref="maven.runtime.classpath" resource="net/sf/antcontrib/antcontrib.properties" />
                        <available file="${project.build.directory}/cobertura/cobertura.ser" property="ser.file.exists" />
                        <if>
                            <equals arg1="${ser.file.exists}" arg2="true" />
                            <then>
                                <echo message="Executing cobertura report" />
                                <mkdir dir="${project.build.directory}/site/cobertura" />
                                <cobertura-report format="xml" destdir="${project.build.directory}/site/coberturaXml" datafile="${project.build.directory}/cobertura/cobertura.ser" />
                                <cobertura-report format="html" destdir="${project.build.directory}/site/cobertura" datafile="${project.build.directory}/cobertura/cobertura.ser"
                                    srcdir="${project.build.directory}/cobertura/sources">
                                    <!-- e.g.: Generated JaxB Files -->
                                    <fileset dir="${project.build.directory}/generated-sources/xjc" />
                                    <!-- Default source directory -->
                                    <fileset dir="${project.build.sourceDirectory}" />
                                </cobertura-report>
                            </then>
                            <else>
                                <echo message="No SER file found." />
                            </else>
                        </if>
                    </tasks>
                </configuration>
                <goals>
                    <goal>run</goal>
                </goals>
            </execution>
        </executions>
        <dependencies>
            <dependency>
                <groupId>ant-contrib</groupId>
                <artifactId>ant-contrib</artifactId>
                <version>20020829</version>
            </dependency>
            <dependency>
                <groupId>net.sourceforge.cobertura</groupId>
                <artifactId>cobertura</artifactId>
                <version>1.9.4.1</version>
            </dependency>
        </dependencies>
    </plugin>
    

    My Changes to the Maven-Surefire-Plugin:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.9</version>
        <configuration>
            <forkMode>always</forkMode>
            <argLine>-Xms500M -Xmx500M -Xmn250M</argLine>
            <excludes>
                <exclude>**/Abstract*Test.java</exclude>
                <exclude>**/*OneTimeTest.java</exclude>
                <exclude>**/All*Tests.java</exclude>
            </excludes>
            <classesDirectory>${project.build.directory}/generated-classes/cobertura</classesDirectory>
            <systemPropertyVariables>
                <net.sourceforge.cobertura.datafile>${project.build.directory}/cobertura/cobertura.ser</net.sourceforge.cobertura.datafile>
            </systemPropertyVariables>
            <additionalClasspathElements>
            	<!-- Add cobertura to the UnitTest Classpath -->
                <additionalClasspathElement>${settings.localRepository}/net/sourceforge/cobertura/cobertura/1.9.4.1/cobertura-1.9.4.1.jar</additionalClasspathElement>
            </additionalClasspathElements>
        </configuration>
        <dependencies>
           <dependency>
              <groupId>net.sourceforge.cobertura</groupId>
              <artifactId>cobertura</artifactId>
              <version>1.9.4.1</version>
           </dependency>
        </dependencies>
    </plugin>
    

    Hopefully, this also helps some people.

    With friendly regards,
    Harald

  3. brabenetz says:

    @ Admin
    Please delete my first three posts.
    A preview- or edit- feature would be nice :)

  4. I’ve been researching a way to get the coverage of our integration tests against implemented page objects/components. From what I’ve seen Cobertura doesn’t instrument stuff in the test directory. This is the closest thing I’ve seen. I assume would work for my situation as well?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s