Test Coverage with Jacoco, Gradle and Sonar

There is no shortage of tools to help Java as well as Groovy and Scala developers to write good tests and ensure code quality. Sonar, an excellent integration dashboard for software quality control, provides visualization of test coverage. In this blog I show how to integrate most performant code coverage tool, JaCoCo, into deployment pipeline.
Why Jacoco? JacoCo is the newest tool for code coverage, which provide more flexibility by providing an agent which can be attached to the Java virtual machines starting from 1.5 by using a simple command line argument. You can even measure coverage of Integration and functional testing with this tool. Here is interesting statistics about JacoCo.

Our goal will be:

  • Measure test coverage during testing phase in gradle build script
  • Confugure Sonar to accept Jacoco generated data

In order for gradle build to get Jacoc, we will use jacoco.gradle script where we provide properties for gradle projects and instrument JVM with jacoco agent during test task execution.
Gradle does not provide jacoco plugin right now, so this script is based on ant task.

import groovy.lang.Closure;

apply plugin: 'java'

def jacocoVersion = '0.6.0.201210061924'
def jacocoAgent = "org.jacoco.agent-${jacocoVersion}.jar"

configurations { jacoco }

dependencies {
  jacoco "org.jacoco:org.jacoco.agent:$jacocoVersion"
  jacoco "org.jacoco:org.jacoco.ant:$jacocoVersion"
}


def jacocoConvention = new JacocoPluginConvention(project)
project.convention.plugins.jacoco = jacocoConvention

class JacocoPluginConvention {
  def reportPath
  def coverageFileName
  def tmpDir
  def includes
  def excludes
  def exclclassloader
  def append
  def sessionid
  def dumponexit
  def output
  def address
  def port
  def iTcoverageFileName

  def jacoco(Closure close) {
	close.delegate = this
	close.run()
  }

  JacocoPluginConvention(Project project) {
	reportPath = "${project.reporting.baseDir.absolutePath}/jacoco"
	tmpDir = "${project.buildDir}/reports/jacoco"
	
	coverageFileName = "${tmpDir}/jacoco.exec"
	iTcoverageFileName = "${tmpDir}/jacocoIt.exec"
	
	
	includes = []
	excludes = []
	exclclassloader = []
	sessionid = null
	append = false
	dumponexit = true
	output = "file"
	address = null
	port = null
  }
  
  def getParams() {
	def params = [:]
	params["property"] = "agentvmparam"
	if (System.properties['test.single']=='*IT') {
		params["destfile"] = iTcoverageFileName
		
	} else {
	params["destfile"] = coverageFileName
	}
	if (includes != null && includes.size() > 0) params["includes"] = includes.join(":")
	if (excludes != null && excludes.size() > 0) params["excludes"] = excludes.join(":")
	if (exclclassloader != null && exclclassloader.size > 0) params["exclclassloader"] = exclclassloader
	if (sessionid != null) params["sessionid"] = sessionid
	params["append"] = append
	params["dumponexit"] = dumponexit
	params["output"] = output
	if (address != null) params["address"] = address
	if (port != null) params["port"] = port
	return params
  }
}

test {
	
	
  //testLogging.showStandardStreams = true
  
  doFirst {
	  
	ant.taskdef(name:'jacocoagent', classname: 'org.jacoco.ant.AgentTask', 
		classpath: configurations.jacoco.asPath)
	ant.jacocoagent(jacocoConvention.getParams())
	jvmArgs "${ant.properties.agentvmparam}"
  }
  
  doLast {
	if (!new File(jacocoConvention.coverageFileName).exists()) {
	  logger.info("Skipping Jacoco report for ${project.name}. The data file is missing. (Maybe no tests ran in this module?)")
	  logger.info("The data file was expected at ${jacocoConvention.coverageFileName}")
	  return
	}
	ant.taskdef(name:'jacocoreport', classname: 'org.jacoco.ant.ReportTask', classpath: configurations.jacoco.asPath)
	ant.mkdir dir: "${jacocoConvention.reportPath}"
	
	ant.jacocoreport {
	  executiondata {
		ant.file file: "${jacocoConvention.coverageFileName}"
	  }
	  structure(name: project.name) {
		classfiles {
		  fileset dir: "${sourceSets.main.output.classesDir}"
		}
		sourcefiles {
		  sourceSets.main.java.srcDirs.each {
			fileset(dir: it.absolutePath)
		  }
		}
	  }
	  xml  destfile: "${jacocoConvention.reportPath}/jacoco.xml"
	  html destdir: "${jacocoConvention.reportPath}"
	}
  }
}

One of the most important properties is coverageFileName = “${tmpDir}/jacoco.exec”. This is where Jacoco will writes result of the test coverage.
Lets configure gradle.build file to use jacoco.gradle.

buildscript {
	apply from: 'jacoco.gradle'
}
	

		

Now we have to execute sequence of gradle build job to get coverage result written into jacoco.exec file.
>gradle clean test
Now we should see jacoc.exec file as well as html code coverage reports in build directory.
Last step will be adding jacoco report to sonar. We need to configure sonar to accept jacoco test coverage results.
On Sonar web console, we should enable jacoco as a default code coverage tool. Login to Sonar as administrator and navigate to Configuration -> General Setting -> Code Coverage.
Make sure jacoco is default code coverage tool.

Then lets add properties to sonar task in the build script.

apply sonar

	// sonar settings -   - check new sonar plugin for milestone 5 release
		sonar {
			server {
				url = "http://mysonar:9000"
			}
			database {
		
			url="jdbc:mysql://mysonar:3306/sonar?useUnicode=true&characterEncoding=utf8"
			driverClassName="com.mysql.jdbc.Driver"
			username="$username"
			password="$password"
			}
			project {
				dynamicAnalysis  = "reuseReports"
				withProjectProperties { props ->
					props["sonar.core.codeCoveragePlugin"] = "jacoco"
					props["sonar.jacoco.itReportPath"] = "${buildDir}/reports/jacoco/jacocoIt.exec"
					props["sonar.jacoco.reportPath"] = "${buildDir}/reports/jacoco/jacoco.exec"
				}
				
				
				name='myProject'
				description='test project'
				
			}
			
		}

No we should execute sonar task as gradle build:
>gradle sonarAnalyze
and Jacoco generated code coverage appears in Sonar dashboard.


You can also add integration of functional testing to the mix and provide reports on those using Sonar and jacoco.
Details are in those posts:


To wrap up, we want to run those task automatically on Continuous Integration Servers. Creating a Jenkins job to run quality checks, including code coverage I left for you.
There is also Jenking Jacoco plugin, in case you don’t want to deal with Sonar and just have test reports directly on a build page.

Test coverage measurement tools are a fantastic addition to the unit testing paradigm. Coverage measurement lends both depth and precision to an already beneficial process. You should, however, view code coverage reports with careful conjecture. High coverage percentages alone do not ensure the quality of your code. Highly covered code isn’t necessarily free of defects, although it’s certainly less likely to contain them. but that another topic.

Resources:
Sonar is an open platform to manage code quality
jacoc-gradle on gitHub
JaCoCo ant task
Measure Coverage by Integration Tests with Sonar
Separating Integration and Unit Tests with Maven, Sonar, Failsafe, and JaCoCo

1 Comment

  1. Thanks for finally talking about > Test Coverage with Jacoco, Gradle
    and Sonar | Art of Software Engineering < Liked it!

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>